If your pages load faster or slower than that of other sites it will make a difference to your page ranking. If its too slow its possible that it might not be indexed at all.
As of April 2010, Google includes Page Load Time in its algorithm. Speed does not carry as much weight as other factors, but if your page takes too long to load it will make a difference to your rank and stands a good chance of not getting into the search results at all.
Speed
Here is a very good video that discusses how the speed of your site effects your business, amazingly only a 400 millisecond slower can result in a drop off in business according to tests done by Yahoo and Google as discussed by Steve Souders in the aforementioned video. Souders also states that 90% of the time it is javaScript that is slowing the load time.
rnk
Server
The server can also be the problem. The more requests to the server the longer it will take for the page to load. Many sites are on shared servers which increase the amount of requests. You can upgrade to a dedicated server which are much more expensive, but is probably worth the cost as the alternative is lose of business
Wait Time
How long is the user willing to wait. The general consensus is:
2 to 5 seconds - Very Good
5.5 seconds to 8.0 - Still ok
More than 8.0 seconds - They are starting to notice
10 seconds - they have probably already left
To increase speed
place all CSS and javaScript in external files
place links to the javaScript in the footer
write good code that conforms to standards
Use CSS Sprites to upload multiple images
Avoid resizing images in the browser
Optimize images
Place any videos on a Video hosting site such as YouTube and Vimeo and link directly to them
W3c Validation
W3C validation alone does not play a role in page ranking but will show all of your code errors. If there are too many errors it could slow down the pace in which the spider indexes your pages and this will make a difference
Bounce Rate
Keep the length of your pages to a manageable size both for the user and seo. A fast loading page can also lower your bounce rate. The bounce rate applies to how quickly the user leaves your site. Its based on a single page visit or visits in which the user left your site after they reached the landing page.
To test the speed of your page load-time, go to: www.webpagetes.org, inset the url of the page you want to check and you will get a variety of information on the speed of the page's load-time.