Site speed is the amount of time a site takes to deliver the contents of a web page to the user after the user types (and enters) or clicks a link.

This is a very important measure because it is has a direct impact on the user.  Indeed, if a site resoponds too slowly, the user may become frustrated and leave the site in favor of another site before the page has fully loaded. Not only does site speed improve the user experience, but there is evidence to show that the number of page views increases as the speed of the site reduces.

There are a number of on-page and 'behind the scenes' factors that affect the page-load (site) speed.  These include:

1. Hosting environment (hardware, bandwidth at the server etc.)
2. Amount of content on the page
3. Number of http (or https) calls needed to load the page
4. The number and file size of images displayed on the page
5. Size and number of script libraries loaded
6. Web traffic at the time of the test

... and more

Google announced in April 2010 that Site Speed is a part of the google ranking algorithm.

Additional Site Speed information on Google Webmaster Central Blog.