WebSite

• The Web Site project really is not a project at all since Visual Studio 2005 solution and project files are not created.
• All pages and code are stored in a directory, much like the traditional ASP model. When it comes to deployment, the easiest approach is to copy all of your files to your Web server and everything compiles on demand
• You could also use the aspnet_compiler.exe utility to precompile your site into a binary file and deploy to your Web site.

Web Application

• consists of a solution (.sln) and project files (.vbproj).
• It provides the benefit of generating a single assembly in the local /bin folder ready for deployment.
• This model also makes it easier to incrementally deploy changes made to your site.

My Source: http://msdn.microsoft.com/en-us/library/aa730880(VS.80).aspx#wapp_topic5