You'll understand why this obscure setting is #1 on the list when you consider the benchmarks of drupal_load below (If you're not using APC, or have no idea what APC is I suggest you keep that fact a secret, and install it now):

Without APC: 286msAPC with 32m shm_size: 700ms (default APC setting)APC with 64m shm_size: 37ms!!!

Basically, 32m is not enough memory for the majority of drupal installs, and APC actually slows down a site when it is full. Typically, 64M is a sufficent value though I've seen some sites need as much as 128M. This setting is changed in apc.ini (or php.ini depending on your setup). You can find the location of the files by doing a find in the phpinfo() page for ".ini". You'll need to restart apache, of course, for the change to take effect.

There's some good further discussion of this and other APC settings at 2bits.com.

Views render cache uses the result set of a query as its cache key. In other words, if you publish an article, all cached views that the article appears in will magically update and no one will ask you why their article isn't showing up where they think it should.

The one limitation of this technique is that if an editor were to change the title of an article that appeared in the view, the change wouldn't appear until the render cache expired. This is usually acceptable in most situations given that this trick doesn't shave, as much as brutally hack off a huge chuck time spent loading the page. Also, if your view's query is terrible, this obviously won't help. However, that's rarely the case: in almost all cases, over 95% of the time spent generating the view will be the rendering, not the querying.

Many, including myself, believe this trick should actually be the default setting for all views.

The bad news is the default cache settings for blocks are stupid (usually NO_CACHE, actually). The good news is you're smart, and can alter them using the block cache alter module. Go ahead: take a look at your site. Are there elements that are the same on every page and don't change (perhaps a mega footer or mega drop down menu?)? Then take 15 seconds and set that cache to global. Such a change will easily shave 0.1 seconds of time off of every page load. Start looking at which blocks only need to update every hour (a most commented / most popular / most emailed quicktabs block for example). There goes another 0.1 second of load time.

If you think shaving 0.1 seconds is an insignificant improvement, then I implore you: raise your standards of how fast a drupal site should be. The impact of these "small" tweaks have a tendency to add up quickly. 

I see this one less often, but it's impact is severe enough that it deserves mention. It's not a performance problem, as much as it is your site crashing into a mountain and burning in flames.  Fortunately, there is a very quick way to scan for, and fix this problem. Expect to shave at least 2-5 (maybe more) seconds of page load time if you fix this issue.