In my blog post Running Lighttpd as a static file server for Drupal 6 on a Debian GNU/Linux server I describe how I set up a static file server here on xdeb.org. I mentioned there that I have not done any benchmarking on the improvements.
I found Robert Douglass article Using Lighttpd as a static file server for Drupal and thought it would be interesting to try out. Check also out the Lighttpd home page (pron. lighty). There are two main reasons for using a separate server for static files.