The C10K problem

It's time for web servers to handle ten thousand clients simultaneously, don't you think? After all, the web is a big place now.

And computers are big, too. You can buy a 500MHz machine with 1 gigabyte of RAM and six 100Mbit/sec Ethernet card for $3000 or so. Let's see - at 10000 clients, that's 50KHz, 100Kbytes, and 60Kbits/sec per client. It shouldn't take any more horsepower than that to take four kilobytes from the disk and send them to the network once a second for each of ten thousand clients. (That works out to $0.30 per client, by the way. Those $100/client licensing fees some operating systems charge are starting to look a little heavy!) So hardware is no longer the bottleneck.

One of the busiest ftp sites, ftp.cdrom.com, serves (as of May 1999) around 5000 clients simultaneously through a 70 megabit/second pipe. Pipes this fast aren't common yet, but technology is improving rapidly.

With that in mind, here are a few notes on how to configure operating systems and write code to support thousands of clients. The discussion centers around Unix-like operating systems, for obvious reasons.

I/O Strategies

There seem to be four ways of writing a fast web server to handle many clients:
  1. serve many clients with each server process or thread, and use select() or poll() to avoid blocking. This is the traditional favorite, and is sometimes referred to as "using nonblocking I/O".
  2. serve many clients with each server process or thread, and use asynchronous I/O to avoid blocking. This has not yet become popular, possibly because of poorly designed asynchronous I/O interfaces. Zach Brown (author of HoserFTPD) thinks this might now be the way to go for highest performance; see his 14 April 1999 post to hftpd-users.
    There are several flavors of asynchronous I/O:
  3. serve one client with each server thread, and let read() and write() block. (This is the only model supported by Java.)
  4. Build the server code into the kernel. Novell and Microsoft are both said to have done this at various times, and at least one NFS implementation does this. IBM and Sun are said to have released specweb benchmark results using this technique.

Richard Gooch has written a paper discussing these options. Interesting reading.

The Apache mailing lists have some interesting posts (one, two, three) about why they prefer not to use select() (basically, they think that makes plugins harder).
I have not yet seen any data comparing the performance of the four approaches.

Mark Russinovich wrote an editorial and an article discussing I/O strategy issues in the 2.2 Linux kernel. Worth reading, even he seems misinformed on some points. In particular, he seems to think that Linux 2.2's asyncrhonous I/O (see F_SETSIG above) doesn't notify the user process when data is ready, only when new connections arrive. This seems like a bizarre misunderstanding. See also comments on an earlier draft, a rebuttal from Mingo, Russinovich's comments of 2 May 1999, a rebuttal from Alan Cox, and various posts to linux-kernel.

Limits on open filehandles

Limits on threads

Other limits/tips

Kernel Issues

For Linux, it looks like kernel bottlenecks are being fixed constantly. See Linux HQ, Kernel Traffic, and the Linux-Kernel mailing list (Example interesting posts by a user asking how to tune, and Dean Gaudet)

In March 1999, Microsoft sponsored a benchmark comparing NT to Linux at serving large numbers of http and smb clients, in which they failed to see good results from Linux. See also my article on Mindcraft's April 1999 Benchmarks for more info.

See also The Linux Scalability Project.

Measuring Server Performance

Two tests in particular are simple, interesting, and hard:
  1. raw connections per second (how many 512 byte files per second can you serve?)
  2. total transfer rate on large files with many slow clients (how many 28.8k modem clients can simultaneously download from your server before performance goes to pot?)
Jef Poskanzer has published benchmarks comparing many web servers. See http://www.acme.com/software/thttpd/benchmarks.html for his results.

I also have a few old notes about comparing thttpd to Apache that may be of interest to beginners.

Interesting select()-based servers

Interesting thread-based servers

Other interesting servers


Copyright 1999 Dan Kegel
dank@alumni.caltech.edu
Last updated: 9 May 1999
[Return to www.kegel.com]