Making GHC’s I/O manager more scalable
December 17th, 2009 by Bryan O'Sullivan
The Glasgow Haskell Compiler supports extraordinarily cheap threads, about as lightweight as those of Erlang. These are implemented using a two-level model, with threads scheduled across a set of OS-level threads. Since the lightweight threads can't afford to block when performing I/O operations, when a Haskell program starts, it runs an I/O manager thread whose job is to notify other threads when they can safely perform I/O.
On Unix systems, the I/O manager has for a long time managed its file descriptors using the select system call. While select performs well for a small number of numerically similar file descriptors, on most platforms it has a small fixed limit on the number of file descriptors it can manage, usually 1,024. Clearly, this makes GHC extremely tricky to use for large-scale server development: if you have more than a few hundred concurrent clients, and a few hundred worker threads opening and closing files, you're quickly going to encounter runtime crashes.
This problem has long been known; Simon Marlow opened a ticket against it 4 years ago. I've got the flu this week, so when I became too bored with sitting still, I took a quick crack at an initial solution to the problem, by writing a patch that replaces the I/O manager's use of select with poll.
The patch works well, but it's only an initial stab at the problem. The internals of the I/O manager are implemented less efficiently than I would like. Efficient is relative, of course: I can still get a dirt-simple Haskell HTTP server to handle 15,000 requests per second when hit by ab -c 75 -n 75000 (i.e. 75 concurrent clients issuing 75,000 requests in total). But I'd bet that number could be somewhat higher, even on a dual-core laptop.
I would not expect even the patched I/O manager to perform especially well if handling a large number of mostly idle clients, but I need to write some benchmarking code to confirm this. My ultimate goal is to be able to get the GHC runtime into a position of being able to handle hundreds of thousands of concurrent network connections. Now that the initial intermediate step is done, that next part, of changing the internals of the I/O manager to use better data structures (perhaps even relying on libev, is likely to be a lot more work.
Hi, first of all — thanks for your work on thit ticket. This is my the most awaited feature I am missing in GHC.
Why do you consider using libev over raw calls to kqueue/epoll? As an another option, there is pure Haskell Event library by Johan Tibell.
If we had a common benchmark I could try to drop the event library in GHC base and see how it performs. It already supports select, epoll, and kqueue and has a very similar interface to the EventManager in your patch.
Johan, I’d like to see your event library in base. Maybe, we should move discussion to GHC trac?
Johan, do you have a darcs or git repo for your event library handy?
I’ll write a benchmark some time in the next few days. It’s not a very hard job: simply open some number of connections between a client and a server, and then send a byte over a random socket every so often.
The tricky part is in accounting where the time spent goes. We might want to use the new event logging mechanism in 6.12 and ThreadScope for that.
Hello,
First of all, thank you for your work. I have been waiting for this kind work for a long time.
I’m implementing a Web server
with GHC. To get over the select() limitation, I wrote a C10K server library which use the *prefork* technique. That is, N processces are preforked, and a process limits acceptable connection to M. So, N * M connections can be handled.
I would contribute the benchmark stuff since my team already
know curl-loader and jMeter well. Please tell me what you want to measure…
Keep in touch.
Bryan,
It’s at http://github.com/tibbe/event
We could use the libev benchmark for inspirations. It opens a number of pipes and sends a byte over each like you said.
Really interesting and promising stuff.
FYI: I investigated libev for an application, had a pretty good impression overall — well thought out and functional. It had a major drawback for us, though: the main loop supports various timers, and to implement that it calls gettimeofday() twice for each call to poll(). This occurs regardless of whether you’ve registered any timers or not.
We decided that was too much overhead for our app (lots of small UDP reads). It also didn’t help that libev uses a direct gettimeofday() syscall by default, instead of glibc’s faster vdso implementation (see EV_USE_CLOCK_SYSCALL).
[...] couple of weeks, I have been working with Johan Tibbell on an event library to use for replacing GHC’s existing I/O manager. The work has been progressing rather nicely: I now have both the epoll and kqueue back ends [...]
[...] Making GHC’s I/O manager more scalable – Dec 17, 2009 [...]