teideal glic deisbhéalach » science

A new pseudo-random number generator for Haskell

Bryan O'Sullivan — Sat, 19 Sep 2009 22:13:17 +0000

I just released version 0.3.3 of the Haskell statistics library, which contains a very fast pseudo-random number generator.

The generator is an implementation of George Marsaglia’s MWC256 multiply-with-carry PRNG, which has a period of 2⁸²²² (for this reason, it’s sometimes referred to as MWC8222). It produces high-quality uniformly distributed pseudo-random numbers extremely quickly.

Here is a brief performance comparison between the statistics library, the mersenne-random library, the normal Haskell PRNG, and a C implementation of MWC256. The numbers represent millions of variates generated and summed per second, so higher is better. (Measured on a 64-bit Core2 Duo laptop running Fedora 11.)

Package	Int64	Double	Word32
System.Random	0.7	1.6	n/a
mersenne-random	101.2	53.8	96.3
statistics	145.7	116.5	251.6
C (normal)	131.0	103.2	292.9
C (inlined)	162.0	118.7	375.6
C (inlined, unrolled)	186.5	171.7	571.2

As the numbers indicate, the MWC256 implementation in the statistics library is up to 3 times faster than the mersenne-random library, and often faster than the C code you’d expect to encounter in practice. And you shouldn’t use the standard Haskell PRNG if you care about performance.

(The last two rows in the table above indicate what you could expect from an aggressive C compiler that performed cross-module inlining and loop unrolling, such as almost works reliably in gcc these days. If you were using normal compilation flags and source file structure, you would be unlikely to see those kinds of numbers in practice.)

A nice feature of the PRNG in the statistics library is that it has a cleaner interface than mersenne-random. Due to the cheesy implementation of the underlying C code, the mersenne-random library enforces the use of a single PRNG for the entire program, which makes it onerous to work with. You have to create a single generator when your program starts, and pass it everywhere that you might eventually need random numbers. In a multi-threaded program, only one thread at a time can use the PRNG.

The PRNG in the statistics library runs inside the ST monad, so you can generate random variates from inside pure Haskell code. You can create or use a generator anywhere without worrying about threading issues, and there’s a handy function provided that lets you seed a generator from your system’s random number source.

As a final tweak, the new PRNG generates floating point numbers in the range (0,1], so the variates it generates are safe for using in statistical computations that are zero-phobic, such as those involving logarithms or division. I’ve also included a generator for normally-distributed floating point numbers that uses Doornik’s modified ziggurat algorithm, so it’s both fast and trustworthy.

Lazy functional yak shaving in Haskell

Bryan O'Sullivan — Sun, 13 Sep 2009 01:01:08 +0000

A few weeks ago, I decided that I'd like to focus for a while on getting a 1.0 release of the Haskell text library ready. That work has gone fairly well so far. I've focused on making sure that I like the API, and that a few key functions have good performance characteristics.

One such function is substring search, which is the underpinning for a number of useful functions (split a string on a delimiter, find all substrings, etc). My initial cut at that function had an obvious API, but ran in O(nm) time, which complexity measure I found mildly embarrassing. So I looked around for an alternative that would be faster, but still simple to implement, and settled upon Fredrik Lundh's elegant and trim adaptation of Boyer-Moore-Sunday-Horspool, which has O(n+m) complexity in the typical case.

Using QuickCheck, I was quickly able to satisfy myself that my new code was correct, so now I was faced with demonstrating to myself that the code was fast, too. I looked on Hackage for a good benchmarking package, but although there were several benchmarking packages available, they were all extremely simple, and I wasn't happy with any of them.

I have sort of a mutt's pedigree in computing, having spent several stints working in the high performance scientific computing world, so I felt antsy about measuring performance reliably. Over the course of a few days of hacking, I came up with a good framework that I am currently finishing off.

One of the things a good benchmarking framework needs is some statistical heavy lifting, to do things like autocorrelation analysis, kernel density function estimation, and bootstrapping, so I had to write some complicated statistical code along the way. I've now packaged that code up and released it as the statistics library on Hackage. Some of the features implemented in this first release of the statistics library include:

Support for common discrete and continuous probability distributions (binomial, gamma, exponential, geometric, hypergeometric, normal, and Poisson)
Kernel density estimation
Autocorrelation analysis
Functions over sample data
Quantile estimation
Resampling techniques: jackknife and bootstrap estimation

The statistics library certainly isn't yet comprehensive, but it has some features that I think make it very attractive as a base for further work:

It's very fast, building on some of the fantastic software that's available on Hackage these days. I make heavy use of Don Stewart's uvector library (itself a port of Roman Leshchinskiy's vector library), which means that many functions allocate no memory and execute tight loops using only machine registers. I use Dan Doel's uvector-algorithms library to perform fast partial sorts. I also use Don's mersenne-random library for fast random number generation when doing bootstrap analysis.
I've put a fair amount of effort into finding and using algorithms that are numerically stable (trying to avoid problems like catastrophic cancellation). Whenever possible, I indicate which methods are used in the documentation. (For more information on numerical stability, see What Every Scientist Should Know About Floating-Point Arithmetic).

For me, one of the killer features of Haskell for statistical computation is its combination of terseness and fabulous libraries. In particular, the modern collection libraries like uvector and text, which are built on stream fusion frameworks, provide a combination of expressiveness and high performance that is simply delightful to work with.

Speaking of improving the statistics library, I've already received a few patches to the statistics library even before announcing it anywhere, and I'm very much looking forward to receiving more. If you want to contribute, go get the source code and hack away:

darcs get http://darcs.serpentine.com/statistics

So now that the statistics library is in good enough shape, I can return to work on the benchmarking framework. Once that's done, I'll get back to finishing off the text library. It's been funny to see how rewriting 50 lines of code has spawned off weeks of enjoyable work on other projects! I hope that people will find all of this work useful.

By the way, the philosophy underlying the text, statistics, and benchmarking libraries is uniform across the lot of them: build and release something thatâ€”even though initially incompleteâ€”is thorough enough and of such high quality that other people will be drawn to improving your work instead of creating half a dozen tiny alternatives. So please, join in, and let the patches fly!

BLDGBLOG interviews Kim Stanley Robinson

Bryan O'Sullivan — Fri, 21 Dec 2007 03:59:59 +0000

Here is an absolute treat: a long, lively interview with Kim Stanley Robinson, conducted on one of my favourite blogs, BLDGBLOG.

At its best (the Three Californias trilogy, Antarctica), Robinson’s writing is at once haunting and beautifully evocative of a sense of place.

The obsessionally perfect news story

Bryan O'Sullivan — Wed, 13 Dec 2006 04:35:39 +0000

Local scientists, ancient reptiles, volcanic eruptions, and Antarctica! All in one story! Really, this article was written precisely and exactly for me. In brief, a paleontologist from Berkeley (across the Bay from me) was involved in a spectacular find: an almost complete skeleton (cartilage and all) of a juvenile plesiosaur, buried in 70 million year old volcanic ash. The team found it on their second trip to Vega Island, off the coast of the Antarctic Peninsula. Strangely, the coverage in the Valley’s local paper, the Mercury News (linked above), is of better quality than the corresponding news article in Nature. That doesn’t happen often.

Book review: Doug Macdougall, “Frozen Earth”

Bryan O'Sullivan — Wed, 06 Dec 2006 18:00:52 +0000

Some time ago, I read a Nature review (subscription required) of Doug Macdougall’s “Frozen Earth“. As is the way of such things, after I ordered my copy, the book suffered several months of neglect before I finally had a chance to pick it up. However, once I started reading it, I quickly became engrossed, and polished it off in a matter of days. The book opens with a few chapters on the history of scientific inquiry into climate change. This I consider its only weak part: it follows a formula of the past decade in popular science writing, of sketching the characters involved in the early study of the field. Many of the founders of the field that Macdougall introduces, and the controversies in which they were involved, merit multiple volumes of their own, and so the first few chapters are necessarily skeletal. I found these passages interesting, but not satisfying. The world probably has enough thumbnail sketches of Louis Agassiz at this point. Once Macdougall leaves behind the early historical narrative, the book kicks into intellectual high gear, and I found the entire rest of the book to be superb. In discussing the Missoula Floods, Macdougall touches on both the evidence for catastrophic flooding and the lengthy controversy that Harlen Bretz’s identification of the source of the Channeled Scablands engendered in the American geological community. This chapter is notable for its discussion of both the astounding physical evidence involved, the research that Bretz and subsequent generations of scientists performed, and how scientific controversy works. Another notable feature of the book is Macdougall’s discussion of the effects of climate on the development of the human species and human culture, both during early human evolution and through the written records of the past few millenia. Among the surprising topics he touches on is a study of paintings from the Little Ice Age (roughly the 13th through 19th centuries) in which someone noted light levels and counted the number of times in which the sky was represented as clear or overcast, as a proxy for weather conditions of the time. I appreciated Macdougall’s discussion of the drawing together of data from a variety of sources to try to portray a consistent and continuous picture of climate over time. Since so many of the signals upon which paleoclimatologists rely are either faint, broken up, or distorted, they look for correlations between as many sources of data as possible, and correct for numerous possible errors as they go. Macdougall discusses many of the proxies used, such as isotopic stratigraphy and measurements of many properties of ice cores; why some of the data are unreliable; and how climate scientists identify and correct errors. Most importantly, Macdougall writes clearly and engagingly throughout the book. He doesn’t shy away from complex topics, but he presents them clearly; he limits his use of jargon, and he remembers to introduce a new term by telling the reader what it means. Once I got over my dubious reaction to the first few chapters, I found the bulk of “Frozen Earth” to be intellectually exhilarating and thoroughly enjoyable.