For instance, in this case, I could immediately tell by inspection that something bad was afoot in the inner loop of the UTF-8 decoder. A decoder more or less has to read a byte of input at a time, as this heavily edited bit of Core illustrates:

let x2 :: Word8
    x2 = case readWord8OffAddr# {- ... -} of
           (# s, x #) -> W8# x
in {- ... -}

What's important in the snippet above is that the value x2 is boxed, i.e. packaged up with a W8# constructor so that it must be accessed via a pointer indirection. Since a decoder must read up to 4 bytes to emit a single Unicode code point, the loop was potentially boxing up 4 bytes, then immediately unboxing them:

case x2 of 
  W8# x2# ->
    case x3 of
      W8# x3# ->
        case x4 of
          W8# x4# -> {- ... -}

While both boxing and unboxing are cheap in Haskell, they're certainly not free, and we surely don't want to be doing either in the inner loop of an oft-used function.

We can see why this was happening at line 96 of the decodeUtf8With function. I'd hoped that the compiler would be smart enough to unbox the values x1 through x4, but it turned out not to be.

Fixing this excessive boxing and unboxing wasn't hard at all, but it made the code uglier. The rewritten code had identical performance on pure ASCII data, but was about 1.7 times faster on data that was partly or entirely non-ASCII. Nice! Right?

Not quite content with this improvement, I tried writing a decoder based on Björn Höhrmann's work. My initial attempt looked promising; it was up to 2.5 times faster than my first improved Haskell decoder, but it fell behind on decoding ASCII, due to the extra overhead of maintaining the DFA state.

In English-speaking countries, ASCII is still the king of encodings. Even in non-English-speaking countries that use UTF-8, a whole lot of text is at least partly ASCII in nature. For instance, other European languages contain frequent extents of 7-bit-clean text. Even in languages where almost all code points need two or more bytes to be represented in UTF-8, data such as XML and HTML still contains numerous extents of ASCII text.

What would happen if we were to special-case ASCII? If we read a 32-bit chunk of data, mask it against 0x80808080, and get zero, we know that all four bytes must be ASCII, so we can just write them straight out without going through the DFA (see lines 82 through 110).

As the numbers suggest, this makes a big difference to performance! Decoding pure ASCII becomes much faster, while both HTML and XML see respectable improvements. Of course, even this approach comes with a tradeoff: we lose a little performance when decoding entirely non-ASCII text.

Even in the slowest case, we can now decode upwards of 250MB of UTF-8 text per second, while for ASCII, we exceed 1.7GB per second!

These changes have made a big difference to decoding performance across the board: it is now always between 2 and 4 times faster than before.

As a final note, I haven't released the new code quite yet - so keep an eye out!

Prior to Steele and White’s "How to print floating-point numbers accurately", implementations of printf and similar rendering functions did their best to render floating point numbers, but there was wide variation in how well they behaved. A number such as 1.3 might be rendered as 1.29999999, for instance, or if a number was put through a feedback loop of being written out and its written representation read back, each successive result could drift further and further away from the original.

Steele and White effectively solved the problem with a clever algorithm named "Dragon4" (the fourth version of the "Dragon" algorithm, which acquired its name because the authors were inspired to obscure puns by Heighway's dragon curve).

The Dragon4 algorithm spread quickly across language runtimes, such that few programmers today understand that this was ever a problem, much less how hairy it was (and is). Indeed, prior to last year, there was almost no activity in this area: two papers proposed widely used refinements to Dragon4, and that was about it. (Alas, the problem was originally solved around a decade before Steele and White published their work, but nobody noticed. If you have a clever idea and sufficient chutzpah, try to enlist Guy Steele as a coauthor. Your work will be read.)

But how solved was the problem? Dragon4 and its derivatives are complicated and tricky, and they have a hefty performance cost, since they rely on arbitrary-precision integer arithmetic to compute their results. There might be a significant performance improvement to be gained if someone could figure out how to use native machine integers instead.

In 2010, Florian Loitsch published a wonderful paper in PLDI, "Printing floating-point numbers quickly and accurately with integers", which represents the biggest step in this field in 20 years: he mostly figured out how to use machine integers to perform accurate rendering! Why do I say "mostly"? Because although Loitsch's "Grisu3" algorithm is very fast, it gives up on about 0.5% of numbers, in which case you have to fall back to Dragon4 or a derivative.

If you're a language runtime author, the Grisu algorithms are a big deal: Grisu3 is about 5 times faster than the algorithm used by printf in GNU libc, for instance. A few language implementors have already taken note: Google hired Loitsch, and the Grisu family acts as the default rendering algorithms in both the V8 and Mozilla Javascript engines (replacing David Gay's 17-year-old dtoa code). Loitsch has kindly released implementations of his Grisu algorithms as a library named double-conversion.

And of course I can't talk about performance without mentioning Haskell somewhere :-) I've taken Loitsch's library and written a Haskell interface, which I've measured to be 30 times faster than the default renderer used in the Haskell runtime libraries. This has some nice knock-on effects: my aeson JSON library is now 10 times faster at rendering big arrays of floating point numbers, for instance. I accidentally noticed in the course of that work that my Haskell text Unicode library's UTF-8 encoder wasn't as fast as it could be, so I improved its performance by about 50% along the way. Hooray for faster code!

(By the way, the punnery in algorithm naming continues: the Grisu algorithms are named for Grisù, the little dragon.)

Original article, for posterity:

A couple of days ago, I released version 0.9 of my attoparsec parsing library. Although its visible API remains completely unchanged over 0.8, I made a large and important change to the semantics. So if you use attoparsec, read on before you bump your build dependencies!

Prior to 0.9, attoparsec backtracked too aggressively, and hence held onto too much memory.

Consider the following parser:

(char 'f' *> char 'o') <|> (char 'f' *> char 'i')

Under attoparsec 0.8 and earlier, the <|> (choice) operator saves the input state before executing the left branch. If the left branch fails, it restores the input state, then executes the right branch. So if we supply an input of "fo", the left branch will succeed, and so the whole parser succeeds. But if we supply an input of "fi", the left branch will fail. The <|> operator will restore the input to "fi", and then the right branch will succeed.

This behaviour is very appealing, because it makes it easy to write readable parsers. Unfortunately, it comes with a big downside: it makes it too easy to write inefficient parsers, since <|> needs to hold onto the entire input of its left branch in case it fails.

In attoparsec 0.9, I’ve dropped that backtracking-by-default behaviour. This will require changes to some parsers that use attoparsec; I’ll describe what’s involved below. (By the way, this change makes attoparsec’s behaviour more compatible with the classic Parsec library.)

Given the parser above, here’s how it will now behave when given an input of "fi". The left branch will succeed on its first match against 'f', so the input will become "i" after 'f' has been consumed. At this point, the left branch will fail because it cannot match 'o' against the input of "i". The <|> combinator will switch to executing the right branch, but it will not reset the input. At this point, the right branch will attempt to match 'f' against an input that is still "i", and will thus fail.

This change affects not just the <|> combinator, but also its synonyms mplus and mappend. It also affects the behaviour of higher-level combinators that are constructed using <|>, such as many.

If your parser is affected by this change, there are two good courses of action.

The easier path is to use the try combinator on the left hand side of a <|> branch. This combinator causes a parser to backtrack on failure: if a parser wrapped by try fails, it will reset the input before passing the failure along. What would this look like in practice?

try (char 'f' *> char 'o') <|> (char 'f' *> char 'i')

This is certainly easy to do, and it won’t affect the performance of your code compared to attoparsec 0.8: you’ll have exactly the same backtracking and memory usage behaviours as before.

There is an alternative that doesn’t require backtracking, and which should make your parsers both faster and less memory-hungry: the classic technique of left factoring your grammar. For the toy parser above, this is easy to do:

char 'f' *> (char 'o' <|> char 'i')

It’s clear that we’ll see a small performance boost here because we only match against 'f' once in this parser, while our first version did some unnecessary extra work by matching 'f' on each side of the branch. Left factoring is not always this easy, and it can make parsers more difficult to read, so sometimes using try is the better course to pursue.

Given that this change in semantics introduces some awkwardness, you might reasonably wonder why I made it. By making the use of backtracking explicit, parsers that don’t need backtracking will become faster “for free”. A realistic instance of this is my aeson JSON library, where by using the no-backtracking version of attoparsec, I saw improvements of up to 10% in parsing performance, and slightly larger reductions in memory footprint.

In September, I’ll be teaching a Haskell workshop at the Strange Loop Conference in St Louis. Here’s the abstract:

Modern programming presents a daunting array of challenges: proliferating technologies, messy inputs, unreliable networks, huge volumes of data, how to verify that results are correct, making it all fast enough. The Haskell programming language is well suited to addressing this broad range of needs, as it uniquely combines conciseness, safety, and high performance.

In this fast-moving, interactive tutorial we will learn Haskell by developing a realistic analytic application: we’ll crawl a web site and rank its pages in order of authority. This combines some very modern concerns: network programming; handling dodgy HTML; big data; and number crunching. With our emphasis on getting real work done, we’ll show off some of Haskell’s compelling features and demonstrate how they help us to develop dependable, easy to understand code.

There’s a ton of other great speakers lined up for this year’s Strange Loop, so if you like learning from, and talking with, people who are excited about software, this will be a great conference to attend.

As if that weren’t enough, I’m excited to be working with David Mazières on a new class at Stanford University, "CS240H: Functional Systems in Haskell".

As far as I know, this will be the first academic course of its kind, where we focus on the use of functional programming techniques to build solid, fast, secure systems software. Intrinsic merits of the class aside, it’s going to be a lot of fun to teach this class this in the educational cradle of Silicon Valley, where systems software is the bread and butter of a lot of research and commercial innovation.

We’ll start off with some familiar topics: the basics of Haskell, laziness, monads, parsers, and all that. But from there, we’ll integrate testing, reliability, and debugging; performance tuning; interfacing to native code; concurrency and I/O paradigms; language extensions; meta-programming; and applications to the web and security.

We’re also looking at bringing in some prominent guest lecturers from the Haskell community to talk about topics of interest to them. I’ll name names when we have them confirmed.

David and I are both very much motivated by writing code that’s clean and fast, so as much as we’ll spend time talking about theory, research, and academics, this will be a very hands-on "let’s bang out some awesome software!" kind of affair.

Here’s a rough outline of what the course will involve:

• A couple of "Haskell basics" classes.

• Laziness: its applications and pitfalls.

• Monads and parsers; programming in continuation-passing style.

• Tricks for debugging and performance tuning.

• Reliability: design for robustness; property-based testing with QuickCheck; code coverage.

• The Iteratee/Enumerator paradigm for I/O.

• How to use the foreign function interface, build interfaces to native code, deal efficiently with data in various contexts, and generally think about memory. Possibly also coupled with discussion of the Haskell networking API.

• Concurrency and parallelism: event- vs thread-based concurrency, Software Transactional Memory, parallel Haskell, data parallel Haskell, repa.

• Library-level optimization: function specialization, dictionary passing, RULES pragmas, stream fusion.

• All the gnarly things you can do with functional dependencies and type families, as well as other relevant extensions such as GADTs.

• How Haskell is implemented, to give students some idea of what is actually going on.

• Generic programming: SYB and fun with Data.

• Web-related topics? formlets, plus maybe some cool Continuation type tricks. Haskell database interfaces and interesting things you can do with types, HList/HaskellDB.

• Language-level information flow control.

• Case studies of real systems in Haskell.

I know that some SF Bay Area locals have questions about the possibility of auditing the class, whether materials will be available online, and the like. I don’t have answers yet, but stay tuned.

And the performance news is good: on real-world data, I’ve improved encoding performance by about a factor of 4. Why don’t we let the graphs do the talking.

Enjoy!

About a week ago, I received a message from Finlay Thompson asking about my Haskell statistics package: he wanted to know how to generate pseudo-random variables using it. I redirected him from that to my mwc-random package, where my pseudo-random number generation code lives.

The mwc-random package currently provides generators for two widely used distributions: uniform and normal. When I was originally writing it, I paid particular attention to making it high quality, fast, and easy to use.

"High quality" sounds a little nebulous, but in the world of pseudo-random number generation, it's actually pretty well defined: a good PRNG should have a large period (the number of samples you need to pull out of it before it repeats itself, assuming a good seed), and the numbers it generates should withstand stringent tests of apparent independence (simply put, given one datum, you shouldn't be able to predict the next).

One algorithm that satisfies these criteria of quality is George Marsaglia's multiply-with-carry algorithm MWC256 (also known as MWC8222). It has a period of about 2⁸²²² (huge enough for all conceivable practical purposes), and stands up well to the "testu01", "diehard" and "big crush" statistical tests.

Due to its simplicity, MWC256 is also very fast, and under appropriate circumstances (e.g. on a 64-bit machine) it can be even faster than the well known Mersenne Twister algorithm (which also fails some statistical tests that MWC256 passes).

The Mersenne Twister is itself available for Haskellers to use, in the form of the mersenne-random package. This package is a wrapper around the Mersenne Twister library, and unfortunately it imposes on its users the underlying library's typically horrible constraints borne of too much Fortran programming: you can only have one PRNG per application, and it can only be used from a single thread! The mwc-random package is less restrictive: fire up as many PRNGs in different threads as you like, and they'll all operate independently. You can also use the PRNGs in either the ST or the IO monad, for further convenience.

When generating normally distributed random variables, the mwc-random package uses an algorithm known as the "modified ziggurat". One of the more popular algorithms for generating normally distributed variables is called the ziggurat, but its popularity belies an ill-understood quality problem: the numbers it generates aren't independent enough! It turns out that they are noticeably correlated. The modified ziggurat is almost as fast, and it sacrifices a little speed in the name of improved independence.

The base-level performance of the random number generators looks like this on my Mac using 32-bit GHC 6.12.3, where the time quoted is to generate a single double-precision floating point number:

• Uniform: 142.6 nanoseconds

• Normal: 15149 nanoseconds

Where does the question of being "done" or not come in? Well, while poking around tonight, I was a little surprised at the large difference in speed between the uniform and normal PRNGs, so I investigated. The ziggurat algorithm gets its name from the precomputed lookup tables it uses to gain its speed. It turns out that GHC's inliner was being too aggressive with the table-related code, causing the ziggurat tables to be regenerated over and over instead of precomputed just once. Ouch!

One small and very quick change, and the performance of the PRNG for normally distributed variables changed dramatically:

• Before: 15149 nanoseconds

• After: 246.8 nanoseconds

That's a little over 61 times faster. Not bad for a couple of lines of changes!

As a final note, now that GHC can build 64-bit programs on a Mac, you might wonder how it performs. Here's a comparison between 32-bit and 64-bit versions of GHC 7.0.2 (times in nanoseconds):

Those are some pretty nice performance improvements! Of course, not all applications come in the form of nice tight numeric kernels, so don't take it as given that you'll see improvements like this in your code.

As a user of it, you can't tell that my attoparsec parsing library is implemented almost entirely using explicit continuations. Every combinator accepts two continuations:

• the failure continuation is invoked if the current function fails

• the success continuation is invoked if the current function succeeds

Making matters more complex, each continuation accepts three other parameters:

• the input currently known to be available

• any additional input that was received when parsing was suspended due to insufficient input (to support backtracking in the case of a failed parse)

• an end-of-input marker, to record whether our caller has no more input to feed us incrementally

I originally implemented attoparsec as a very traditional state transformer monad. Every bind action would check to see if the previous action succeeded. If yes, it would invoke the next action, otherwise it would pass the failure forwards. While this worked, its performance was disappointing: my parsers didn't run any faster than those built on the venerable parsec package!

The work of switching from a traditional state transformer over to CPS took just a couple of days. I've had my bacon repeatedly saved by Haskell's type system, which ensures that I'm not passing the wrong kind of continuation in the wrong slot, or mixing up input-I'm-going-to-consume with input-I've-been-fed.

Switching over to CPS bought a factor-of-eight performance improvement, which delighted me. The code remains fairly easy to follow, because I was careful to write it cleanly (of all the places where excessive cleverness can bite you, working with CPS must come close to the top).

Looking at the Core generated by GHC, though, suggests that there's plenty of room for improvement: it allocates tons of closures. And sure enough, they show up at runtime.

See those items marked THUNK in the chart above? Yeah, ouch.

Nevertheless, that 8x performance improvement is real, but I actually managed to further improve on it for the aeson JSON library. In that library, some careful profiling indicated that dealing with text in general, and Unicode escapes in particular, was a serious bottleneck. Having run out of obvious paths to speed the code up in its existing form, I wrote a tiny and highly focused module, Data.Attoparsec.Zepto, that foregoes CPS in favour of the state transformer approach. For non-recursive parsers that shouldn't fail (e.g. dealing with escaped text), this module performs very well: I achieved a 35% performance boost in JSON parsing when I introduced it! That approach really only seems to work well in that single isolated instance, unfortunately: state transformers continue to kill performance if I try to use them more widely within attoparsec or aeson.

In the short to medium term, then, CPS is usually a win, but we can do better: I'm hoping to help the Simons ferret out whatever's causing continuations to be compiled into closures instead of straightline code. GHC 7 currently contains some pretty big performance regressions for CPS-heavy code, but they've been partly ameliorated by a recent patch (regression dropped from 70% to 22%). I think we've plenty of scope to make performance substantially better just via compiler improvements.

The major advance in the new version of aeson is a considerable speed improvement.

The datasets I’m using are Twitter search results, from the Twitter JSON search API. For mostly-English results, 0.2.0.0 is up to 30% faster than before, while on Japanese data (which makes heavy use of Unicode escapes), I’ve bumped performance by more than 50%.

To see how well aeson performs compared to JSON parsers for other languages, I compared it against the json module in Python 2.7. That module’s JSON parser is written in C, so it’s very fast indeed, and the amount of actual Python being executed in my microbenchmark is tiny. How do we fare?

On mostly-English data, aeson is actually faster than Python’s native-code json parser. Nice! And on Japanese data, we’re a little slower, but still very competitive.

What if you’ve been using the Haskell json package, which was the first open source Haskell JSON parser to be published? Well, I do think that aeson is easier to use, but it’s also 3x faster than the json package:

The new version of aeson introduces some other useful improvements.

• There’s a new Generic module, which lets you convert almost any instance of the Data typeclass to and from JSON without writing boilerplate code. (Be warned: generics are slow. If performance is important to you, write that boilerplate!)
• We introduce a Number type that represents integers to full accuracy, and which handles floating point numbers efficiently.
• Instead of parsing via the Applicative typeclass, we now use a custom parsing monad, improving both ease of use and performance.

Since I’ve spent the past two decades in the world of open source, of course I have goodies to show off:

• A fast, powerful library for working with the Riak decentralized data store (mailrank/riak-haskell-client on github). I think it’s the best Riak client library available for any programming language
• An efficient, easy to use JSON library for Haskell (mailrank/aeson on github). Twice as fast as the competition.

> pack "\55296"
"\9216"

Hmm, that doesn't look right. What happens if we try again?

> pack "\55296"
"\74488"

Yikes! Run the same function twice, and get not just wrong answers, but different wrong answers?

What is the pack function actually supposed to do, anyway?

> :t pack
pack :: String -> Text

It takes a traditional Haskell String value and convert it into a tightly-packed, optimized Text value. Unfortunately, Unicode code points in the range 0xd800 through 0xdfff are reserved, but the Haskell interpreter isn't rejecting '\55296', which happens to be 0xd800. The pack function is thus passed garbage, and responds with garbage itself.

That's a little inconvenient. I think that the right thing to do is to have pack check its input and reject it with an exception if it hits any reserved code points.

Update: Mark Lentczner has the following comments on how to deal with this:

1. error (ick — though catchable with IO exceptions)
2. reject the whole input to pack and just return an empty string
3. replace the offending code point with U+FFFD REPLACEMENT CHARACTER
4. just drop the offending code point on the floor silently

Mark says "I vote #3, but I suppose there is argument for the others."