criterion 1.0

Almost five years after I initially released criterion, I'm delighted to announce a major release with a large number of appealing new features.

As always, you can install the latest goodness using cabal install criterion, or fetch the source from github.

Please let me know if you find criterion useful!

New documentation

I built both a home page and a thorough tutorial for criterion. I've also extended the inline documentation and added a number of new examples.

All of the documentation lives in the github repo, so if you'd like to see something improved, please send a bug report or pull request.

New execution engine

Criterion's model of execution has evolved, becoming vastly more reliable and accurate. It can now measure events that take just a few hundred picoseconds.

benchmarking return ()
time                 512.9 ps   (512.8 ps .. 513.1 ps)

While almost all of the core types have changed, criterion should remain API-compatible with the vast majority of your benchmarking code.

New metrics

In addition to wall-clock time, criterion can now measure and regress on the following metrics:

  • CPU time
  • CPU cycles
  • bytes allocated
  • number of garbage collections
  • number of bytes copied during GC
  • wall-clock time spent in mutator threads
  • CPU time spent running mutator threads
  • wall-clock time spent doing GC
  • CPU time spent doing GC

Linear regression

Criterion now supports linear regression of a number of metrics.

Here's a regression conducted using --regress cycles:iters:

cycles:              1.000 R²   (1.000 R² .. 1.000 R²)
  iters              47.718     (47.657 .. 47.805)

The first line of the output is the R² goodness-of-fit measure for this regression, and the second is the number of CPU cycles (measured using the rdtsc instruction) to execute the operation in question (integer division).

This next regression uses --regress allocated:iters to measure the number of bytes allocated while constructing an IntMap of 40,000 values.

allocated:           1.000 R²   (1.000 R² .. 1.000 R²)
  iters              4.382e7    (4.379e7 .. 4.384e7)

(That's a little under 42 megabytes.)

New outputs

While its support for active HTML has improved, criterion can also now output JSON and JUnit XML files.

New internals

Criterion has received its first spring cleaning, and is much easier to understand as a result.

Acknowledgments

I was inspired into some of this work by the efforts of the authors of the OCaml Core_bench package.

Posted in haskell, open source
26 comments on “criterion 1.0
  1. Thanks for the sharing. The Criterion is very useful to me

  2. inz executive condo brickland says:

    Great posting. I am searching for this

  3. cck ec says:

    Thankful for this page. This is my year end topic for school project

  4. It is a parsing combinator library that is “aimed particularly at dealing efficiently with network protocols and complicated text/binary

  5. Anam says:

    Thanks for the sharing.

  6. Julia says:

    Thankful for this page. This is my year end topic for school project

  7. Angel17 says:

    I am thankful to this site for posting such a wonderful and informative blog. Keep posting. accessories and more

  8. Kelly says:

    So if you’d like to see something improved, please send a bug report or pull request.

    Kelly | ceiling patterns

  9. Jonel says:

    handyman near me is looking forward to exploring the latest goodness and continued excellence in benchmarking with Criterion! 👏🚀

  10. Kath says:

    Criterion’s model of execution has evolved, becoming vastly more reliable and accurate for Pasadena Drywall Contractors daily activities.

  11. hejedi98 says:

    Kudos on the spring cleaning too! Looking forward to exploring the latest goodness. – Bridgeport

  12. Dipova25 says:

    Laredo peeling wallpaper is looking forward to exploring the latest goodness. Great work!

  13. myWarBible says:

    Hey there! I’ve got some exciting news for you. You can now easily get your hands on the latest and greatest by installing criterion through cabal, or even grabbing the source from Git Hub. How cool is that?

  14. Looking forward to exploring the latest goodness and continuing to find Criterion useful in my projects.

  15. More power to your site!

  16. Great site to visit. Thanks for sharing.

  17. ct drywall says:

    Looking forward to exploring its capabilities further.

  18. Fix walls says:

    I’m looking forward to investigating the latest awesomeness and continue to utilize Criterion in my projects.

  19. Nice post! It’s very informative content you shared.

  20. Mindy says:

    Criterion’s model of execution has evolved, becoming vastly more reliable and accurate. It can now measure what is purple drywall and events that take just a few hundred picoseconds.

  21. James says:

    Thanks for the info you shared here.Fencing near me

  22. Linda says:

    I can’t wait to learn more about criterion once i finish my ceiling texture drywall project. Thank you so much!

  23. McAllen says:

    Congratulations on the major release of Criterion 1.0!

  24. Colet says:

    The first line of the output is the R² goodness-of-fit measure for this regression, and the second is the number of CPU cycles (measured using the rdtsc instruction) in Wilmington Landscaping
    to execute the operation in question (integer division).

  25. Greg says:

    Thanks for sharing here some interesting content. Memphis Artificial Grass Experts Memphis TN

  26. Janice says:

    It’s nice seeing this great content here. concrete driveways

Leave a Reply

Your email address will not be published. Required fields are marked *

*