There's a fairly painless way around this. Firstly, we'll need to use the filemap option to make hg convert strip out the files we want to lose from our new repository. Here's an idea of how that should work. To start off, we create a demo repository, named hg-100.

$ hg clone -r 100 http://www.selenic.com/repo/hg hg-100
$ ls -F hg-100
comparison.txt	hgweb.py     mercurial/  PKG-INFO  setup.py  tkmerge
hg		MANIFEST.in  notes.txt	 README    tests/

Next, we set up a filemap that eliminates all tests from the tree.

$ echo exclude tests > no-tests.map
$ hg convert --quiet --filemap no-tests.map hg-100 nukeme
$ hg --cwd nukeme update --quiet
$ ls -F nukeme
comparison.txt	hgweb.py     mercurial/  PKG-INFO  setup.py
hg		MANIFEST.in  notes.txt	 README    tkmerge

Now that we have run hg convert, it's changed the changeset ID of every commit that either contains a change to a file in tests or is a child of such a commit. Ouch, right? Haven't we now lost the ability to merge contaminated repositories with the scrubbed one?

Fortunately, no. We could always pull future changes into hg-100, then rerun hg convert with the same --filemap option, and successfully scrub later changes, which we could then share with our collaborators.

The only trouble with this is that it relies on a non-version-controlled file named .hg/shamap in our nukeme tree. That file contains the mapping from source changeset ID to target changeset ID. This is what hg convert uses to tell whether a changeset has been seen before, and if so, what its new ID is. So if we lose the nukeme tree, we can't run hg convert again, and we're stuck, right? Well, once again, not necessarily.

A final trick up our sleeves is to run every instance of hg convert with --config convert.hg.saverev=True. This is documented in the output of hg help, but sadly not on the wiki page. This saves each source changeset ID in a special field in the corresponding changeset in the target repo.

$ rm -rf nukeme
$ hg convert --quiet --filemap no-tests.map --config convert.hg.saverev=True \
    hg-100 nukeme
$ hg --cwd nukeme tip --debug | grep 'extra:'
extra:       branch=default
extra:       convert_revision=526722d24ee5b3b860d4060e008219e083488356

The convert_revision data gives us enough to create a new .hg/shamap file whenever we need to. First, we create a style file to extract the necessary data from Mercurial.

$ cat >> pants.map << EOF
changeset = '{extras}\n'
extra = '{key} {value} {node}\n'
EOF

Then, in any clone of our converted repo, it becomes simple to regenerate the .hg/shamap file:

$ hg log -r0: --style pants.map | awk '/convert_revision/{print $2,$3}' \
    > $(hg root)/.hg/shamap

And finally: remember, kids, if you like this kind of information, not only is my Mercurial book free to read online, but if you buy a paper or ebook copy, all the royalties go to the Software Freedom Convervancy, whose work is so worthy of support you should buy five copies, not just one.

First, the cosmetic news. The change in appearance is due to my switching over to the system I used to publish Real World Haskell. I’ve always made the source code to the Mercurial book available, but now it includes the source of the Django app and jQuery code that drives the comment system. So if you’re writing an online book, have at it!

The change in title occurs because O’Reilly Media is going to publish the book in a few months, once I’ve tidied it up (with, I hope, your help!). The content is still licensed under the OPL, so you remain free to use and modify my work under remarkably liberal terms.

O’Reilly and I are donating my royalties on this book to the Software Freedom Conservancy, so if you buy a hardcopy, you’ll be helping one of the finest causes in the software world.

This is a great time to get involved in the production and polishing of the book, and as I did with the Haskell book, I’ll give credit to everyone who provides their name. If you’re interested in helping out, be it by contributing content, corrections, typo fixes, or suggestions, you have two easy ways to do so:

• Clone a copy of the source tree and start sending patches
• Use the comment system on the web site to point out errors and omissions, or to make suggestions
• For bigger stuff, report an issue

So please dive on in, and let’s make this book an even more outstanding resource for the Mercurial community!

The talk is in room F150, on Wednesday, July 26, at 2:35pm. It’ll be a lot of fun.

Yesterday, I posted an announcement of the availability of the first chapter of the Mercurial manual to the mailing list. You can read the first chapter here (PDF only thus far); it documents Mercurial Queues, which I’ve enthused about at length in front of several audiences recently.

I’m working on the manual in the open, and releasing it under the Open Publication License. It’s a lot of fun to finally be writing a publication-quality manual for a piece of software I like so much.

As the manual progresses, I’ll be making it available in both HTML and PDF formats, with integrated feedback links available. More about that later, as time permits.

Of course we didn’t come to any conclusions about one project subsuming the other, but we did gain a better understanding of where our respective projects stand relative to each other.

The sprint lasted three days. We spent much of the first day talking about how we got to where we are; how did people get interested in revision control, why and how they started contributing to their respective projects, and so on. The second day involved more discussions of details, while thinking in terms of how we could do things that benefited each project individually while assisting both. A few prospects, for example, were a common format for exporting metadata to the likes of GUIs, and a common plugin framework. The third day was much less structured; we showed off interesting bits of functionality, hacked a little, and talked about whatever topics were interesting at the time.

In terms of “stuff we have that they don’t”, the standouts on each side were performance and complete handling of renamed files and directories. Mercurial has performance, and the Bazaar-NG team people want it; and Bazaar-NG handles merging edits across renames, while the Mercurial team wants it.

For me, it was very energising to meet so many people, all deeply interested in distributed revision control, all at once. I’d already known Matt from his days living in San Francisco, but everyone else was new to me. I was struck by how fortunate each project is to have a core of people who are whip-smart, thoughtful, and a lot of fun to spend time with. The tone of our conversations was great; everyone was amicable and engaged, able to both talk well and listen carefully.

So we’ve converged on such similar user interfaces, we found it easy to talk to each other, and we found so many cases where we were tackling or ignoring the same problems in similar ways. What will be our respective paths forward? I don’t have a clear answer. Some of the possibilities have non-technical constraints that may prove amenable to compromise; we’ll have to see.

Of course, it is ultimately not in low cunning, marketing, or funding that one project shall in the end dominate, but in mindless feats of strength (that’s Mark Shuttleworth on the left, Matt Mackall on the right).

Thanks to everyone who took the time to attend and, through their willingness to talk and listen, made the weekend a memorable and worthwhile event. In particular, I’d like to thank Mark Shuttleworth for sponsoring the event.

If you’re a local Mercurial user and we haven’t met, please feel free to come along. If you don’t use Mercurial, you’re welcome to attend and see what it’s all about.

I got to come along as gadfly-in-chief, or filler-of-the-bug-database, or some such thing, but I am writing this entry from London, and the sprint will kick off tomorrow morning.

Each project is already actively snarfing code from the other. For example, Alexander Schremmer’s lovely branchview work lets you view the branch and merge history of a Mercurial repository, graphically, from within a regular web browser. It uses code from Bazarr-NG’s bzrk plugin to do the abstract graph rendering. On the other side, Goffredo Baroncelli has ported Mercurial’s built-in web server to run on top of bzr.

In terms of basic user interface (the stuff you need to learn in the first five minutes of use), the two projects are very similar. Indeed, many think-alike commands have identical names in each. Even the underlying data structures are similar, as evinced by Robert Collins recently writing a Bazaar-NG plugin that can pull from Mercurial repositories.

I’ll refrain from discussing the respective merits and drawbacks of each project, as I see them, for now. That can wait for a bit.

Over the next few days, we’ll see where and how we can find useful ways to discuss the differences between our respective approaches, and whether we can map a way to work together. It should be fun!

As I mentioned the other day, I will not be contributing to Mercurial development for a while. Several people have asked me why.

At my workplace, we use a commercial SCM tool called BitKeeper to manage a number of source trees. Last week, Larry McVoy (the CEO of BitMover, which produces BitKeeper) contacted my company’s management.

Larry expressed concern that I might be moving BitKeeper technology into Mercurial. In a phone conversation that followed, I told Larry that of course I hadn’t done so.

However, Larry conveyed his very legitimate worry that a fast, stable open source project such as Mercurial poses a threat to his business, and that he considered it “unacceptable” that an employee of a customer should work on a free project that he sees as competing.

To avoid any possible perception of conflict, I have volunteered to Larry that as long as I continue to use the commercial version of BitKeeper, I will not contribute to the development of Mercurial.

As such, Mercurial can stand entirely on its own merits in comparison to BitKeeper. This, I am sure, is a situation that we would all prefer.