The only mild annoyance has been that I’d prefer to also not be forced to choose a hosting winner: although bitbucket is pretty good, github is currently far slicker, and has a much larger community of potential collaborators.

I’ve hosted most of my code on bitbucket for quite a while. Until this morning, I had a somewhat awkward way to mirror code to github. I just automated the problem away.

My automation scheme is implemented as a Mercurial hook. Here’s how I’ve enabled it in my $HOME/.hgrc file:

[hooks]
post-push = python:/home/bos/share/python/github_mirror.py:post_push

That github_mirror.py hook is very simple. Every time I push, it checks to see if I'm pushing to a bitbucket repository, and if so checks my local repo's .hg/hgrc file to see if I have a mirror on github. If I do, it pushes to github, too.

1
2
3
4
5
6
7
8
9
10

from mercurial import commands

def post_push(ui, repo, pats, opts, *args, **kwargs):
    dest = pats and pats[0]
    dest = ui.expandpath(dest or 'default-push', dest or 'default')
    if 'bitbucket.org' in dest:
        github = ui.config('paths', 'github')
        if github:
            return commands.push(ui, repo, github, **opts)
        ui.warn('no github mirror!?\n')

How do I tell Mercurial that I have a girhub mirror? In a repo's .hg/hgrc file, I have something like this (taken from a real repo):

[paths]
default = http://bitbucket.org/bos/pcap
default-push = ssh://hg@bitbucket.org/bos/pcap
github = git+ssh://git@github.com/bos/pcap.git

The github_mirror.py hook looks for that github key in the paths section of the file, and uses it if present.

There's a fairly painless way around this. Firstly, we'll need to use the filemap option to make hg convert strip out the files we want to lose from our new repository. Here's an idea of how that should work. To start off, we create a demo repository, named hg-100.

$ hg clone -r 100 http://www.selenic.com/repo/hg hg-100
$ ls -F hg-100
comparison.txt	hgweb.py     mercurial/  PKG-INFO  setup.py  tkmerge
hg		MANIFEST.in  notes.txt	 README    tests/

Next, we set up a filemap that eliminates all tests from the tree.

$ echo exclude tests > no-tests.map
$ hg convert --quiet --filemap no-tests.map hg-100 nukeme
$ hg --cwd nukeme update --quiet
$ ls -F nukeme
comparison.txt	hgweb.py     mercurial/  PKG-INFO  setup.py
hg		MANIFEST.in  notes.txt	 README    tkmerge

Now that we have run hg convert, it's changed the changeset ID of every commit that either contains a change to a file in tests or is a child of such a commit. Ouch, right? Haven't we now lost the ability to merge contaminated repositories with the scrubbed one?

Fortunately, no. We could always pull future changes into hg-100, then rerun hg convert with the same --filemap option, and successfully scrub later changes, which we could then share with our collaborators.

The only trouble with this is that it relies on a non-version-controlled file named .hg/shamap in our nukeme tree. That file contains the mapping from source changeset ID to target changeset ID. This is what hg convert uses to tell whether a changeset has been seen before, and if so, what its new ID is. So if we lose the nukeme tree, we can't run hg convert again, and we're stuck, right? Well, once again, not necessarily.

A final trick up our sleeves is to run every instance of hg convert with --config convert.hg.saverev=True. This is documented in the output of hg help, but sadly not on the wiki page. This saves each source changeset ID in a special field in the corresponding changeset in the target repo.

$ rm -rf nukeme
$ hg convert --quiet --filemap no-tests.map --config convert.hg.saverev=True \
    hg-100 nukeme
$ hg --cwd nukeme tip --debug | grep 'extra:'
extra:       branch=default
extra:       convert_revision=526722d24ee5b3b860d4060e008219e083488356

The convert_revision data gives us enough to create a new .hg/shamap file whenever we need to. First, we create a style file to extract the necessary data from Mercurial.

$ cat >> pants.map << EOF
changeset = '{extras}\n'
extra = '{key} {value} {node}\n'
EOF

Then, in any clone of our converted repo, it becomes simple to regenerate the .hg/shamap file:

$ hg log -r0: --style pants.map | awk '/convert_revision/{print $2,$3}' \
    > $(hg root)/.hg/shamap

And finally: remember, kids, if you like this kind of information, not only is my Mercurial book free to read online, but if you buy a paper or ebook copy, all the royalties go to the Software Freedom Convervancy, whose work is so worthy of support you should buy five copies, not just one.

Since the publication of the article, a few people have asked why I didn’t write about their favourite revision control system. The simple answer is that I was already about 50% over my initial word budget by the time I finished up what I did write. (I’ll confess that I was thoroughly charmed by a conspiratorial suggestion that I was editing some beloved tool out of history by not mentioning it.)

First, the cosmetic news. The change in appearance is due to my switching over to the system I used to publish Real World Haskell. I’ve always made the source code to the Mercurial book available, but now it includes the source of the Django app and jQuery code that drives the comment system. So if you’re writing an online book, have at it!

The change in title occurs because O’Reilly Media is going to publish the book in a few months, once I’ve tidied it up (with, I hope, your help!). The content is still licensed under the OPL, so you remain free to use and modify my work under remarkably liberal terms.

O’Reilly and I are donating my royalties on this book to the Software Freedom Conservancy, so if you buy a hardcopy, you’ll be helping one of the finest causes in the software world.

This is a great time to get involved in the production and polishing of the book, and as I did with the Haskell book, I’ll give credit to everyone who provides their name. If you’re interested in helping out, be it by contributing content, corrections, typo fixes, or suggestions, you have two easy ways to do so:

• Clone a copy of the source tree and start sending patches
• Use the comment system on the web site to point out errors and omissions, or to make suggestions
• For bigger stuff, report an issue

So please dive on in, and let’s make this book an even more outstanding resource for the Mercurial community!

Learn more about this video and support EFF!

The talk is in room F150, on Wednesday, July 26, at 2:35pm. It’ll be a lot of fun.

Yesterday, I posted an announcement of the availability of the first chapter of the Mercurial manual to the mailing list. You can read the first chapter here (PDF only thus far); it documents Mercurial Queues, which I’ve enthused about at length in front of several audiences recently.

I’m working on the manual in the open, and releasing it under the Open Publication License. It’s a lot of fun to finally be writing a publication-quality manual for a piece of software I like so much.

As the manual progresses, I’ll be making it available in both HTML and PDF formats, with integrated feedback links available. More about that later, as time permits.