Bazaar as a Subversion "super client"

Nearly a year ago, I read an article by Ben Collins-Sussman called A Mercurial “super client”. About a month or so prior to that, I had started playing with Bazaar and, in particular, bzr-svn. Since then, I have really wanted to write a similar article for Bazaar, but haven’t found the time. I’m happy to say, that both Bazaar and bzr-svn have come to a head, with Bazaar releasing 2.0 and bzr-svn releasing 1.0. So, it seems like the right time to get this done!

In the 2.0 release, Bazaar has addressed most–if not all–of the speed concerns and the format issue. The default format in Bazaar 2.0 is fast, compact, and supports bzr-svn out of the box. That’s all fine and good, but it’s really bzr-svn that I’m clamoring over. bzr-svn was written by the incredible Jelmer Vernooij, and out of the box, bzr-svn gives you the ability to interact with a Subversion repository in the same way as Bazaar branches. This means that if you already know how to use Bazaar, then you know how to use Bazaar against a Subversion repository.

Getting Started

If you’re on Windows, simply grab the installer from the Windows download area on the Bazaar site. It includes bzr-svn by default. If you’re on Ubuntu, then you can grab the latest version from the Bazaar PPA. For other *nix, you probably need to install from source.

Installing from source is rather easy. You’ll need the Subversion development libraries installed, and you’ll need to grab subvertpy–which are a set of Python bindings, also written by Jelmer. Once you have the source for bzr, bzr-svn, and subvertpy we can start building:

# Build and install Bazaar
$ tar zxvf bzr-2.0.0.tar.gz
$ cd bzr-2.0.0
$ python setup.py build
$ sudo python setup.py install
$ cd ..

# Build and install subvertpy
$ tar zxvf subvertpy-0.6.9.tar.gz
$ cd subvertpy-0.6.9
$ python setup.py build
$ sudo python setup.py install
$ cd ..

If you built your own version of Subversion (like I commonly do), then you’ll need to build bzr-svn slightly differently:

# Build subvertpy against a custom SVN
$ tar zxvf subvertpy-0.6.9.tar.gz
$ cd subvertpy-0.6.9
$ export SVN_PREFIX=/path/to/custom/version
$ python setup.py build
$ sudo python setup.py install
$ cd ..

The difference here, is the export. You need to point to the prefix where you installed Subversion. For me, that’s /Users/jszakmeister/local/.

Before we go any further we should configure our identity, which also has the side effect of creating the ~/.bazaar area:

$ bzr whoami "John Doe <john.doe@example.com>"

Obviously, change the name and email to match what you typically use for development.

Next, install the bzr-svn plugin. You can do this two ways: for everyone, or just yourself. I’m going to show you how to setup bzr-svn just for you:

# Install bzr-svn
$ cd ~/.bazaar/plugins
$ tar zxvf /path/to/bzr-svn-1.0.0.tar.gz
$ mv bzr-svn-1.0.0 svn

You’ll likely want to install bzr-rewrite as well. Hopefully, the page will be updated soon with the latest release. Until then, you can grab it directly from Jelmer’s bzr archive. For this article, I’m using bzr-rewrite 0.5.4. Installing it is very similar to installing bzr-svn:

# Install bzr-svn
$ cd ~/.bazaar/plugins
$ tar zxvf /path/to/bzr-rewrite-0.5.4.tar.gz
$ mv bzr-rewrite-0.5.4 rewrite

Once you’ve installed bzr and the above plugins, let’s to a quick check to ensure the installation went well:

$ bzr plugins
launchpad 2.0.0
  Launchpad.net integration plugin for Bazaar.

netrc_credential_store 2.0.0
  Use ~/.netrc as a credential store for authentication.conf.

rewrite 0.5.4
  Rebase support.

svn 1.0
  Support for Subversion branches

First steps

Once you’ve got everything installed, we’re ready to rock-n-roll. First, I’m going to set up a shared repository to help save some space:

$ bzr init-repo project-name
$ cd project-name

Then, we create a mirror of the project’s trunk:

# Create a mirror of trunk
$ bzr branch http://svn.example.com/svn/project-name/trunk/ trunk

Next, let’s create a branch to work on and fix a few things:

# Create a branch to work on
$ bzr branch trunk myhacks

# Get to work!
$ cd myhacks
(hack, hack, hack)
...

# Ready to commit
$ bzr ci -m "Fixed blah, and added foo."

At this point, we now have a local branch with some changes in it. Now we want to integrate them back into trunk:

$ cd ../trunk
$ bzr merge ../myhacks
$ bzr ci -m "Fixed blah, and added foo."
$ bzr push :parent

In Subversion, this will appear as a single commit that introduces all of the changes. In Bazaar, it records the merge history, so you can actually see the individual commits as long as you have access to the Bazaar branch.

The Catch

I think it can be summed up into a single rule:

When reintegrating your change back into the Subversion mainline, always merge your Bazaar branch back into the Subversion mainline if you’ve merged changes from the Subversion mainline. IOW, don’t push your changes back into the mainline, merge them.

For short term branches, this isn’t an issue. You’ll generally branch, add some feature, and then merge it back to trunk. The commit graph for this looks like the following:

The catch comes into play when you want to develop a long-term feature branch. At that point, you probably want to track trunk, so you’d execute:

$ cd ../myhacks
$ bzr merge ../trunk
$ bzr ci -m "Merge parent."

Then you’ll keep working, and perhaps make a few extra commits to myhacks, and the resultant commit graph will look like the following:

Looking at the commit graph from the viewpoint of myhacks, it looks like:

This is where the potential problem comes into play. You could push myhacks directly into trunk, after all, it’s up-to-date with the changes. But look at the commit ordering. Right now, F is the last commit on trunk. If you push myhacks into trunk, it needs to re-order the commits to make them look like what is present in myhacks. From a Subversion user standpoint, that’s confusing to see, although correct from a commit graph perspective. For this reason, bzr-svn does not allow this to occur by default.

A better answer is to merge your feature branch back to trunk, just as you would have done with Subversion or a short term branch. The resultant commit graph is now:

Everything has trade-offs, and this is no different. Notice in the mainline, you get a single commit, just as you would with Subversion. However, newer versions of Subversion can do merge tracking and tell you where a particular line came from. It can do this because the revisions were there in the repository. This isn’t necessarily true with Bazaar. Everything could have been done offline, and you lose that ability to see what happened. However, if you push your long term branch into, say, the branches/ area of Subversion, then those revisions will be present, and bzr-svn will record the merge-info for both Bazaar and Subversion.

Rebasing

If you’re working on a branch, and you really want to see those individual commits, I’d highly recommend using Bazaar just like Subversion and follow the centralized development model described in the Bazaar User’s Guide.

An alternative is to use rebase. Using bzr rebase from myhacks will undo your commits, pull the most recent changes in from trunk, and replay your commits on top of it. When you’re ready, you merge to trunk just as I showed in the Getting Started section. The downside is that rebasing can have negative consequences when sharing. The problem isn’t limited to Bazaar either. git-svn and hgsubversion both have similar issues. It’s just how the merge graph works out.

To rebase myhacks against the tip of it’s parent, you run:

# Mirror back the latest changes
$ cd ../trunk
$ bzr pull

# Rebase myhacks onto the tip of it's parent (trunk)
$ cd ../myhacks
$ bzr rebase :parent
(...)

Then to get your changes back into trunk, you execute:

$ cd ../myhacks
$ bzr push http://svn.example.com/svn/project-name/trunk/

You can get conflicts while rebasing, so you’ll want to read up on rebase and see how that process works. Also, if you’re using rebase, don’t merge the parent into your branch. Rebase is bringing in those changes for you.

Other minor issues

There are some other niggles where the feature set of Bazaar and Subversion are not identical. I tried to document the caveats here. The short form is that there is enough there to make this solution compelling, but there a few hiccups to watch out for (like the commit re-ordering mentioned above).

It’s a net win

All of that said, Bazaar and bzr-svn are quite powerful tools. I’ve been using them together in a production environment for nearly a year. It started out rocky, but myself and many others have worked with Jelmer to get a number of small bugs fixed. I’m happy to say it’s a very solid set up, and has been extraordinarily useful to me because I’ve been working in an environment where my connection to the Subversion repositories has been flaky.

Great toolchain

Even though it’s awesome that Bazaar has kept me productive when our external network connection is down, it’s really the toolchain that I love most. The ability to shelve changes is incredibly useful for putting off a change, but not losing it while I work on something else. QBzr, and, in particular, bzr qlog is a great entry point into your branch. It’s more of a “branch explorer” than a log viewer. I can see the commits, where they came from, the color syntax highlighted diffs, and a few other miscellaneous tidbits. I mean, how can you not love this screen:

It’s very easy to add my own capability to bzr. For instance, I’m working on a plugin that will help post my changes right into ReviewBoard for doing our code reviews. I love tools that enable me to be more productive, and I love them more when they let me plug into them because that gives me a great way to deploy capability to the rest of my developers.

That’s all fine and good. Do these tools work when using Bazaar against Subversion? Yes! Bazaar branches created from a Subversion branch are still Bazaar branches, so all of this just works. Here’s a snapshot of qlog against a mirror of SCons trunk:

Give it a try

Are using Subversion? Wish you could get a little more out of it? Why not give Bazaar and bzr-svn a try? They’re both ready for production use and can really improve your productivity.

Updates

I attempted to clarify the “rule,” as I got a couple of questions about it. I also corrected a few minor grammar/spelling issues. Thanks to David Roberts and John Arbash Meinel to their feedback.

I published a follow-on article going over some of the issues I ran into after another year of use.