Good Idea
Have a common dump file format that uses key/value pairs of the form:
key: value\n
Bad Idea
Chose not to have full paths start with a leading ‘/’, so that you end up with oddities like:
Node-path: \n
I figured I’d take a look at Bazaar again, since they’ve made so many performance improvements. So, as a starting point, I decided to see how well svn2bzr worked. So I ran it on a dump file of The Subversion Repository. After some other tweaks to the script, I managed to get to revision 26429 where it choked:
Traceback (most recent call last):
File “./svn2bzr.py”, line 1086, in
main()
File “./svn2bzr.py”, line 1079, in main
opts.prefix, opts.filter)
File “./svn2bzr.py”, line 1001, in svn2bzr
dump = Dump(dump_file)
File “./svn2bzr.py”, line 662, in __init__
self._read()
File “./svn2bzr.py”, line 900, in _read
field, value = line.split(’: ‘, 1)
ValueError: need more than 1 value to unpack
Exception exceptions.OSError: (2, ‘No such file or directory’, ‘/var/folders/9W/9WK-dXFfH2eMq+dEVqJZg++++TI/-Tmp-/tmproKrc3-saved-trees’) in > ignored
Classic sign of expecting a value to be there, only to find out it’s not. :-( Turns out the empty Node-path is technically correct, since it was the root of the repository that was modified. It got there because a merge ticket was written as a property of the root directory by SVK in r26429. However, we really should have included a leading slash in the node paths to prevent having an “empty” value. I guess hind-sight is always 20/20.
I’ve spent some more time playing with Mercurial, and discovered a few things. First, renames actually track unlike Subversion, where updating my working copy may result in the old filename being updated, but the new won’t receive those changes. Continue reading ‘More playing with Mercurial…’
So, now that the dust has settled–to some degree–on distributed version control systems being the answer to every development problem, I decided to try a few out. To be honest, I tried a number of tools out months (a year?) ago, but many of them were still in the early stages of development and had a number of shortcomings. One of the tools I revisited recently was Mercurial. Continue reading ‘Taking a look at Mercurial’
There was a regression in fsfsverify that prevented it from detecting overflowed windows when the instruction stream wasn’t corrupted. That has been fixed, along with a couple of other minor adjustments. The new version is up and ready for use!
I’ve been hard at work for the past couple of weeks, and I’m proud to announce that an updated fsfsverify script is available. Functionally, the only difference is the addition of svndiff1 support. Underneath, the great majority of the code has changed to make it easier to support different types of svndiff streams. It should also be easier to hack, if you find yourself needing to add a feature. Enjoy!
Things are going well on the fsfsverify front. I’ve run the new code through a number of tests, and it’s performing just as well as the old one. If this keeps up, I’ll have it out the door this week.
I’ve been hard at work on a new version of fsfsverify that handles the new svndiff1 format. I also took it as an opportunity to rewrite large portions of it, because it’s clearly leading a life of it’s own. So far it has seen 330 downloads by 278 unique ip addresses since June of last year. I’m sure many of those were people just curious about the script. However, I know people are indeed using it, since I get emails declaring success, and occasionally ones that ended in failure (hey, I can’t fix data loss).
At any rate, I hope to finish up most the remaining items over the next week or so. As always, real life takes precedence, so that may end up being more like a month.
I appreciate those folks who have emailing me to let me know how things went. I try to keep tabs on just how wide-spread the issue is, and the emails help greatly. The only thing I ask is that along with your emails, please provide some details about your setup. Which OS, which version of the OS and Subversion. How are you accessing it (svn://, http://, etc)? What quality was the link when the corruption occured? Local lan? Over a modem? Is it reliable? Is it latent? Have you experienced any hard disk errors? Memory errors?
It’s important to provide that information because it can help us narrow the field and make sure we fix the issue. To date, no dev has been able to reproduce the problem. Until we can reproduce the problem, we can’t really understand what’s causing it.
A recent Subversion user had troubles with the middle of their svndiff stream becoming corrupt. Thanks to his openess, and sharing of the corrupted revisions, I was able to verify that it is the same issue as the one causing the read length line error. I was also able to update fsfsverify to attempt to fix this issue too. If you’re suffering from a similar problem, you might want to download the new version and give fsfsverify a try.
If you haven’t downloaded and tested the release candidate yet, please read this email, and download it now. A number of folks contributed greatly to the working copy performance, and the FSFS backend. I’ve been using it on a Linux box with a kernel tree working copy, and the improvement is very dramatic. There is a new svnsync tool, and a new –summarize option to svn diff that I think are particularly useful, along with the option to ignore white space changes in both diffs and in the blame output. Download the source and give it a try!