Git cutting corners

meta = {'category': bazaar, 'date': Sun 27 April 2008}

My relationship with git is still one of love and hate. It cuts corners to increase performance in a couple of places and that can be really bloody annoying.

For example, jerry renamed one of the top-level directories in Samba 3 (revision 9f672c26d63955f613088489c6efbdc08b5b2d14). Git will skip rename detection in this revision because of the number of files it affects, thus causing the output of “git log <path>” of this particular directory to be useless.

I’m the first to admit “bzr log” on directories and files in large history projects is painfully slow, but at least it gets the output right.

Currently Playing: Brandi Carlile - The Story

comments.

SambaXP and other travel

meta = {'category': samba, 'date': Sat 26 April 2008}

It’s been a busy two weeks. Wilco and I drove up to Göttingen on Sunday two weeks ago to spend some days hacking and meeting up with the other developers before the start of SambaXP. It was really nice to see everybody again after more than 7 months.

SambaXP was a bit different this year. There were three tracks during the second part of the conference this year, one more than previously and of course, there were several engineers from Microsoft attending this time! Some of the interesting talks this year included Julien’s update on OpenChange, Tridge’s talk on PFIF, the talk from the likewise folks and of course the talk from Microsofts’ Wolfgang Grieskamp on SMB2. We also had some other informal discussions with the Microsoft folks about specific topics - very useful!

There are some photos up on the SambaXP homepage. And just to be ahead of the comments: yes, I know I need a haircut.

I did some initial work on several bits and pieces of code that I hope to expand over the next few months. Volker has started working on ncacn_ip_tcp support and I have been working on making the Samba 3 DCE/RPC library compatible with Samba 4. This should allow OpenChange to use Samba 3 in the future.

Guenther, Wilco and I made some initial progress on the policy library, allowing client-side manipulation of (group) policies in Samba. I worked with Simo on trying to get rid of an evil hack in Samba4’s event subsystem.

David Holder blogged about some of the IPv6 development that we did during the conference: http://www.ipv6consultancy.com/ipv6blog/?p=34

And lots of other things I can’t remember at the moment…

After the conference Andrew, Wilco and I drove back to the Netherlands and I played tour guide for a bit showing Andrew around the country during the afternoon and hacking Samba together in the morning. Later this week we took the train to Brussels, Eurostar to London and visited Sam’s company in the UK Midlands for a couple of days.

And in the midst of all this, it seems Ubuntu Hardy was released. Congratulations to all those involved!

Currently Playing: Brandi Carlile - Turpentine

comments.

Using bzr-builddeb as a svn-buildpackage replacement

meta = {'category': bazaar, 'date': Wed 26 March 2008}

This slightly evil hack to bzr-svn allows using bzr-builddeb as a drop-in replacement for svn-buildpackage, making it recognize the “mergeWithUpstream” property svn-buildpackage uses.

Currently Playing: Jeff Healey - Mess O’ Blues

comments.

Adaption blockers Bazaar sprint

meta = {'category': bazaar, 'date': Sun 09 March 2008}

The London Bazaar sprint is over again for this year. It was really good to meet everybody in person again and also to meet some of the folks who hadn’t been to a sprint before.

Last years sprint was mainly about improving performance; this year, we discussed adoption blockers and how to remove them. A short summary of the brainstorming is on the wiki.

Martin’s Blog has some pictures.

comments.

OpenChange Evolution plugin preview and Debian packages

meta = {'category': openchange, 'date': Mon 21 January 2008}

Srini writes that a preview of the Evolution OpenChange plugin has just been published. This plugin is now developed in the Evolution Subversion repository, but is based on the original plugin that was written by the Epitech team that was assigned to OpenChange earlier this year.

I’ve packaged new snapshots of Samba and OpenChange for Debian/Ubuntu. They’re available from my personal apt repository and will hopefully soon also be available from Debian experimental.

Update: The packages are now in Debian experimental as well as the upcoming Intrepid release of Ubuntu. I have removed them from my personal repository because I was running out of disk quota.

comments.

Samba’s tdbdump reimplemented in Python

meta = {'category': samba, 'date': Thu 10 January 2008}

Less than 150 characters in Python, while the original implementation in C requires more than 2000 characters

1
2
3
4
5
6

import tdb, sys

db = tdb.Tdb(sys.argv[1])
for (k, v) in db.iteritems():
    print "{\nkey(%d) = %r\ndata(%d) = %r\n}\n" % (len(k), k, len(v), v)
}

comments.

GTK+ LDB Browser

meta = {'category': samba, 'date': Sun 16 December 2007}

As some may have noticed, a large portion of my Samba 4 work during the last few months has been focussed on adding Python bindings for our various public libraries and the refactoring necessary to make it possible to add Python bindings. So far, we have bindings for LDB and TDB but I intend to add bindings for most of our public API so it is possible to, for example, open Windows registry files, join domains, etc. from Python.

LDB is our LDAP-like embedded database, and is for LDAP what sqlite is for SQL. Last night I decided to see how hard it would be to write a graphical browser for LDB using Python, and it turned out to be quite easy, thanks to PyGTK. There is a screenshot of what it looks like here. Packages with the Python bindings for LDB are already in Debian.

The sources for gtkldb are available in the samba-gtk bzr branch at http://people.samba.org/bzr/jelmer/samba-gtk/trunk, along with some of the GTK+ frontends for Samba 4 I wrote earlier (gregedit, gwcrontab, gwsvcctl, gepdump and gwsam).

comments.

Building shared and static libs on different platforms

meta = {'category': misc, 'date': Wed 14 November 2007}

One of the main portability problems across different POSIX implementations is the way shared and static libs should be built.

Just came across a nice overview of the flags to use on the different platforms: http://www.fortran-2000.com/ArnaudRecipes/sharedlib.html

it would be nice if POSIX could standardise this at some point.

comments.

Epitech/OpenChange meeting

meta = {'category': openchange, 'date': Mon 12 November 2007}

I’ve had a lot of time to practice my French skills this weekend while visiting Epitech and meeting up with Julien, Ali, Dan and the other OpenChange folks in Paris.

There was a forum at Epitech where student teams present the projects that they’ve been working on over the course of the last year. One of the Epitech student teams has done an excellent job on an OpenChange plugin for Evolution that now even appears to have been picked up by upstream. The other projects were also quite interesting as well and varied from a fun to play 3D racing game to a much improved LGPL’ed implementation of the ClamAv virus scanner or a personal logging application for diabetics. Afterwards there was a little bit of an Open Source conference, where Dan and I gave talks about Open Source and Samba, respectively.

comments.

OpenChange team member

meta = {'category': openchange, 'date': Mon 22 October 2007}

As of today, I’m officially a member of the OpenChange development team. I already had access to the Subversion server and have contributed a number of patches in the last couple of years, mainly related to the build system.

comments.

Bazaar: Need for a “Product” object?

meta = {'category': bazaar, 'date': Wed 10 October 2007}

This is something that’s been lingering in the back of my head for the last year or so. I think I am missing something in the sequence of [Branch, Repository, WorkingTree]. Here are some of the reasons why I think this is the case:

• Tags should ideally be shared amongst a set of related branches. This has come up often during discussions about where tags belong.
• Management of sets of bzr branches is hard
  • It seems to make sense for the configuration of several plugins to be project-specific:
    • bzr-pqm-submit’s pqm address
    • bzr-email target address
    • bzr-cia’s project setting
    • …
  • may be useful to override whoami
  • having a way to group branches allows mass-pushes/pulls
• I often find that the public_location I set is almost the same for related branches, with only the last part of the url differing and containing the branch nick
• It would be nice if “bzr register-branch” could automatically determine what product to register as

I’m not looking for repositories: - repositories can contain data from multiple totally unrelated branches. a tag “1.0” could conflict because there are multiple unrelated projects that have it. - repositories are a storage optimization and I like them that way

although other projects (mercurial, git) seem to be using repositories to allow talking about a group of related branches.

I’m not looking for “just” directories: - There’s no place in a directory to store settings or tags - Having a long list of settings in ~/bazaar/locations.conf doesn’t scale and the settings won’t be able to propagate

Having another semantic object (”Product”?) on which options/tags can be set would help. Perhaps based on the root id (where available) ?

Currently Playing: Symphony X - The Odyssey

comments.

SerNet asks Microsoft for specs

meta = {'category': samba, 'date': Sun 07 October 2007}

Now that the European court has decided SerNet has asked Microsoft to publish specifications on SMB/CIFS.

Martijn has posted some photos of the concert we went to a couple of days ago.

comments.

Podcast with the Samba team

meta = {'category': samba, 'date': Tue 02 October 2007}

Last week was the annual Samba get-together in the US. Jeremy and Leslie from Google organized a somewhat more informal event than the SNIA CIFS interop lab that we usually attend.

Several Samba team members, including myself, took part in a podcast about our involvement in the Summer of Code afterwards.

comments.

Converting CSV files to QIF

meta = {'category': misc, 'date': Fri 07 September 2007}

Recently I’ve started using GnuCash for my personal finances. Eventually I got it running well, after I recompiled it myself, because the version in Debian was segfaulting on start-up (it is also very outdated).

My bank, PostBank only supports exporting transactions in it’s own custom CSV files which, of course, aren’t supported by Gnucash. I wrote a simple Python script that converts PostBank CSV files to QIF, which Gnucash can handle ok.

I’m publishing the script here for others interested in importing PostBank CSV files in their finance apps:

#!/usr/bin/python
# Very simple converter from PostBank mijn.postbank.nl .csv files to
# QIF files for use by (among others) gnucash.
#
# Written by Jelmer Vernooij <jelmer@samba.org>, 2007

import csv, sys

rows = csv.reader(sys.stdin)

header = rows.next()

assert header == (['Datum', 'Naam / Omschrijving', 'Rekening']
    + ['Tegenrekening', 'Code', 'Af Bij', 'Bedrag (EUR)']
    + ['MutatieSoort', 'Mededelingen'])

print '!Type:Bank\n'

for l in rows:
    if "-" in l[0]:
        p = l[0].split("-")
        print "D%s/%s/%s" % (p[1], p[0], p[2])
    else:
        print 'D%s/%s/%s' % (l[0][4:6], l[0][6:8], l[0][0:4]) # date
    if l[5] == 'Bij':
        print 'T%s' % l[6] # amount
    elif l[5] == 'Af':
        print 'T-%s' % l[6] # amount
    else:
        raise Error("Unknown field value '%s'" % l[5])
    print 'P%s' % l[1] # payee / description
    print 'M%s' % l[8] # comment
    print 'L%s' % l[7]
    print '^\n' # end transaction

comments.

Bzr-Svn

meta = {'category': bazaar, 'date': Fri 03 August 2007}

The next major release of bzr-svn, 0.4, has now been released. The main change in this release is that the behaviour of push is now intuitive. The big hack that allowed push to somewhat work in the previous release has been replaced by proper push which behaves the same way as it would against a Bazaar branch.

It’s now also possible to branch from non-standard branch locations such as /foo in a repository and not necessarily standard locations like /trunk or /trunk/foo. See the release announcement for a list of other changes.

It’s interesting to see what other people are saying about bzr-svn on their blogs:

• Migrating to bzr using bzr-svn
• Missing support for svn:externals
• Issues with branching schemes
• using bzr-svn for rhythmbox
• bzr-svn on Gentoo
• issues with Windows support

Note that bzr-svn 0.4 has been tested on Windows and that branching schemes are now more flexible.

comments.