Friday, June 27, 2008

Automating .deb building

Some time ago, Stani helped me to create a Task Coach package in Debian package format (.deb). This is the package format that is also used by Linux distributions derived from Debian, such as Ubuntu. Since I want the release process of Task Coach to be as easy as possible, I decided to automate the package build process as much as possible.

I wrote a distutils command that creates a Debian package from a source distribution, as created by python sdist. This new distutils command, called bdist_deb, copies the source distribution, unpacks it, adds the necessary Debian control files and compiles the package using the regular packaging tools.

The bdist_deb command takes a large number of parameter since it needs a lot of information to create the .deb. For example, application title, description, version, license information, copyright, author, maintainer, etc. In the case of Task Coach, most of the information is already available in a meta data source file, and easily passed to the bdist_deb command from the distutils setup script.

Since this was specifically developed for Task Coach it is probably not completely generalized. Nevertheless, I hope it provides a starting point for other developers that want to create proper Debian packages for their Python applications.

Check out the bdist_deb command and the invocation of the command in the distutils setup script (called, starting at line 109.

Sunday, April 06, 2008

Migrating from CVS to Subversion

I recently migrated the Task Coach source code from its Sourceforge CVS repository to a Subversion repository, also on Sourceforge. I followed the migration instructions at the Sourceforge website and all went smoothly.

Because the cvs2svn script needs the CVSROOT module present, you can only remove it after cvs2svn has run, i.e. you need remove it from the svndump file that cvs2svn creates. svndumpfilter does do that for you. Unfortunately, the CVSROOT module is present in the svndump file multiple times: not only in the trunk folder, but also in all branch and version folders of the svndump. A script to help with finding all those CVSROOT occurences would be nice.

Marcin ZajÄ…czkowski helped me out with a shell script. I used it as the basis for a little python script that removes specific folders or files from your svndump file. Use as you like.

#!/usr/bin/env python

import os, optparse

usage = '''Usage: %prog dumpfile path [path...]
Remove path(s) from <dumpfile> and write new dump file to <dumpfile>.out'''

class PathRemoverOptionParser(optparse.OptionParser):
def __init__(self):
optparse.OptionParser.__init__(self, usage)

def parse_args(self):
options, args = optparse.OptionParser.parse_args(self)
if len(args) < 2:
self.error('provide both dumpfile and path to remove')
dumpFile, paths = args[0], args[1:]
if not os.path.exists(dumpFile):
self.error('dumpfile (%s) does not exist'%dumpFile)
return dumpFile, paths

def branches(dumpFile, paths, pathPrefix='Node-path: '):
''' Find branches in the dumpFile that contain the paths. '''
for line in file(dumpFile):
if line.startswith(pathPrefix):
for path in paths:
if line.endswith(path+'\n'):
yield line[len(pathPrefix):-1] # yield branch

parser = PathRemoverOptionParser()
dumpFile, paths = parser.parse_args()
excludes = ' '.join(branches(dumpFile, paths))
if not excludes:
parser.error('path(s) (%s) not found'%', '.join(paths))

os.system('svndumpfilter exclude %s < %s > %s.out'%(excludes, dumpFile, dumpFile))