Maturity of open source projects

In my work as an IT consultant I sometimes use open source projects as reference for judging the quality and maturity of in-house or commercial software projects. Task Coach, for example, has more than 3500 automated unit tests that cover 63% of the 100.000 Python lines of code. Since Task Coach is just a hobby project, this in my mind makes it a lower boundary for assessing the amount and coverage of unit tests in other projects.

Last week I was attending the official Introduction to the CMMI course taught by André Heijstek (of Improvement Focus). While we were discussing the different process areas in the CMMI for Development, I started wondering if and how CMMI would apply to open source organizations and projects. Maybe the CMMI doesn't apply to open source projects at all. However, if a project like Task Coach does achieve a significant portion of the CMMI goals, then that would be another stick in the ground to compare other organization and projects against.

So, the plan is to investigate which of the CMMI for Development v1.2 (and Services too probably; user support is an integral part of open source projects) goals are met by the Task Coach organization and project. Since there are many CMMI process areas I will assess each of the CMMI process areas in a separate posting. As I'm obviously biased, I'll invite André, who is a certified lead appraiser for CMMI, to review my assessments.

Testing translations

A recent bug in Task Coach was caused by one of the translations being incorrect. So, I decided it was time to unittest the translations. For each translated string I wanted to check that certain conditions hold. For example, if the original string has a formatting operator (e.g. '%d' for digits) the translated string should contain the same formatting operator. These tests are relatively simple:

for formatter in '%s', '%d', '%.2f':
  self.assertEqual(self.englishString.count(formatter), 
                   self.translatedString.count(formatter))

The challenge is how to create one unittest for each (language, string)-pair. This is not a good solution:

def testMatchingFormatting(self):
  for language in getLanguages():
    for english, translated in language.dictionary():
      ...

because this unittest stops as soon as one translation is incorrect.

My first thought was that I could use decorators to unfold the loop, but after a few feeble attempts I decided I am not smart enough to wrap my head around decorators. After some more experimenting I ended up with the code below. I put the loop outside the test class and explicitly create a new TestCase class for each (language, string)-pair. This generates a lot of unittests (over 7600 for the current version of Task Coach), but they run in less than 0.5 seconds, so that's a small price to pay for increased test coverage.

import test, i18n, meta, string

class TranslationIntegrityTests(object):
  ''' Unittests for translations. This class is 
      subclassed below for each translated string 
      in each language. '''
      
  def testMatchingFormatting(self):
    for formatter in '%s', '%d', '%.2f':
      self.assertEqual(self.englishString.count(formatter), 
                      self.translatedString.count(formatter))
            
  def testMatchingAccelerators(self):
    # snipped


def getLanguages():
  return [language for language in \
          meta.data.languages.values() \
          if language is not None]
        

def createTestCaseClassName(language, englishString, 
  prefix='TranslationIntegrityTest'):
  ''' Generate a class name for the test case class based 
      on the language and the English string. '''

  # Make sure we only use characters allowed in Python 
  # identifiers:
  englishString = englishString.replace(' ', '_')
  allowableCharacters = string.ascii_letters + \
                        string.digits + '_'
  englishString = ''.join([char for char in englishString \
                           if char in allowableCharacters])
  className = '%s_%s_%s'%(prefix, language, englishString)
  count = 0
  while className in globals(): # Make sure className is unique
      count += 1
      className = '%s_%s_%s_%d'%(prefix, language, 
                                 englishString, count)
  return className


def createTestCaseClass(className, language, englishString, 
                        translatedString):
  class_ = type(className, 
                (TranslationIntegrityTests, test.TestCase), 
                {})
  class_.language = language
  class_.englishString = englishString
  class_.translatedString = translatedString
  return class_


for language in getLanguages():
  translation = __import__('i18n.%s'%language, 
                           fromlist=['dict'])
  for english, translated in translation.dict.iteritems():        
    className = createTestCaseClassName(language, english)
    class_ = createTestCaseClass(className, language, 
                                 english, translated)
    globals()[className] = class_

About this blog

Software development is fun and hard at the same time. This blog discusses my experience with developing an open source desktop application called Task Coach. Task Coach is a task manager that supports hierarchical tasks, budget and time registration, categories, and more. For me, Task Coach is also a vehicle to experiment with software development, tools, and techniques. For example, I try to use test-driven development to steer the development of the software. So far, I like the way test-driven development allows me to add new functionality in a safe and gradual manner. I also like how having an extensive set of automated unit tests (+/- 1700 at the moment, that run in about 30 seconds) allows me to refactor the source code without (too much) fear of seriously breaking the application. Anyway, the plan is to use this blog to talk about ideas about software development, and as much as possible, discuss experience with applying this ideas to Task Coach.