Sunday, August 26, 2007

Testing translations

A recent bug in Task Coach was caused by one of the translations being incorrect. So, I decided it was time to unittest the translations. For each translated string I wanted to check that certain conditions hold. For example, if the original string has a formatting operator (e.g. '%d' for digits) the translated string should contain the same formatting operator. These tests are relatively simple:

for formatter in '%s', '%d', '%.2f':

The challenge is how to create one unittest for each (language, string)-pair. This is not a good solution:

def testMatchingFormatting(self):
for language in getLanguages():
for english, translated in language.dictionary():

because this unittest stops as soon as one translation is incorrect.

My first thought was that I could use decorators to unfold the loop, but after a few feeble attempts I decided I am not smart enough to wrap my head around decorators. After some more experimenting I ended up with the code below. I put the loop outside the test class and explicitly create a new TestCase class for each (language, string)-pair. This generates a lot of unittests (over 7600 for the current version of Task Coach), but they run in less than 0.5 seconds, so that's a small price to pay for increased test coverage.

import test, i18n, meta, string

class TranslationIntegrityTests(object):
''' Unittests for translations. This class is
subclassed below for each translated string
in each language. '''

def testMatchingFormatting(self):
for formatter in '%s', '%d', '%.2f':

def testMatchingAccelerators(self):
# snipped

def getLanguages():
return [language for language in \ \
if language is not None]

def createTestCaseClassName(language, englishString,
''' Generate a class name for the test case class based
on the language and the English string. '''

# Make sure we only use characters allowed in Python
# identifiers:
englishString = englishString.replace(' ', '_')
allowableCharacters = string.ascii_letters + \
string.digits + '_'
englishString = ''.join([char for char in englishString \
if char in allowableCharacters])
className = '%s_%s_%s'%(prefix, language, englishString)
count = 0
while className in globals(): # Make sure className is unique
count += 1
className = '%s_%s_%s_%d'%(prefix, language,
englishString, count)
return className

def createTestCaseClass(className, language, englishString,
class_ = type(className,
(TranslationIntegrityTests, test.TestCase),
class_.language = language
class_.englishString = englishString
class_.translatedString = translatedString
return class_

for language in getLanguages():
translation = __import__('i18n.%s'%language,
for english, translated in translation.dict.iteritems():
className = createTestCaseClassName(language, english)
class_ = createTestCaseClass(className, language,
english, translated)
globals()[className] = class_