Test Driven Development in Python

One of my favorite aspects of Python is that it makes practicing TDD very easy. What makes it so frictionless is the doctest module. It allows you to write a test at the same time you define a function. No setup, no boilerplate, just write a function call and the expected output in the docstring. Here’s a quick example of a fibonacci function.

def fib(n):
        '''Return the nth fibonacci number.
        >>> fib(0)
        0
        >>> fib(1)
        1
        >>> fib(2)
        1
        >>> fib(3)
        2
        >>> fib(4)
        3
        '''
        if n == 0:
                return 0
        elif n == 1:
                return 1
        else:
                return fib(n - 1) + fib(n - 2)

If you want to run your doctests, just add the following three lines to the bottom of your module.

if __name__ == '__main__':
        import doctest
        doctest.testmod()

Now you can run your module to run the doctests, like python fib.py.

So how well does this fit in with the TDD philosophy? Here’s the basic TDD practices.

  1. Think about what you want to test
  2. Write a small test
  3. Write just enough code to fail the test
  4. Run the test and watch it fail
  5. Write just enough code to pass the test
  6. Run the test and watch it pass (if it fails, go back to step 4)
  7. Go back to step 1 and repeat until done

And now a step-by-step breakdown of how to do this with doctests, in excruciating detail.

1. Define a new empty method

def fib(n):
'''Return the nth fibonacci number.'''
pass

if __name__ == '__main__':
import doctest
doctest.testmod()

2. Write a doctest

def fib(n):
        '''Return the nth fibonacci number.
        >>> fib(0)
        0
        '''
        pass

3. Run the module and watch the doctest fail

python fib.py
**********************************************************************
File "fib1.py", line 3, in __main__.fib
Failed example:
    fib(0)
Expected:
    0
Got nothing
**********************************************************************
1 items had failures:
   1 of   1 in __main__.fib
***Test Failed*** 1 failures.

4. Write just enough code to pass the failing doctest

def fib(n):
        '''Return the nth fibonacci number.
        >>> fib(0)
        0
        '''
        return 0

5. Run the module and watch the doctest pass

python fib.py

6. Go back to step 2 and repeat

Now you can start filling in the rest of function, one test at time. In practice, you may not write code exactly like this, but the point is that doctests provide a really easy way to test your code as you write it.

Unit Tests

Ok, so doctests are great for simple tests. But what if your tests need to be a bit more complex? Maybe you need some external data, or mock objects. In that case, you’ll be better off with more traditional unit tests. But first, take a little time to see if you can decompose your code into a set of smaller functions that can be tested individually. I find that code that is easier to test is also easier to understand.

Running Tests

For running my tests, I use nose. I have a tests/ directory with a simple configuration file, nose.cfg

[nosetests]
verbosity=3
with-doctest=1

Then in my Makefile, I add a test command so I can run make test.

test:
        @nosetests --config=tests/nose.cfg tests PACKAGE1 PACKAGE2

PACKAGE1 and PACKAGE2 are optional paths to your code. They could point to unit test packages and/or production code containing doctests.

And finally, if you’re looking for a continuous integration server, try Buildbot.

  • oblivious

    TDD is great, but nosetests, not doctest, is the way to go for that.

    I love doctest, but I don’t think anybody should be using it for TDD. The syntax gets tricky for embedded strings, and it gets awkward if you need to refer to module globals.

    doctest is for literate, reliable documentation. The worst kind of doc is one that is no longer true. With a doctest example, the reader knows that the docs are still correct since the doctests still pass. It also enforces a uniform standard for displaying examples, which improves readability. So don’t stop using doctests, but try something else for TDD.

  • oblivious

    TDD is great, but nosetests, not doctest, is the way to go for that.

    I love doctest, but I don’t think anybody should be using it for TDD. The syntax gets tricky for embedded strings, and it gets awkward if you need to refer to module globals.

    doctest is for literate, reliable documentation. The worst kind of doc is one that is no longer true. With a doctest example, the reader knows that the docs are still correct since the doctests still pass. It also enforces a uniform standard for displaying examples, which improves readability. So don’t stop using doctests, but try something else for TDD.

  • http://streamhacker.com/ Jacob Perkins

    I agree with your points about doctests as documentation. In my experience, there’s many times when those are all the tests you need. If the doctest examples run, then you don’t need to write any extra unit tests. And for those cases of simple input-output functions, doctests can be integrated into your TDD practices.

  • http://weotta.com Jacob

    I agree with your points about doctests as documentation. In my experience, there’s many times when those are all the tests you need. If the doctest examples run, then you don’t need to write any extra unit tests. And for those cases of simple input-output functions, doctests can be integrated into your TDD practices.

  • foo

    I’d say the natural workflow for doctests is demonstrate the use of a module in interactive interpreter and just copy-paste the results as documentation. TDD beats the purpose of this free documentation, and thus other testing methods might be more appropriate.

  • foo

    I’d say the natural workflow for doctests is demonstrate the use of a module in interactive interpreter and just copy-paste the results as documentation. TDD beats the purpose of this free documentation, and thus other testing methods might be more appropriate.

  • http://streamhacker.com/ Jacob Perkins

    The workflow I’m suggesting is a bit different: write the doctests in the docstring at the same time you write the code, then run it to see if you get the expected results. I see doctests as hitting two birds with one stone: working tests plus free documentation.

  • http://weotta.com Jacob

    The workflow I’m suggesting is a bit different: write the doctests in the docstring at the same time you write the code, then run it to see if you get the expected results. I see doctests as hitting two birds with one stone: working tests plus free documentation.