Retrofitting Unit Tests Part 2 – When A Unit Test Isn’t

19Oct07

Picture by icathing Some rights reserved.

One of the most common mistakes I’ve found, particularly with teams who haven’t had a good deal of experience of unit testing or TDD, is that if they do have unit tests they’re often really integration tests with fake mustaches. A small, relatively self-contained piece of code needs a database with test data or a running ORB and server process otherwise the test “fails”.

The issue here is one of scope and the key word is “unit”. Typically this is a single class but could even be a single method in a class. A unit test should only test a single unit of code and not any of its external dependencies. If you’re writing a test to see if a particular Java class aggregates some data correctly prior to writing it to a database then you shouldn’t need a database to find out if it works or not. The test ends when the database is invoked with the correct (or incorrect!) data and the database is treated like a black box that just works.

The reasons for this are quite simple. If you are relying on a database and the test fails you’ll have no idea if it failed because of some problem in the code or in the database. You should be able to work this out, perhaps with a little digging, but false positives like this waste time and seriously reduce your confidence in the ability of unit tests to perform decent regression testing. Imagine having a test that ran in the past but now fails after you make a change to the code being tested. You have to figure out did it fail because of something you did or because the external dependency (e.g. the database) is broken or isn’t set up properly?

Another good reason is that your unit tests should run quickly. If it takes a long time to run all your tests as part of your build the inclination will be to avoid running them. Reducing the amount of execution time spent outside the code under test will help keep them quick. With Java projects I prefer to have the default Ant build target run my unit tests and have a separate “quick” build target to build without the tests. Apart from anything else it makes sure that I don’t forget to run my unit tests while I’m making changes and before I check any of those changes into source control.

Integration tests are important and if you have some masquerading as unit tests don’t bin them. You can do one of two things:

• Move them to a separate area and run them using a different test regime¹. Your unit tests have come out of the closet and are now proud integration tests. Now go off and write unit tests to replace them, being careful to avoid dragging in the external dependencies.
• Alternatively change the existing tests to decouple your external dependencies so that they don’t rely on them being in place to work. Drop the orb and the database and mock them.

Either way mock objects can be incredibly useful and I use them extensively in my tests. If you don’t know what mock objects are have a read of this great article by Martin Fowler and some of the more meaty posts on the Mock Objects blog. The basic idea is that you create a dummy version of the external component, set an expectation of how it should be used and behave but without it actually having to do any real work.

Treating external dependencies as black boxes like this is okay. Imagine you have a class A which uses class B and you have a passing unit test for class B. You can test class A with a mock class B and, if the test passes, can be highly confident that when you put A with a real B it’ll work. On top of that your unit test for class A will run quicker and be less brittle.

In the next part of this series I’ll talk a bit more about mock objects and the problems with introducing them when retrofitting unit tests.

– Fintan

---

1 – There is a school of thought that Continuous Integration should be a continuous run of your integration tests but that’s a whole different argument to be had.

Filed under: Agile, Software Development, Unit Testing   |  Leave a Comment