Unit Testing with CPPUnit

Click here for a larger image.

Introduction

Within a Quality Assurance process, we have mainly two kinds of tests:

• Unit tests (or acceptance tests): a set of verifications we can make to each logic unit in our system. With each test, we're checking its behavior, without keeping in mind all collaborations with other units.
• System tests (or integration tests): every test allows you to check system's behavior, emphasizing unit collaborations.

We're going to speak about "unit testing" and how we can apply it in our C/C++ project, through a CPPUnit unit testing framework.

I'm going to consider that you know what unit testing is, and why it is very important in the software development process. If you want to read more about the unit testing basis, you can check the JUnit Web site.

Unit Testing Design

Think about a typical scenario in a development team: A programmer is testing his or her code by using the debugger. With this tool, you can check each variable value in every program at any time. By running step by step, you can verify whether a variable has the expected value. This is powerful, but pretty slow and it might have plenty of errors. A few programmers can keep their mind in a deep, hard, and long debugging process and, after one or two hours, the programmer's brain is near break down. All these repetitive and hard verifications can be done automatically, with a few programming instructions and proper tools.

These tools I'm going to speak about are called "unit testing frameworks." With them, you can write small modules that help you to test modules (classes, functions, or libraries) of your applications.

Let's see this example: We're programming a small program module, whose main responsibility is just to add two numbers. As we're coding in plain C, this module is represented by a C function:

BOOL addition(int a, int b)
{
    return (a + b);
}

Our testing unit should be coded with another module, that is: another C function. This function checks all possible addition cases, and returns TRUE or FALSE, denoting whether the module does or doesn't pass the test:

BOOL additionTest()
{
    if ( addition(1, 2) != 3 )
        return (FALSE);

    if ( addition(0, 0) != 0 )
        return (FALSE);

    if ( addition(10, 0) != 10 )
        return (FALSE);

    if ( addition(-8, 0) != -8 )
        return (FALSE);

    if ( addition(5, -5) != 0 )
        return (FALSE);

    if ( addition(-5, 2) != -3 )
        return (FALSE);

    if ( addition(-4, -1) != -5 )
        return (FALSE);

    return (TRUE);
}

As we can see, we've tested all possible addition cases:

• Positive + Positive
• Zero + Zero
• Positive + Zero
• Negative + Zero
• Positive + Negative
• Negative + Positive
• Negative + Negative

Each test compares the addition result with expected value, and it returns FALSE if the result is a value that is different than the expected one. If the execution path reaches the last line, we consider that all tests have been passed correctly, and it returns TRUE.

This small module (or function) is called Test Case, and it shows a set of checks we do over a single unit. Every verification must be related to a single unit scenario. In this case, we check how "addition operation" behaves about the operand's sign. We can write other Test Cases, for checking others scenarios. For example, we can code another Test Case to check our module behavior with typical addition properties:

int additionPropertiesTest()
{
    // conmutative: a + b = b + a
    if ( addition(1, 2) != addition(2, 1) )
        return (FALSE);

    // asociative: a + (b + c) = (a + b) + c
    if ( addition(1, addition(2, 3)) !=
         addition(addition(1, 2), 3) )
        return (FALSE);

    // neutral element: a + NEUTRAL = a
    if ( addition(10, 0) != 10 )
        return (FALSE);

    // inverse element: a + INVERSE = NEUTRAL
    if ( addition(10, -10) != 0 )
        return (FALSE);

    return (TRUE);
}

In this example, we've checked some mathematical addition properties. These two Test Cases build a Test Suite, that is: a collection of Test Cases that test the same unit.

All those Test Cases and Test Suites must be developed while we're coding the units, and every time the unit changes, the corresponding unit test should reflect changes, modifying a Test Case or adding new one.

For instance, if we improve our "addition" module to add decimal numbers, we have to change our tests, adding, for example, a new addDecimalNumbersTest Test Case.

Extreme programming recommends you that you code all these unit tests before you code the target unit. The main reason is very simple: When you're involved in a development process, you're in a permanent research stage, in which you're thinking about how a unit should behave, what public interface you should publish, what parameters you should pass in methods, and other concrete aspects about external access, internal behavior... By coding "unit tests" before its development, you're getting this set of knowledge, and, when you code the main unit, you'll be able to develop faster and better than the other way.

Each time a team wants to deploy a new release, it should perform a complete unit tests battery. All units must pass their unit (or acceptance) tests, and in this case, we can release a successful new version. If at least one unit doesn't pass all its tests, we've found a bug. In that case, we must code another test, even add a new Test Case if necessary, checking all conditions to reproduce this bug. When our newly coded test can reproduce the bug properly, we can fix it, and perform the test again. If unit passes the test, we consider the bug is resolved and we can release our new, bug-free version.

Testing Tools

Once upon a time, two guys called Kent Beck & Eric Gamma wrote a set of Java classes to make unit testing as automatic as they can. They called them JUnit and it became a great hit in the unit testing world. Other developers ported their code to other languages, building a big collection of products, called xUnit frameworks. Among them, we can find one for C/C++ (CUnit and CPPUnit), Delphi (DUnit), Visual Basic (VBUnit), NUnit (.NET platform), and many others.

All these frameworks apply similar rules, and probably you can use one if you've used another one, with few language-dependent exceptions.

Now, we're going to explain how you can use CPPUnit to write you own unit tests and improve your units' quality.

CPPUnit uses object-oriented programming, so we're going to work with concepts such as inheritance, encapsulation, and polymorphism. Also, CPPUnit uses C++'s SEH (Structured Exception Handling), so you should understand concepts such as "exception" and instructions, and structures such as throw, try, finally, catch, and so on.

CPPUnit

Each Test Case should be coded inside a class derived from TestCase. This class brings us all basic functionality to run a test, register it inside a Test Suite, and so on.

For instance, we've written a small module that stores some data on a disk. This module (coded as a class called DiskData) has two main responsibilities: load and store data inside a file. Let's take a look:

typedef struct _DATA
{
    int  number;
    char string[256];
} DATA, *LPDATA;


class DiskData
{
public:
    DiskData();
    ~DiskData();

    LPDATA getData();
    void setData(LPDATA value);

    bool load(char *filename);
    bool store(char *filename);

private:
    DATA m_data;
};

For now, it isn't important how these methods are coded because the most important thing is that we must be sure this class is doing all the things it must do, that is: load and store data correctly into a file.

To do this verification, we're going to create a new Test Suite with two test cases: one to load data and another to store data.

Using CPPUnit

You can get latest CPPUnit version here, where you can find all libraries, documentation, examples, and other interesting stuff. (I've downloaded 1.8.0 and it works fine.)

In the Win32 world, you can use CPPUnit under Visual C++ (6 and later), but because CPPUnit uses ANSI C++, there are a few ports to other environments, such as:

All steps and information about building libraries can be found in the INSTALL-WIN32.txt file, inside the CPPUnit distribution. Once all binaries are built, you can write your own Test Suites.

To write your own unit test applications under Visual C++, you must follow these steps:

1. Create a new dialog-based MFC application (or doc-view one).
2. Enable RTTI: Project Settings - C++ - C++ Language.
3. Add a CPPUnit\include folder to include these directories: Tools - Options - Directories - Include.
4. Link your application with cppunitd.lib (for static link) or cppunitd_dll.lib (for dynamic link), and testrunnerd.lib. If you're compiling under the "Release" configuration, you should link with same libraries, but without the "d" suffix.
5. Copy testrunnerd.dll to your executable folder, or any other folder in your path, and cppunitd_dll.dll if you linked dynamically (or testrunner.dll and cppunit_dll.dll if you're under Release).

Once your project is ready, we can code our first unit test class.

We're going to test our DiskData class, which mainly performs two operations: load and store data into a disk file. Our test case should test these two operations, with two Test Cases: one to load and the other to store the data.

Let's take a look at the unit test class definition:

#if !defined(DISKDATA_TESTCASE_H_INCLUDED)
#define DISKDATA_TESTCASE_H_INCLUDED

#if _MSC_VER > 1000
#pragma once
#endif    // _MSC_VER > 1000

#include <cppunit/TestCase.h>
#include <cppunit/extensions/HelperMacros.h>

#include "DiskData.h"

class DiskDataTestCase : public CppUnit::TestCase
{
  CPPUNIT_TEST_SUITE(DiskDataTestCase);
      CPPUNIT_TEST(loadTest);
      CPPUNIT_TEST(storeTest);
  CPPUNIT_TEST_SUITE_END();

public:
    void setUp();
    void tearDown();

protected:
    void loadTest();
    void storeTest();

private:
    DiskData *fixture;
};

#endif

First of all, we must include TestCase.h and HelperMacros.h. For the first one, lets us derive our new class from the TestCase base class. The second one helps us with some macros to define unit tests faster, such as CPPUNIT_TEST_SUITE (to start the Test suite definition), CPPUNIT_TEST (to define a test case), or CPPUNIT_TEST_SUITE_END (to end our test suite definition).

Our class (called DiskDataTestCase) overrides two methods called setUp() and tearDown(). These methods are called automatically, and are executed when each Test Case starts and ends, respectively.

Protected methods implement our test logic, one for each Test Case. In the next few lines, I'll explain how you can code your test logic.

And finally, we define an attribute called fixture. This pointer will hold the target object of our tests. We should create this object inside the setUp() method, which is called before each Test Case. Then, the Test Case code will be executed using our fixture object, and finally we destroy this object inside tearDown, after each Test Case execution. In this way, we get a new, fresh object each time we execute a test case.

Our test sequence should be something like this:

1. Start the test application.
2. Click the "Run" button.
3. Call the setUp() method; create our fixture object.
4. Call the first test case method.
5. Call the tearDown() method; free the fixture object.
6. Call the setUp() method; create our fixture object.
7. Call the second test case method.
8. Call the tearDown() method; free the fixture object.
9. ...

Our test sequence should be something like this:

#include "DiskDataTestCase.h"

CPPUNIT_TEST_SUITE_REGISTRATION(DiskDataTestCase);


void DiskDataTestCase::setUp()
{
    fixture = new DiskData();
}

void DiskDataTestCase::tearDown()
{
    delete fixture;
    fixture = NULL;
}


void DiskDataTestCase::loadTest()
{
    // our load test logic
}


void DiskDataTestCase::storeTest()
{
    // our store test logic
}

Implementation is very simple for now: setUp and tearDown methods, create and free fixture objects, respectively. Next, you can see test case methods, which we're going to explain.

Test Case Programming

Once we know what aspects we should test, we must be able to program it. We can perform all operations we need: Use base library calls, third-party library calls, Win32 API calls, or simply use internal attributes with C/C++ operators and instructions.

Sometimes, we'll need external helps such as an auxiliary file or database table that stores correct data. In our test case, we should compare internal data with the external file data to check that they're the same.

Each time we find an error (for instance, if we detect that the internal data isn't the same as the external correct data), we should raise a concrete exception. You can do this with the CPPUNIT_FAIL(message) helper macro, which raises an exception showing a message parameter.

There is another way to check a condition and raise an exception if it's false, all in a single step. The way to do this is through assertions. Assertions are macros that let us check a condition, and they raise the proper exception if the condition is false, with other options. Here are some assertion macros:

• CPPUNIT_ASSERT(condition): Checks the condition and throws an exception if it's false.
• CPPUNIT_ASSERT_MESSAGE(message, condition): Checks the condition and throws an exception and showing specified message if it is false.
• CPPUNIT_ASSERT_EQUAL(expected,current): Checks whether the expected condition is the same as current, and raises an exception showing the expected and current values.
• CPPUNIT_ASSERT_EQUAL_MESSAGE(message,expected,current): Checks whether the expected is the same as the actual, and raises an exception showing the expected and current values, and specified message.
• CPPUNIT_ASSERT_DOUBLES_EQUAL(expected,current,delta): Checks whether the expected and current difference is smaller than delta. If it fails, the expected and current values are shown.

Following with our example, we should code our loadTest method. We're going to follow the next algorithm: We need an auxiliary file that stores one correct DATA structure. The way you create this auxiliary file isn't important, but it is very important that this file is correctly created and the DATA structure must be correctly stored. To check our load method behavior, we're going to call it with our auxiliary file, and then check whether the loaded data is the same as what we know is stored in our file. We can write the code like this:

//
// These are correct values stored in an auxiliary file
//
#define AUX_FILENAME   "ok_data.dat"
#define FILE_NUMBER    19
#define FILE_STRING    "this is correct text stored in auxiliary
                        file"

void DiskDataTestCase::loadTest()
{
    // convert from relative to absolute path
    TCHAR    absoluteFilename[MAX_PATH];
    DWORD    size = MAX_PATH;

    strcpy(absoluteFilename, AUX_FILENAME);
    CPPUNIT_ASSERT( RelativeToAbsolutePath(absoluteFilename,
                                           &size) );

    // executes action
    CPPUNIT_ASSERT( fixture->load(absoluteFilename) );

    // ...and check results with assertions
    LPDATA    loadedData = fixture->getData();

    CPPUNIT_ASSERT(loadedData != NULL);
    CPPUNIT_ASSERT_EQUAL(FILE_NUMBER, loadedData->number);
    CPPUNIT_ASSERT( 0 == strcmp(FILE_STRING, 
            fixture->getData()->string) );
}

With a single test case, we're testing for four possible errors:

• load method's return value
• getData method's return value
• number structure member's value
• string structure member's value

In our second test case, we'll follow a similar scheme, but things get a little harder. We're going to fill our fixture data with known data, store it in another temporal disk file, and then open both files (new and auxiliary one), read them, and compare the contents. Both files should be identical because the store method must generate the same file structure.

void DiskDataTestCase::storeTest()
{
    DATA     d;
    DWORD    tmpSize, auxSize;
    BYTE     *tmpBuff, *auxBuff;
    TCHAR    absoluteFilename[MAX_PATH];
    DWORD    size = MAX_PATH;

    // configure structure with known data
    d.number = FILE_NUMBER;
    strcpy(d.string, FILE_STRING);

    // convert from relative to absolute path
    strcpy(absoluteFilename, AUX_FILENAME);
    CPPUNIT_ASSERT( RelativeToAbsolutePath(absoluteFilename,
                                           &size) );

    // execute action
    fixture->setData(&d);
    CPPUNIT_ASSERT( fixture->store("data.tmp") );

    // Read both files contents and check results
    // ReadAllFileInMemory is an auxiliary function that allocates
    // a buffer and saves all file content inside it. Caller should
    // release the buffer.
    // Check demo project for details
    tmpSize = ReadAllFileInMemory("data.tmp", tmpBuff);
    auxSize = ReadAllFileInMemory(absoluteFilename, auxBuff);

    // files must exist
    CPPUNIT_ASSERT_MESSAGE("New file doesn't exist?", tmpSize > 0);
    CPPUNIT_ASSERT_MESSAGE("Aux file doesn't exist?", auxSize > 0);

    // sizes must be valid
    CPPUNIT_ASSERT(tmpSize != 0xFFFFFFFF);
    CPPUNIT_ASSERT(auxSize != 0xFFFFFFFF);

    // buffers must be valid
    CPPUNIT_ASSERT(tmpBuff != NULL);
    CPPUNIT_ASSERT(auxBuff != NULL);

    // both files' sizes must be the same as DATA's size
    CPPUNIT_ASSERT_EQUAL((DWORD) sizeof(DATA), tmpSize);
    CPPUNIT_ASSERT_EQUAL(auxSize, tmpSize);

    // both files' content must be the same
    CPPUNIT_ASSERT( 0 == memcmp(tmpBuff, auxBuff, sizeof(DATA)) );

    delete [] tmpBuff;
    delete [] auxBuff;

    ::DeleteFile("data.tmp");
}

As we can see, we've configured a DATA structure with known data and stored it using our fixture object. Then, we read the resulting file (data.tmp) and compare it with our pattern file. We made all kinds of verifications, such as buffers' and files' sizes or buffers' contents. If both buffers are identical, our store method works fine.

Launching the User Interface

And finally, we're going to see how we can show an MFC-based user interface dialog, compiled inside the TestRunner.dll library.

We should open our application class implementation file (ProjectNameApp.cpp) and add these lines to our InitInstance method:

#include <cppunit/ui/mfc/TestRunner.h>
#include <cppunit/extensions/TestFactoryRegistry.h>

BOOL CMy_TestsApp::InitInstance()
{
    ....

    // declare a test runner, fill it with our registered tests,
    // and run them
    CppUnit::MfcUi::TestRunner runner;

    runner.addTest( CppUnit::TestFactoryRegistry::getRegistry().
                    makeTest() );

    runner.run();

    return TRUE;
}

This is simpler isn't it? Just define a "runner" instance, and add all registered tests. Tests are registered through the CPPUNIT_TEST_SUITE_REGISTRATION macro call inside our CPP file. Once tests are registered and added to the runner, we can show the dialogs with the run method.

Now, we're ready to run our test cases. Just compile your new project and run it from Visual Studio. You'll see the MFC-based dialog, as above. Just click Browse and you'll see this dialog:

Just select one test (green node), or select the parent blue node to run all registered tests.

Downloads