CxxTest User Guide

The --error-printer option creates a runner using the ErrorPrinter test listener, and it indicates that the standard library is used in the test runner. The ErrorPrinter test listener prints dots to summarize test execution, along with a summary of the test results. For example, the command

generates the following output:

If your compiler does not support std::cout, then the ErrorPrinter test listener cannot be used. In this case, the StdioPrinter test listener can be used; it provides the same output as ErrorPrinter but it uses the printf function. For example, the command line:

generates the following output:

The --runner=ParenPrinter option creates a similar test runner:

This test runner generates output that is similar to the ErrorPrinter test listener:

The only difference is the parentheses used in the output. This test listener provides a format that can be recognized by Visual Studio.

The --runner=XmlPrinter option creates a test runner whose output is an XML summary of the test results. For example, the command:

generates the following output:

This XML format is conforms to the XML standard used by other xUnit tools. Thus, this output can be used as input in other tools, like Jenkins, to generate test summaries.

The XUnitPrinter test listener generates output using both the ErrorPrinter+ and XmlPrinter test listeners. This allows the user to interactively view a simple test summary, while simultaneously generating an XML summary of the test results. The --xunit-printer option specifies the use of XUnitPrinter:

This test runner generates the following output:

The default filename for the XML results is TEST-cxxtest.xml. The --xunit-file option can be used to specify an alternative filename. Additionally, the value of the --world option can be used to specify the filename TEST-<world>.xml.

The default test discovery mechanism does a very simple analysis of the input files, which can easily fail when test classes are not formated in a standard manner. For example, consider the following test suite:

This test suite is not recognized by the default test discovery mechanism, but the FOG parser correctly parsers this file and recognizes the test suite. A variety of similar discovery failures arise due to the simple process used by the test discovery mechanism.

Adding and disabling tests are two common steps in test development. The process of test discovery makes adding tests very easy. However, disabling tests is somewhat more complicated. Consider the following header file, which defines four tests (three of which are disabled):

The first is commented out with C++-style comments, the second test is commented out with C-style comments, and the third test is named in a manner that is not recognized through test discovery (i.e., it does not start with test).

The default test discovery mechanism only works with the first and third methods for disabling tests, but the FOG parser works with all three. The FOG parser performs a complex, multi-line parse of the source file, so it can identify multi-line C-style comments.

Note, however, that the use of C macros will not work:

The cxxtestgen discovery mechanisms do not perform a C preprocessing step, since that would generally require using externally defined preprocessing variable definitions. Additionally, preprocessor macros that act like functions will cause the FOG parser to fail unless they are followed by a semicolon.

In some contexts, it is appropriate to nest test classes in namespaces. This allows the same class name to be used without creating conflicts in the test runner. The default test discovery mechanism can only do this by declaring the namespaces with the class name, and then declaring the class with the explicit namespace prefix. For example:

However, the default test discovery mechanism cannot recognize the more typical declaration of test classes within nested namespaces:

The FOG parser can discover tests nested within arbitrary namespaces, and unique names are used within the test runner to distinguish the test classes in different namespaces.

CxxTest test fixtures can be customized in several ways to manage the environment for test suites and individual tests. A common feature of test suites is that they share a common logic for setting up data used in the tests. Thus, there may be duplicate code for creating objects, files, inputs, etc. Similarly, the tests may share common logic for cleaning up after the test is finished (e.g. deleting temporary objects).

You can put this shared code in a common place by overriding the virtual functions TestSuite::setUp() and TestSuite::tearDown(). The setUp() function is called before each test, and tearDown() is called after each test.

For example, the following test suite employs setUp() and tearDown() methods to allocate and deallocate memory for a string buffer:

Note that test assertions cannot be used in within the setUp() or tearDown() methods. The scope of these methods is outside any test case, so an assertion failure does not have a clear semantics. Similarly, test assertions cannot be used in world setup and teardown or in createSuite() or destroySuite() methods for for dynamicly created test suites (see below).

CxxTest test fixtures can also be customized during the construction and deconstruction of test suites. By default, CxxTest test suites are instantiated statically in the test runner. However, dynamically created test suites can be used to perform suite-level setup and teardown operations, verify the environment needed to execute a test suite, and construct test suites that require a nontrivial constructor.

CxxTest instantiates a test suite dynamically if the createSuite() or destroySuite() methods are defined. For example, the following test suite checks to see if it is being compiled with Microsoft Visual Studio. If not, the createSuite() returns a null pointer, indicating that the test suite was not created.

CxxTest supports two related mechanisms for performing global setup and teardown operations. Global fixtures are classes that inherit from CxxTest::GlobalFixture, and they define setUp and tearDown methods. The setUp method for all global fixtures is called before each test is executed, and the tearDown method for all global fixtures is called after each test is completed. Thus, this mechanism provides a convenient way of defining setup and teardown operations that apply to all test suites.

For example, consider the following test suite:

This test suite defines a runner that generates the following output:

Note that the global fixtures are instantiated with static global values. This ensures that these fixtures are created before the runner is initialized. Also, note that the setUp methods are called in the same sequence that the global fixtures are instantiated, and the tearDown methods are called in the reverse sequence. Finally, note that the setUp and tearDown methods in global fixtures return a boolean value, which indicates success or failure of that operation.

This example also illustrates the use of world fixtures, which perform setup and teardown operations that are executed once each when beginning and finishing tests in each test suite. World fixtures are defined with the setUpWorld and tearDownWorld methods in a global fixture.

CxxTest defines several functions that can be called in a test suite to modify the default behavior of CxxTest.

Note that the the configuration parameters are reset to their default values after each test is executed (more precisely, after tearDown() is called). Consequently, calling these functions in the setUp() function has the effect of setting that value for the entire test suite.

The CXXTEST_MOCK_GLOBAL macro is used to declare mock global functions. It is often convenient to include these declarations in a header file, which is used in both the test suite as well as the code that is being tested:

The tested code uses mock global functions, rather than using the global functions directly. You access mock functions in the T (for Test) namespace, so the tested code calls T::time() instead of time(). This is the equivalent of using abstract interfaces instead of concrete classes.

A source file needs to be defined that implements T::time() by calling the real global function. This definition is performed automatically by defining CXXTEST_MOCK_REAL_SOURCE_FILE before the header file is defined:

This source file is not used for testing, but instead it supports normal use of the tested code.

Similarly, a source file needs to be defined that implements T::time() by calling the mock global function. This definition is performed automatically by defining CXXTEST_MOCK_TEST_SOURCE_FILE before the header file is defined:

A mock object for the time() function is created using the T::Base_time class, which is automatically created by CxxTest. This class includes a time() method whose API is the same as the global time() function. Thus, this method can be defined to have whatever behavior is desired during testing. For example, the following example defines a mock object that increments a counter to define an incremental value for time().

Note that CxxTest uses global data to associate calls made with T::time() to calls to MockObject::time(). The MockObject class simply needs to be instantiated prior to the call to T::time().

The cxxtestgen command is used to create a test runner with mock functions in a normal manner:

The test runner source file, runner.cpp, needs to be compiled an linked to the mock function definition, time_mock.cpp, as well as the code being tested, rand_example.cpp:

This generates a test runner that generates the following output:

The CXXTEST_MOCK_VOID_GLOBAL is used to define mock global functions that return void. This is identical to CXXTEST_MOCK_GLOBAL except that it does not specify the return type. Take a look in sample/mock/T/stdlib.h for a demonstation.

During testing it is sometimes necessary to call the real global function instead of the mock global function. CxxTest allows a user to do this by creating a special mock object. For a global mock function of time(), the object T::Real_time represents the real function. If this class is created, then T::time() will be redirected to the real function.

Sometimes the tested code calls functions that are not available when testing. For example, this can happen when testing driver code that calls kernel functions that are not available to a user-mode test runner. CxxTest can provide mock global function definitions for the test code while using the original functions in the tested code.

The CXXTEST_SUPPLY_GLOBAL and CXXTEST_SUPPLY_VOID_GLOBAL macros are used to provide mock global function definitions. For example, the following declaration creates a mock global function for the Win32 kernel function IoCallDriver:

The tested driver code calls IoCallDriver() normally; there is no need for the T:: syntax. The test suite is defined using the T::Base_IoCallDriver as with normal mock objects.

CxxTest also provides the macros CXXTEST_SUPPLY_GLOBAL_C and CXXTEST_SUPPLY_GLOBAL_VOID_C that declare the functions with C linkage (i.e., using extern "C"). These macros are used to declare function prototypes, since you may not be able to include the header files in the test suite that are associated with the mock global function.

The CXXTEST_MOCK macro is used to declare a mock global function that is associated with a function in a namespace, including static class member functions. For example, consider the function bool Files::FileExists( const String &name ); the namespace Files contains the function FileExists. The mock class will be called T::Base_Files_FileExists and the function to implemented would be fileExists. The CXXTEST_MOCK macro declares this mock global function as follows:

Similarly, the CXXTEST_MOCK_VOID macro is used to declare a mock global function that returns void.

The CXXTEST_SUPPLY and CXXTEST_SUPPLY_VOID macros are used to provide mock global function definitions for nonexistent functions. For example:

Similarly, the CXXTEST_SUPPLY_C and CXXTEST_SUPPLY_VOID_C macros declare the functions with C linkage.

The CXXTEST_MOCK and CXXTEST_MOCK_VOID macros have a flexible interface that can provide mock global function definitions for overloaded functions. The arguments simply need to specify different mock class names, mock member names and different prototype definitions. These different mock declarations will generate different mock objects that can be explicitly referenced in a test suite.

The default namespace for mock functions is T::. This namespace can be changed by defining the CXXTEST_MOCK_NAMESPACE macro.

The script is located at build_tools/SCons/cxxtest.py. You need SCons to be able to find this file, so do one of the following:

This builder assumes that tests have a different suffix than other files in your project (by default: .t.h, configure via CXXTEST_SUFFIX). This isn’t a bad idea to begin with, since test "header files" are not really header files in the traditional sense. This is how it separates files it should run through cxxtestgen from the ones it should not.

By default, you build and run the tests by issuing the following command:

Of course, the actual name of the target is configurable (CXXTEST_TARGET).

This section describes what you do in your SConstruct file in order to be able to use the builder.

First, you must tell the environment that you want to use the cxxtest tool and how it should set itself up.

The easiest way to set the environment up is to pray the defaults are ok. The builder does some autodetection and tries its best to figure everything out. In the event this works, setting up the environment is as easy as telling SCons it should use the cxxtest tool.

If this doesn’t work, the builder tries its best to tell you what went wrong. In most cases, it just wasn’t able to find where you installed cxxtest. You can fix this like so (assuming you have CxxTest in #/extern_libs/cxxtest/):

It is advisable to pass most options to the builder at creation time, although the CxxTest call can also accept most of them if you need to do some weird thing with a particular test.

You can use the builder the same way you would the Program builder.