Using WxGuiTesting

General

There are four sorts of test which WxGuiTesting handles:

• Simple Unit Testing (No GUI interaction required, nor GUI facilities required by the components being tested)
• Complex Unit Testing (GUI interaction or facilities needed)
• Whole Application Testing
• GUI Interaction event capturing

In the first case WxGuiTesting is just fronting for CppUnit. You can use the UnitTestApp.cpp from the sample directory for your testrunner and write ordinary CppUnit test suites, registering them in the usual way, or you could just ignore WxGuiTesting altogether.

The other categories require WxGuiTesting to provide a wxWidgets application environment to run the test cases. Copy CppTextTestApp.cpp from the sample directory to your project and comment/uncomment the various statments according to your needs.

CppUnit is designed to run short test cases which abort immediately on any failed test. This can be a bit limiting, particularly when using a test for diagnosing a problem. It's therefore advisable to write a lot of short test cases instead of a few big ones: More gets tested that way.

Complex Unit Testing

In unit testing, your test suite sets up the component(s) to test in setUp() and destroys them in tearDown(). Each test-case uses either direct calls on a component's functions (like in the simple unit testing above) or uses wxTst::WxGuiTestEventSimulationHelper to send events to it.

Be sure to leave the line

MyApp* myApp = new MyApp ();

commented out in CppTextTestApp.cpp, and don't include your application class in the build. This means that components to be tested must be in header and implementation files separate from the ones holding your application declaration and implementation

Write some test classes. Each test header file will need

#include <cppunit/extensions/HelperMacros.h>

and each class will have the following signature:

class TestSuite : public CppUnit::TestFixture {
    CPPUNIT_TEST_SUITE( TestSuite );
    CPPUNIT_TEST( testCase1 );
    CPPUNIT_TEST( testCase2 );
    //And so on for every test case member function
    CPPUNIT_TEST_SUITE_END();

public:
    virtual void setUp();
    virtual void tearDown();
    void testCase1();
    void testCase2();
};

The class implementations will need the following includes as noted:

#include <wxGuiTest/wxGuiTestHelper.h>
#include <wxGuiTest/wxGuiTestEventSimulationHelper.h>
#include <wxGuiTest/wxGuiTestTempInteractive.h>

If you're automatically killing off modal dialog boxes, you'll need

#include <wxGuiTest/wxGuiTestTempInteractive.h>

For capturing, you need

#include <wxGuiTest/swCRCapture.h>

The implementation (.cpp) file for the test classes needs a line like

CPPUNIT_TEST_SUITE_NAMED_REGISTRATION( TestSuite, "WxGuiTest" );

for each test class. Substitute the class name for TestSuite; note that WxGuiTest is hard-coded into the library; if you use any other string, the test suite won't execute.

The classes and test case functions can be named anything you like, and you can have as many test cases as you need. setUp() and tearDown() are mandatory member functions which are called before and after each test case respectively to prepare the fixture for testing and to clean it up afterwards. If you're testing a whole app, they needn't do anything... and may cause trouble if they do.

Setting Up Application Testing

Define USE_WXGUITESTING in the compiler command line (add it to CXXFLAGS or something similar in your Makefile. This macro is used to redefine IMPLEMENT_APP() so that the test runner is in control for testing a whole application. You can use it to toggle other test features in your code, as we'll see.

For testing a whole application, your wxApp class (the one with OnInit()) must be derived from swTst::WxGuiTestApp, like this:

#ifdef USE_WXGUITESTING
#include <wxGuiTest/wxGuiTestApp.h>
class MyApp : public swTst::WxGuiTestApp
#else
#include <wx/app.h>
class MyApp : public wxApp
#endif

then, in your wxApp implementation,

#include <wx/wx.h>
#ifdef USE_WXGUITESTING
#include <wxGuiTest/wxGuiTestHelper.h>
#endif
IMPLEMENT_APP(MyApp)

Obviously, its code must be included in the build, and you'll uncomment the

MyApp* myApp = new MyApp ();

line in CppTextTestApp.cpp

Otherwise, everything else is the same as for complex unit testing.

GUI Event Capture

If you're capturing, the testcase needs only invoke

CAPTURE

the library will take care of the rest. Otherwise, setup is exactly the same for complex unit testing or application testing, depending on which environment you need for capturing.

Create a target in your build environment for your test and you're ready to go!

After capturing, your capture test case file foo.cpp will have a new version, foo.cpp.cr1 (the number increments every time you run capture). You can rename the foo.cpp to something else (you'll want to name it back before running capture again!) and rename foo.cpp.cr1 (or whichever) to foo.cpp and recompile. (Alternatively, you could just rename foo.cpp.cr1 to something else, like replay.cpp and create a new build target for that.) That's now a regression test replaying your capture. Cool, huh?

CppUnit is designed to abort the test case when an assert fails. This means that you want to write test cases where each of the tests is dependent on the ones before it. Independent tests should be separated into multiple test cases. The problem here is that Capture isn't smart enough to do that, so you have to break up the single test case after you've completed a capture.

Writing your program to be testable:

Programs have to be written in such a way that they're testable. For example, if an event handling function creates a frame on the stack, the details of that frame will get recorded during capture, but the test case will never be able to see it because the frame will be destroyed when the handler returns. This will cause the test case to fail. To be testable, GUI elements must be created on the heap and must be available to the test functions.

Blocking GUI Elements (Modal Dialogs and Popup Menus):

Note very well the discussion about blocking in Reinhold's Thesis, section 1.5.4. He's not quite right about the internals here: They're implemented in the platform-dependent code (that is, there's an implementation for each platform, and generally use the appropriate platform features to implement them. The problem is that that makes it impossible for wxGuiTesting, being separate from the wxWidgets library, to reach inside those implementations and manipulate the GUI elements. In short, the use of modal dialogs and popup menus requires user intervention during the test. For fully automatic testing, your code will have to disable blocking elements during testing. The USE_WXGUITESTING macro is a useful test since you have to define it anyway:

#ifndef  USE_WXGUITESTING
    msgDlg.ShowModal();
#endif

Known problems:

There are a large number of controls which don't have event simulators or capture event handlers.

Every time you visit the menu, capture redeclares topFrame and menuBar, so recompiling will produce a bunch of errors. This isn't necessarily bad, because it makes it easier to start a new test case (see the next item).

If you don't disable dialog boxes and popups, the capture function will record events for them... but the resulting test won't be able to see them, so it will fail.

A side effect of the fact that the application is created by the test framework which in turn runs your application under test-runner control is that the application name defaults to "bla" (hard-coded in InitWxGuiTestSetUp.cpp, line 104. If this matters to you, set it in your app's OnInit() function, perhaps guarded by

     #ifdef USE_WXGUITESTING. 

When you have a bunch of dialog boxes with similar controls on them, Capture seems always to use the first one it finds. You'll have to figure out what is the right one in each case in the capture file and hand-correct it.

Note that capture results may not be portable. For example, in wxMac and wxWin32, menu names may include a '&' to indicate which letter-key is used to select the item from the keyboard, while in wxGTK a '_' is used for the same purpose. You can see one way to get around this in the sample.

There's an interesting side effect on wxGTK when testing a whole application: The CppUnit CompilerOutputter won't write to std::cerr. Use std::cout instead. This doesn't appear to affect unit testing.