Testing in C++

Motivation

Testing in software development is the process of evaluating and validating a software system to ensure its correctness, functionality, and performance. Automated testing is especially important for large projects.

The modular development of large projects breaks codes into smaller modules and units such as functions, and classes. Within in this context, Unit Testing and Integration Testing are extremely effective. Both of them are designed to be automated with the help of testing frameworks.

• Large projects are hard to test manually.

• Automated testing is more efficient and reliable.

• Automated testing is more scalable.

• Tests can describe the functionality of the system (requirements).

• Tests can serve as documentation.

• Testable code is usually modular and well-organized.

Unit test

It is a type of testing strategy to test software units individually in isolation. To ensure isolation, techniques like mocking, stubbing, and faking are introduced.

Integration test

It is a type of testing strategy to test how a group of units works together. Isolation can also be reinforced, similar to unit testing.

System test

It is a type of testing strategy to test the whole software system as a whole.

Acceptance test

It is a final system test that runs the whole software system all together in order to decide if the project can be accepted as a finished work.

Testing framework

A library/software to facilitate the testing process.

Simple Test Run

As a beginner taking entry-level courses, you should learn testing as soon as possible to build a correct workflow on project. You can start writing tests by simply add cout and if statements to check whether your function or class is doing its job as expected. These codes can be kept in separate cpp files with its own main function or as functions in your driver file, such as main.cpp.

For example, assume that we have two classes StoreItem and Store to test. A test driver may look like this:

test-driver.cpp

#include <iostream>

#include "store.hpp"
#include "store-item.hpp"

using namespace std;

void testStoreItem() {
  StoreItem item1("Apple");
  item1.print();
  StoreItem item2("apple");
  item2.print();
  if (item1.equals(item2))
    cout << "They are equal as expect!\n";
  else
    cout << "They are not equal, wrong!\n";
}

void testStore() {
  Store myStore("Test Store");
  myStore.addItem(StoreItem("Apple"));
  myStore.addItem(StoreItem("Banana"));

  myStore.print();
  if (myStore.size() == 2)
    cout << "The size is correct";
  else
    cout << "The size is wrong";
}

int main() {
  testStoreItem();
  // testStore();
  return 0;
}

Assertions

The assert macro function (similar to a function when used) allows a more efficient way to run tests. This macro takes a bool parameter and abort your program when the bool expression evaluates to false. You can add as many as you like in your program during debugging and disable all of them by adding a line of #define NDEBUG before the #include <cassert> directive.

test.cpp

#include <iostream>
#include <cassert>

#include "store.hpp"
#include "store-item.hpp"

using namespace std;

void testStoreItem() {
  StoreItem item1("Apple");
  StoreItem item2("apple");
  assert(item1.equals(item2));
  item1.print();
  item2.print();
}

void testStore() {
  Store myStore("Test Store");
  myStore.addItem(StoreItem("Apple"));
  myStore.addItem(StoreItem("Banana"));
  assert(myStore.size() == 2);
  myStore.print();
}

int main() {
  testStoreItem();
  // testStore();
  return 0;
}

Testing Frameworks

Choosing a good testing framework is more effective in terms of easier test writing and creation, better test organization, and better test result presentations. We choose the Catch2 framework to write tests in some projects. It is simple but feature-rich. The syntax to write tests are simple:

test-catch.cpp

#include <iostream>
#include <cassert>

#include "catch.hpp"
#include "store.hpp"
#include "store-item.hpp"

using namespace std;

TEST_CASE("Test Store class") {
  Store myStore("Test Store");
  myStore.addItem(StoreItem("Apple"));
  myStore.addItem(StoreItem("Banana"));
  REQUIRE(myStore.size() == 2);
  // CHECK(myStore.size() == 2);
}

TEST_CASE("Test StoreItem class") {
  StoreItem item1("Apple");
  StoreItem item2("apple");
  REQUIRE(item1.equals(item2));
  // CHECK(item1.equals(item2));
}

The TEST_CASE macro defines a test case with a description and a tag string. The REQUIRE macro will check the condition in the parenthesis. It will pass if the condition is true. The whole testing process will be aborted on false. Replace REQUIRE with CHECK if you want the testing process to continue on false. Thus, REQUIRE behaves like assert and is good for debugging so we can focus on the first error. On the contrast, CHECK is good for summarizing or grading when the overall passing rate is important. You will see CHECK in the testing cases in our projects rather than REQUIRE.

A comprehensive official tutorial is available online.

Testability

Some codes are easier to test than others. In most of the cases, a more testable code base has better quality than something not very testable. Testable codes are usually well-designed.

There are many principles in how to write testable codes. Here are some simple rules we can follow:

1. keep the size of each unit small

2. keep each unit coherent

3. keep each unit focusing on one responsibility

4. keep the units decoupled (each unit knows minimal numbers of other units)

5. keep units deterministic (same inputs always give the same results)

6. keep most units side effect free if possible