cpp_test

Page author: @pineapple

Unit-tests are an ESSENTIAL part of any codebase to assert the behaviour of a new feature and the consequences of any modification in the codebase. Any piece of production code that is not covered by a unit-test will probably later lead to a longer debug session or worse: undetected bad/undefined behaviours.

Table of contents

Structure of unit_test folder

Each Aidge module can be supplemented by a test directory called unit_tests with the following hierarchy:

unit_tests/
  |__<any_sub_dir_name_1>/
  |    |__<any_file_name>
  |    |...
  |__<any_sub_dir_name_2>/
  |
  ...
  |__CMakeLists.txt

An example of CMakeLists.txt file can be found in aidge_core module here

How to write a test

Aidge unit-tests are performed with the C++ test framework Catch2.

A typical test file has the following structure:

TEST_CASE("test_title", "[tag1][tag2]...[tagN]") {
  ...
  SECTION("section1_title") {
    ...
  }
  SECTION("section2_title") {
    ...
    SECTION("secton2_1_title") {
      REQUIRE(<true_false_statement>)
    }
  }
}

There can be any number of TEST_CASE and SECTION as you want. [tag1][tag2]...[tagN] are tag declaration for individual tests calls that we'll see later.

About test sections

Each SECTION of a test case will be executed in standalone, meaning that that program will start from the beginning of the TEST_CASE(and if the SECTION is nested, go through each parent SECTION).

This allows for 2 things :

1. Factorize all the common setup & teardown required for tests.
2. Creating scenarios for your TEST_CASE and test each one separately.

Why sections names are important

Tests are a place in a codebase where people go to understand how to interact with a function or an object. To help them it is advised to have explicit and verbous tests. This can be done in 2 ways :

1. with a lot of comment
2. a bit of context : having explicit section name to understand the scenario can help with that.

Create assertions

You can add assertions in your tests that should fail and stop the execution (or not) if the program behaviour is not as expected. You can find every assertions available on the official Catch2 documentation. Here are some examples:

• REQUIRE(<true_false_statement>)
• REQUIRE_THROWS(...)
• REQUIRE_NOTHROW(...)
• ...

CHECK & REQUIRE Assertions :

Each REQUIRE assertion has his CHECK equivalent. The difference lies in the fact that if a REQUIRE assertion fails, the test suite will stop with SIGABRT where for a CHECK the test suite will keep going.

Hence it is recommended to use REQUIRE only when you know that the tested condition, if not met, will impair the whole test suite. This allows for more visibility on the failures of your test suite.

Compiling unit-tests

With setup.sh

On Unix-based systems, unit-tests can be compiled using the setup.sh file left at your disposal at the root of the Aidge project. Simply add the optional flag --tests (or -t) to your compilation command.

e.g I want to compile the unit-tests of Aidge core, CPU bakcned and CUDA backend. I will run

<path_to_aidge>/setup.sh -m core -m backend_cpu -m backend_cuda --tests

Every available test in these modules will be compiled, stored in <path_to_aidge>/aidge/<module_name>/build_bundle_cpp/unit_tests/tests_<module_name>and run.

With cmake/make

example with aidge_core

1. Go in the folder of the project you want to test

   cd aidge_core

2. If you compiled the project with setup.sh you will notice a build_bundle_cpp folder. If its missing, create it and go in it.

mkdir build_bundle_cpp && cd build_bundle_cpp

3. Configure and build the project

cmake .. -DCMAKE_BUILD_TYPE=Debug -DPYBIND=0 -DTEST=1
make -j8

4. Launch the test executable, it can be found under unit_tests/tests_<project_name>

./unit_tests/tests_aidge_core

Running unit-tests

Once the test binary file has been created with the latest command, you can run it or select and run individual unit-tests with optional flags to alter the output.

Select a given test case

Select individual TEST_CASE using their user-defined tags

<path_to_bin>/tests_<module_name> [tag1][tag34]

or with wildcards of their names (example with the Unsqueeze operator) :

<path_to_bin>/tests_<module_name> *nsqu*

Select a given section within the test_case

<path_to_bin>/tests_<module_name> *nsqu* -c "section1" -c "subsection1.1"

⚠️ No need to write the whole path to the subsection you want to call, its name is enough -c "subsection1.1"

Tips & tricks

• Specify a seed for the random number generator : very useful when trying to recreate a bug.
• Time your functions with the BENCHMARK macro

Every option for test call can be found on the command line page