|
|
|
**Page author:** @pineapple
|
|
|
|
|
|
|
|
Unit-tests are an **ESSENTIAL** part of any codebase to assert the behaviour of a new feature and the consequences of any modification in the codebase. Any piece of production code that is not covered by a unit-test will probably later lead to a longer debug session or worse: undetected bad/undefined behaviours.
|
|
|
|
|
|
|
|
Table of contents
|
|
|
|
|
|
|
|
## Structure of unit_test folder
|
|
|
|
|
|
|
|
Each Aidge module can be supplemented by a test directory called `unit_tests` with the following hierarchy:
|
|
|
|
|
|
|
|
```
|
|
|
|
unit_tests/
|
|
|
|
|__<any_sub_dir_name_1>/
|
|
|
|
| |__<any_file_name>
|
|
|
|
| |...
|
|
|
|
|__<any_sub_dir_name_2>/
|
|
|
|
|
|
|
|
|
...
|
|
|
|
|__CMakeLists.txt
|
|
|
|
```
|
|
|
|
|
|
|
|
An example of `CMakeLists.txt` file can be found in aidge_core module [here](https://gitlab.eclipse.org/eclipse/aidge/aidge_core/-/blob/main/unit_tests/CMakeLists.txt?ref_type=heads)
|
|
|
|
|
|
|
|
## How to write a test
|
|
|
|
|
|
|
|
Aidge unit-tests are performed with the C++ test framework [Catch2](https://github.com/catchorg/Catch2).
|
|
|
|
|
|
|
|
A typical test file has the following structure:
|
|
|
|
|
|
|
|
```
|
|
|
|
TEST_CASE("test_title", "[tag1][tag2]...[tagN]") {
|
|
|
|
...
|
|
|
|
SECTION("section1_title") {
|
|
|
|
...
|
|
|
|
}
|
|
|
|
SECTION("section2_title") {
|
|
|
|
...
|
|
|
|
SECTION("secton2_1_title") {
|
|
|
|
REQUIRE(<true_false_statement>)
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
```
|
|
|
|
|
|
|
|
There can be any number of TEST_CASE and SECTION as you want. `[tag1][tag2]...[tagN]` are tag declaration for individual tests calls that we'll see later.
|
|
|
|
|
|
|
|
### About test sections
|
|
|
|
|
|
|
|
Each `SECTION` of a test case will be executed in standalone, meaning that that program will start from the beginning of the `TEST_CASE`(and if the `SECTION` is nested, go through each parent `SECTION`).
|
|
|
|
|
|
|
|
This allows for 2 things :
|
|
|
|
|
|
|
|
1. Factorize all the common setup & teardown required for tests.
|
|
|
|
2. Creating scenarios for your `TEST_CASE` and test each one separately.
|
|
|
|
|
|
|
|
> **Why sections names are important**
|
|
|
|
>
|
|
|
|
> Tests are a place in a codebase where people go to understand how to interact with a function or an object. To help them it is advised to have explicit and verbous tests. This can be done in 2 ways :
|
|
|
|
>
|
|
|
|
> 1. with a lot of comment
|
|
|
|
> 2. a bit of context : having explicit section name to understand the scenario can help with that.
|
|
|
|
|
|
|
|
### Create assertions
|
|
|
|
|
|
|
|
You can add assertions in your tests that should fail and stop the execution (or not) if the program behaviour is not as expected. You can find every assertions available on the official Catch2 [documentation](https://github.com/catchorg/Catch2/blob/devel/docs/tutorial.md). Here are some examples:
|
|
|
|
|
|
|
|
- `REQUIRE(<true_false_statement>)`
|
|
|
|
- `REQUIRE_THROWS(...)`
|
|
|
|
- `REQUIRE_NOTHROW(...)`
|
|
|
|
- ...
|
|
|
|
|
|
|
|
> **`CHECK` & `REQUIRE` Assertions :**
|
|
|
|
>
|
|
|
|
> Each `REQUIRE` assertion has his `CHECK` equivalent. The difference lies in the fact that if a `REQUIRE` assertion fails, the test suite will stop with SIGABRT where for a `CHECK` the test suite will keep going.
|
|
|
|
>
|
|
|
|
> Hence it is recommended to use `REQUIRE` only when you know that the tested condition, if not met, will impair the whole test suite. This allows for more visibility on the failures of your test suite.
|
|
|
|
|
|
|
|
## Compiling unit-tests
|
|
|
|
|
|
|
|
### With setup.sh
|
|
|
|
|
|
|
|
On Unix-based systems, unit-tests can be compiled using the `setup.sh` file left at your disposal at the root of the Aidge project. Simply add the optional flag `--tests` (or `-t`) to your compilation command.
|
|
|
|
|
|
|
|
e.g I want to compile the unit-tests of Aidge core, CPU bakcned and CUDA backend. I will run
|
|
|
|
|
|
|
|
```
|
|
|
|
<path_to_aidge>/setup.sh -m core -m backend_cpu -m backend_cuda --tests
|
|
|
|
```
|
|
|
|
|
|
|
|
Every available test in these modules will be compiled, stored in `<path_to_aidge>/aidge/<module_name>/build_bundle_cpp/unit_tests/tests_<module_name>`and run.
|
|
|
|
|
|
|
|
### With cmake/make
|
|
|
|
|
|
|
|
> example with `aidge_core`
|
|
|
|
|
|
|
|
1. Go in the folder of the project you want to test
|
|
|
|
|
|
|
|
```bash
|
|
|
|
cd aidge_core
|
|
|
|
```
|
|
|
|
2. If you compiled the project with setup.sh you will notice a `build_bundle_cpp` folder. If its missing, create it and go in it.
|
|
|
|
|
|
|
|
```bash
|
|
|
|
mkdir build_bundle_cpp && cd build_bundle_cpp
|
|
|
|
```
|
|
|
|
|
|
|
|
3. Configure and build the project
|
|
|
|
|
|
|
|
```bash
|
|
|
|
cmake .. -DCMAKE_BUILD_TYPE=Debug -DPYBIND=0 -DTEST=1
|
|
|
|
make -j8
|
|
|
|
```
|
|
|
|
|
|
|
|
4. Launch the test executable, it can be found under `unit_tests/tests_<project_name>`
|
|
|
|
|
|
|
|
```bash
|
|
|
|
./unit_tests/tests_aidge_core
|
|
|
|
```
|
|
|
|
|
|
|
|
## Running unit-tests
|
|
|
|
|
|
|
|
Once the test binary file has been created with the latest command, you can run it or select and run individual unit-tests with optional flags to alter the output.
|
|
|
|
|
|
|
|
### Select a given test case
|
|
|
|
|
|
|
|
Select individual TEST_CASE using their user-defined tags
|
|
|
|
|
|
|
|
```
|
|
|
|
<path_to_bin>/tests_<module_name> [tag1][tag34]
|
|
|
|
```
|
|
|
|
|
|
|
|
or with wildcards of their names (example with the `Unsqueeze` operator) :
|
|
|
|
|
|
|
|
```
|
|
|
|
<path_to_bin>/tests_<module_name> *nsqu*
|
|
|
|
```
|
|
|
|
|
|
|
|
### Select a given section within the test_case
|
|
|
|
|
|
|
|
```
|
|
|
|
<path_to_bin>/tests_<module_name> *nsqu* -c "section1" -c "subsection1.1"
|
|
|
|
```
|
|
|
|
|
|
|
|
> :warning: No need to write the whole path to the subsection you want to call, its name is enough `-c "subsection1.1"`
|
|
|
|
|
|
|
|
### Tips & tricks
|
|
|
|
|
|
|
|
- [Specify a seed for the random number generator](https://github.com/catchorg/Catch2/blob/devel/docs/command-line.md#specify-a-seed-for-the-random-number-generator) : very useful when trying to recreate a bug.
|
|
|
|
- Time your functions with the [BENCHMARK macro](https://github.com/catchorg/Catch2/blob/devel/docs/benchmarks.md)
|
|
|
|
|
|
|
|
Every option for test call can be found on the [command line page](https://github.com/catchorg/Catch2/blob/devel/docs/command-line.md) |
|
|
\ No newline at end of file |