| ... | @@ -2,13 +2,13 @@ |
... | @@ -2,13 +2,13 @@ |
|
|
|
|
|
|
|
Unit-tests are an **ESSENTIAL** part of any codebase to assert the behaviour of a new feature and the consequences of any modification in the codebase. Any piece of production code that is not covered by a unit-test will probably later lead to a longer debug session or worse: undetected bad/undefined behaviours.
|
|
Unit-tests are an **ESSENTIAL** part of any codebase to assert the behaviour of a new feature and the consequences of any modification in the codebase. Any piece of production code that is not covered by a unit-test will probably later lead to a longer debug session or worse: undetected bad/undefined behaviours.
|
|
|
|
|
|
|
|
[TOC]
|
|
Table of contents
|
|
|
|
|
|
|
|
## Structure of unit_test folder
|
|
## Structure of unit_test folder
|
|
|
|
|
|
|
|
Each Aidge module can be supplemented by a test directory called ``unit_tests`` with the following hierarchy:
|
|
Each Aidge module can be supplemented by a test directory called `unit_tests` with the following hierarchy:
|
|
|
|
|
|
|
|
```bash
|
|
```
|
|
|
unit_tests/
|
|
unit_tests/
|
|
|
|__<any_sub_dir_name_1>/
|
|
|__<any_sub_dir_name_1>/
|
|
|
| |__<any_file_name>
|
|
| |__<any_file_name>
|
| ... | @@ -27,7 +27,7 @@ Aidge unit-tests are performed with the C++ test framework [Catch2](https://gith |
... | @@ -27,7 +27,7 @@ Aidge unit-tests are performed with the C++ test framework [Catch2](https://gith |
|
|
|
|
|
|
|
A typical test file has the following structure:
|
|
A typical test file has the following structure:
|
|
|
|
|
|
|
|
```cpp
|
|
```
|
|
|
TEST_CASE("test_title", "[tag1][tag2]...[tagN]") {
|
|
TEST_CASE("test_title", "[tag1][tag2]...[tagN]") {
|
|
|
...
|
|
...
|
|
|
SECTION("section1_title") {
|
|
SECTION("section1_title") {
|
| ... | @@ -43,62 +43,108 @@ TEST_CASE("test_title", "[tag1][tag2]...[tagN]") { |
... | @@ -43,62 +43,108 @@ TEST_CASE("test_title", "[tag1][tag2]...[tagN]") { |
|
|
```
|
|
```
|
|
|
|
|
|
|
|
There can be any number of TEST_CASE and SECTION as you want. `[tag1][tag2]...[tagN]` are tag declaration for individual tests calls that we'll see later.
|
|
There can be any number of TEST_CASE and SECTION as you want. `[tag1][tag2]...[tagN]` are tag declaration for individual tests calls that we'll see later.
|
|
|
|
|
|
|
### About test sections
|
|
### About test sections
|
|
|
|
|
|
|
Each `SECTION` of a test case will be executed in standalone, meaning that that program will start from the beginning of the `TEST_CASE`(and if the `SECTION` is nested, go through each parent `SECTION`).
|
|
Each `SECTION` of a test case will be executed in standalone, meaning that that program will start from the beginning of the `TEST_CASE`(and if the `SECTION` is nested, go through each parent `SECTION`).
|
|
|
|
|
|
|
|
This allows for 2 things :
|
|
This allows for 2 things :
|
|
|
1. Factorize all the common setup & teardown required for tests.
|
|
|
|
|
|
1. Factorize all the common setup & teardown required for tests.
|
|
|
2. Creating scenarios for your `TEST_CASE` and test each one separately.
|
|
2. Creating scenarios for your `TEST_CASE` and test each one separately.
|
|
|
|
|
|
|
> **Why sections names are important**
|
|
> **Why sections names are important**
|
|
|
>
|
|
>
|
|
|
> Tests are a place in a codebase where people go to understand how to interact with a function or an object. To help them it is advised to have explicit and verbous tests.
|
|
> Tests are a place in a codebase where people go to understand how to interact with a function or an object. To help them it is advised to have explicit and verbous tests. This can be done in 2 ways :
|
|
|
> This can be done in 2 ways :
|
|
>
|
|
|
> 1. with a lot of comment
|
|
> 1. with a lot of comment
|
|
|
> 2. a bit of context : having explicit section name to understand the scenario can help with that.
|
|
> 2. a bit of context : having explicit section name to understand the scenario can help with that.
|
|
|
|
|
|
|
|
### Create assertions
|
|
### Create assertions
|
|
|
|
|
|
|
You can add assertions in your tests that should fail and stop the execution (or not) if the program behaviour is not as expected. You can find every assertions available on the official Catch2 [documentation](https://github.com/catchorg/Catch2/blob/devel/docs/tutorial.md). Here are some examples:
|
|
You can add assertions in your tests that should fail and stop the execution (or not) if the program behaviour is not as expected. You can find every assertions available on the official Catch2 [documentation](https://github.com/catchorg/Catch2/blob/devel/docs/tutorial.md). Here are some examples:
|
|
|
|
|
|
|
- `REQUIRE(<true_false_statement>)`
|
|
- `REQUIRE(<true_false_statement>)`
|
|
|
- `REQUIRE_THROWS(...)`
|
|
- `REQUIRE_THROWS(...)`
|
|
|
- `REQUIRE_NOTHROW(...)`
|
|
- `REQUIRE_NOTHROW(...)`
|
|
|
- ...
|
|
- ...
|
|
|
|
|
|
|
> **`CHECK` & `REQUIRE` Assertions :**
|
|
> **`CHECK` & `REQUIRE` Assertions :**
|
|
|
>
|
|
>
|
|
|
>Each `REQUIRE` assertion has his `CHECK` equivalent. The difference lies in the fact that if a `REQUIRE` assertion fails, the test suite will stop with SIGABRT where for a `CHECK` the test suite will keep going.
|
|
> Each `REQUIRE` assertion has his `CHECK` equivalent. The difference lies in the fact that if a `REQUIRE` assertion fails, the test suite will stop with SIGABRT where for a `CHECK` the test suite will keep going.
|
|
|
>
|
|
>
|
|
|
> Hence it is recommended to use `REQUIRE` only when you know that the tested condition, if not met, will impair the whole test suite. This allows for more visibility on the failures of your test suite.
|
|
> Hence it is recommended to use `REQUIRE` only when you know that the tested condition, if not met, will impair the whole test suite. This allows for more visibility on the failures of your test suite.
|
|
|
|
|
|
|
|
## Compiling unit-tests
|
|
## Compiling unit-tests
|
|
|
|
|
|
|
|
On Unix-based systems, unit-tests can be compiled using the ``setup.sh`` file left at your disposal at the root of the Aidge project. Simply add the optional flag `--tests` (or `-t`) to your compilation command.
|
|
### With setup.sh
|
|
|
|
|
|
|
|
On Unix-based systems, unit-tests can be compiled using the `setup.sh` file left at your disposal at the root of the Aidge project. Simply add the optional flag `--tests` (or `-t`) to your compilation command.
|
|
|
|
|
|
|
|
e.g I want to compile the unit-tests of Aidge core, CPU bakcned and CUDA backend. I will run
|
|
e.g I want to compile the unit-tests of Aidge core, CPU bakcned and CUDA backend. I will run
|
|
|
```bash
|
|
|
|
|
|
```
|
|
|
<path_to_aidge>/setup.sh -m core -m backend_cpu -m backend_cuda --tests
|
|
<path_to_aidge>/setup.sh -m core -m backend_cpu -m backend_cuda --tests
|
|
|
```
|
|
```
|
|
|
|
|
|
|
|
Every available test in these modules will be compiled, stored in `<path_to_aidge>/aidge/<module_name>/build_bundle_cpp/unit_tests/tests_<module_name>`and run.
|
|
Every available test in these modules will be compiled, stored in `<path_to_aidge>/aidge/<module_name>/build_bundle_cpp/unit_tests/tests_<module_name>`and run.
|
|
|
|
|
|
|
|
|
### With cmake/make
|
|
|
|
|
|
|
|
> example with `aidge_core`
|
|
|
|
|
|
|
|
1. Go in the folder of the project you want to test
|
|
|
|
|
|
|
|
```bash
|
|
|
|
cd aidge_core
|
|
|
|
```
|
|
|
|
2. If you compiled the project with setup.sh you will notice a `build_bundle_cpp` folder. If its missing, create it and go in it.
|
|
|
|
|
|
|
|
```bash
|
|
|
|
mkdir build_bundle_cpp && cd build_bundle_cpp
|
|
|
|
```
|
|
|
|
|
|
|
|
3. Configure and build the project
|
|
|
|
|
|
|
|
```bash
|
|
|
|
cmake .. -DCMAKE_BUILD_TYPE=Debug -DPYBIND=0 -DTEST=1
|
|
|
|
make -j8
|
|
|
|
```
|
|
|
|
|
|
|
|
4. Launch the test executable, it can be found under `unit_tests/tests_<project_name>`
|
|
|
|
|
|
|
|
```bash
|
|
|
|
./unit_tests/tests_aidge_core
|
|
|
|
```
|
|
|
|
|
|
|
## Running unit-tests
|
|
## Running unit-tests
|
|
|
|
|
|
|
|
Once the test binary file has been created with the latest command, you can run it or select and run individual unit-tests with optional flags to alter the output.
|
|
Once the test binary file has been created with the latest command, you can run it or select and run individual unit-tests with optional flags to alter the output.
|
|
|
|
|
|
|
|
### Select a given test case
|
|
### Select a given test case
|
|
|
|
|
|
|
Select individual TEST_CASE using their user-defined tags
|
|
Select individual TEST_CASE using their user-defined tags
|
|
|
```bash
|
|
|
|
|
|
```
|
|
|
<path_to_bin>/tests_<module_name> [tag1][tag34]
|
|
<path_to_bin>/tests_<module_name> [tag1][tag34]
|
|
|
```
|
|
```
|
|
|
|
|
|
|
or with wildcards of their names (example with the `Unsqueeze` operator) :
|
|
or with wildcards of their names (example with the `Unsqueeze` operator) :
|
|
|
```bash
|
|
|
|
|
|
```
|
|
|
<path_to_bin>/tests_<module_name> *nsqu*
|
|
<path_to_bin>/tests_<module_name> *nsqu*
|
|
|
```
|
|
```
|
|
|
|
|
|
|
### Select a given section within the test_case
|
|
### Select a given section within the test_case
|
|
|
```bash
|
|
|
|
|
|
```
|
|
|
<path_to_bin>/tests_<module_name> *nsqu* -c "section1" -c "subsection1.1"
|
|
<path_to_bin>/tests_<module_name> *nsqu* -c "section1" -c "subsection1.1"
|
|
|
```
|
|
```
|
|
|
|
|
|
|
> :warning: No need to write the whole path to the subsection you want to call, its name is enough `-c "subsection1.1"`
|
|
> :warning: No need to write the whole path to the subsection you want to call, its name is enough `-c "subsection1.1"`
|
|
|
|
|
|
|
|
### Tips & tricks
|
|
### Tips & tricks
|
|
|
|
|
|
|
- [Specify a seed for the random number generator](https://github.com/catchorg/Catch2/blob/devel/docs/command-line.md#specify-a-seed-for-the-random-number-generator) : very useful when trying to recreate a bug.
|
|
- [Specify a seed for the random number generator](https://github.com/catchorg/Catch2/blob/devel/docs/command-line.md#specify-a-seed-for-the-random-number-generator) : very useful when trying to recreate a bug.
|
|
|
- Time your functions with the [BENCHMARK macro](https://github.com/catchorg/Catch2/blob/devel/docs/benchmarks.md)
|
|
- Time your functions with the [BENCHMARK macro](https://github.com/catchorg/Catch2/blob/devel/docs/benchmarks.md)
|
|
|
|
|
|
| ... | | ... | |