[Unit Tests] Speedup python unit tests and Add some foundation for factorized aidge_core.testing module between components
Context
This MR improves the speed of the aidge_core python unit tests test_export.py
which always recompiled the whole export even when no change.
This allows to speedup greatly uni testing when we re-run the unit tests several times:
- Without this MR
- first python unit tests run: 36 secs
- subsequent runs: 32 secs
- With this MR:
- first python unit tests run: 36 secs
- subsequent runs: 3 secs
Hence a factor of x10 when retesting if the export part has not changed. There is a great advantage of keeping unit tests fast in order to not discourage local and frequent runs of unit tests.
Additionally, the context of this MR is in a work to provide some common python testing tools for aidge_core and also other aidge components (aidge_export_cpp, aidge_backend_cpu, etc).
As for the particular case of the MR (speedup of test_export unit test) I add to develop some file tree utilities functions, I created thus a new submodule:
-
aidge_core.testing
: supposed to rassemble common test cases and utilities for unit testing-
aidge_core.testing.utils
: common testing utils functions/classes- contributed for the purpose of this MR some reusable functions:
tree_remove
,tree_move
,tree_update_cache
- contributed for the purpose of this MR some reusable functions:
-
Refer below and the commits messages/changes for the commits details
Modified files
Commits in order:
- Fix sources of non-determinism in tests:
- [Export] Make src/dnn.c generation deterministic: improved the
aidge_export_aidge.export
function to be deterministic when generating .cpp files. This is necessary to keep incremental build, otherwise each time the same graph is re-exported, the generated code is different (due to the use of python set). It is simply using list + make unique instead of a set. - [Tests] Set test_export inputs fixed: Fix determinism of test for export by fixing the random seed before calling filler
- [Export] Make src/dnn.c generation deterministic: improved the
- Add aidge_core.testing module and aidge_core.testing.utils submodule:
- [Tests] Add aidge_core.testing module and file tree utilities: add modules and file tree/cache utilities
- Improve test speed to test_export:
- [Tests] Improve python unit_tests speed for test_export: use file tree cache tools from the above commit to enable iterative compilation and speedup re-execution of the same test by x10.
- Reduce export test problem size: this reduces test time by reducing the compilation time of large parameters arrays, which is not really necessary for testing the export
Detailed major modifications
The major modification is an additional exposed submodule:
aidge_core.testing
-
aidge_core.testing.utils
which provides some starting point with functions:tree_remove
,tree_move
,tree_update_from_cache
In the near future, the aim is to add to aidge_core.testing
other submodules for sharing predefined operator use cases for instance .operators
(used by all of aidge_core, aidge_export_, aidge_backend_ for their unit tests), or graphs uses cases for instance .networks
(used by the previously mentioned modules but also aidge_onnx for instance).
TODO
-
Possible discussions with reviewers on aidge_core.testing
choice and directions