Add property-based tests to exporter
Context
Current tests are mostly written by hand with some expertise required to identify corner cases and assess where failure is expected, especially regarding broadcasting of dimensions and values over dimensions. While a number of issues have been identified that way, it is both time consuming, error prone, and creates numerous test cases.
The shift to property-based testing would allow the automated generation of test cases, their evaluation, and the memoization of failures where required, with minimal user input. It does however require the definition of strategies to generate the tested components of the framework, and the definition of properties to validate for each.
Property-based testing is supported by the Hypothesis Python framework.
Modified files
-
Supporting property-based testing
-
test/strategies/*.py, to add strategies to generate various operators (nodes.py), and tensors (core.py) -
test/test_strategies.py, to add test for strategies ensuring the generated elements are valid under AIDGE rules -
test/operators/*.py, to add property-based tests for specific operators (Add,FC,MatMul,Conv2D) -
test/export_acetone*.py, removed when replaced by property-based tests -
test/utils.py, to generalise tests comparing the AIDGE CPU backend and exporters -
test_export_acas.py, to use new test utils
-
-
Supporting named tensors and better templates:
-
acetone/generator.pyandoperators.pyandexport_registry.pyto add support for named tensors in code templates (matching the SONNX specification where applicable (and applied)) -
conv2d.jinjaandmatmul_forward.jinjato use named tensors -
conv2d.jinjaandmatmul_forward.jinjaanddims.jinjato use multi-dimensional array and simplify index computation (where applicable)
-
-
Miscellaneous project management
- Module
__init__.py. to cleanup extraneous imports
-
pyproject.tomlandversion.txt, to bump dependencies and module version, and fix project description issues -
.gitgnore, to exclude hypothesis data from version control - Removed unused files leftover from the AIDGE module template `
MakeLists.txt` - Misc fixes for typing and linter issues
- Module
Detailed major modifications
- Added property-based testing support for (most) supported nodes
- Added support for named tensors in templates
- Aligned (some) template with the proposed SONNX specification
To be improved
- Some operators still rely on the AIDGE
attributesdirectory (e.g.Add) - Find better strategies to generate values in tensors
- Uniform strategy for
weightsandbiasgeneration