Add property-based tests to exporter
Context
Current tests are mostly written by hand with some expertise required to identify corner cases and assess where failure is expected, especially regarding broadcasting of dimensions and values over dimensions. While a number of issues have been identified that way, it is both time consuming, error prone, and creates numerous test cases.
The shift to property-based testing would allow the automated generation of test cases, their evaluation, and the memoization of failures where required, with minimal user input. It does however require the definition of strategies to generate the tested components of the framework, and the definition of properties to validate for each.
Property-based testing is supported by the Hypothesis Python framework.
Modified files
-
Supporting property-based testing
-
test/strategies/*.py
, to add strategies to generate various operators (nodes.py
), and tensors (core.py
) -
test/test_strategies.py
, to add test for strategies ensuring the generated elements are valid under AIDGE rules -
test/operators/*.py
, to add property-based tests for specific operators (Add
,FC
,MatMul
,Conv2D
) -
test/export_acetone*.py
, removed when replaced by property-based tests -
test/utils.py
, to generalise tests comparing the AIDGE CPU backend and exporters -
test_export_acas.py
, to use new test utils
-
-
Supporting named tensors and better templates:
-
acetone/generator.py
andoperators.py
andexport_registry.py
to add support for named tensors in code templates (matching the SONNX specification where applicable (and applied)) -
conv2d.jinja
andmatmul_forward.jinja
to use named tensors -
conv2d.jinja
andmatmul_forward.jinja
anddims.jinja
to use multi-dimensional array and simplify index computation (where applicable)
-
-
Miscellaneous project management
- Module
__init__.py
. to cleanup extraneous imports
-
pyproject.toml
andversion.txt
, to bump dependencies and module version, and fix project description issues -
.gitgnore
, to exclude hypothesis data from version control - Removed unused files leftover from the AIDGE module template `
MakeLists.txt`
- Misc fixes for typing and linter issues
- Module
Detailed major modifications
- Added property-based testing support for (most) supported nodes
- Added support for named tensors in templates
- Aligned (some) template with the proposed SONNX specification
To be improved
- Some operators still rely on the AIDGE
attributes
directory (e.g.Add
) - Find better strategies to generate values in tensors
- Uniform strategy for
weights
andbias
generation