feat : Add tests for ONNX models test suite and fix some operators
Context
Note this MR depends upon merge of MR !83 (merged)
Update test utils to support ONNX models test suite in addition to Nodes test suite.
Add import and import+forward tests from ONNX models test suite, actually containing:
- light_bvlc_alexnet
- light_densenet121
- light_inception_v1
- light_inception_v2
- light_resnet50
- light_shufflenet
- light_squeezenet
- light_vgg19
- light_zfnet512
The MR also contains additional operator fixes:
- fix ReduceMean and Gemm import
- implement Pad and Sum with limitations
As the aidge scheduler is quite long and the run of the models take several minutes, the
test_onnx_models_import_forward
tests are skipped by default. The import tests (w/o execution) are active though.
Run the full tests including models forward with envvar TEST_ONNX_ALL=1
as in:
# do not skip models forward test
> env TEST_ONNX_ALL=1 pytest -v test_onnx_models_import_forward.py
...
# or for the full unit tests tests
> env TEST_ONNX_ALL=1 pytest
...
Here is the current result when running the new test files (note that by default forward is skipped):
$ pytest -rXF test_onnx_models*.py
test_onnx_models_import_forward.py sssssssss [ 50%]
test_onnx_models_import.py x.x.....x [100%]
=========================== short test summary info ============================
XFAIL test_onnx_models_import.py::test_onnx_import_models[light_bvlc_alexnet] - LRN not supprted
XFAIL test_onnx_models_import.py::test_onnx_import_models[light_inception_v1] - LRN not supprted
XFAIL test_onnx_models_import.py::test_onnx_import_models[light_zfnet512] - LRN not supprted
=================== 6 passed, 9 skipped, 3 xfailed in 1.56s ====================
And with all tests (takes ~= 1 min):
$ env TEST_ONNX_ALL=1 pytest -rXF test_onnx_models*.py
test_onnx_models_import_forward.py xxxx.xx.x [ 50%]
test_onnx_models_import.py x.x.....x [100%]
=========================== short test summary info ============================
XFAIL test_onnx_models_import_forward.py::test_onnx_import_models_forward[light_bvlc_alexnet] - LRN not supprted
XFAIL test_onnx_models_import_forward.py::test_onnx_import_models_forward[light_densenet121] - conv shape mismatch
XFAIL test_onnx_models_import_forward.py::test_onnx_import_models_forward[light_inception_v1] - LRN not supprted
XFAIL test_onnx_models_import_forward.py::test_onnx_import_models_forward[light_inception_v2] - concat input shapes mismatch
XFAIL test_onnx_models_import_forward.py::test_onnx_import_models_forward[light_shufflenet] - convdepthwise shape mismatch
XFAIL test_onnx_models_import_forward.py::test_onnx_import_models_forward[light_squeezenet] - output shape mismatch
XFAIL test_onnx_models_import_forward.py::test_onnx_import_models_forward[light_zfnet512] - LRN not supprted
XFAIL test_onnx_models_import.py::test_onnx_import_models[light_bvlc_alexnet] - LRN not supprted
XFAIL test_onnx_models_import.py::test_onnx_import_models[light_inception_v1] - LRN not supprted
XFAIL test_onnx_models_import.py::test_onnx_import_models[light_zfnet512] - LRN not supprted
=================== 8 passed, 10 xfailed in 79.97s (0:01:19) ==================
Modified files
Ref to the last 5 commits (later commits are from MR !83 (merged)):
- [Pad] Add onnx import for Pad
- [ReduceMean] Fix invalid axes input value
- [Gemm] Import error with non constant bias/weight
- [Sum] Add import of ONNX Sum operator
- [Tests] Add ONNX light models test suite
Detailed major modifications
Add new test files for import and import+forward of ONNX light models testsuite:
test_onnx_models_import.py
test_onnx_models_import_forward.py
TODO
-
should merge MR !83 (merged) first