[upd] benchmark script

Context

Greatly improve the user interface of the benchmark. (Part of #262 (moved))

User interface

As an example, let's compare the updated benchmark.py script with its previous version. We will try to bench the Convolution operation like in the previous MR !360 (merged)

//////////////////////////////
// old command line

python aidge/aidge_core/benchmark/benchmark.py \
    --time \
    --compare-with-onnxruntime \
    --config-file ./aidge/aidge_core/benchmark/operator_config/conv2d_config.json \
    --results-directory ./aidge/aidge_core/benchmark/results/ \
    --module-to-bench aidge_backend_cpu

//////////////////////////////
// new command line

aidge_benchmark \
    --config-file conv2d.json \
    --time \
    --compare \
    --modules aidge_backend_cpu torch aidge_backend_cuda onnxruntime

Here, we have the added bonus the the old version only tested one module at once (aidge_backend_cpu in the example).

Now let's compare the outputs:

//////////////////////////////
// old output
'aidge_backend_cpu' module successfully imported
Starting tests...
▷ feature_map_size -- 10
 ├┬─Measuring kernel inference time...
 │└─[ time = 1.80e-05 ± 1.21e-06 seconds ]
 └┬─Assessing results are the same as 'onnxruntime' module...
  └─[ o ]

▷ feature_map_size -- 100
 ├┬─Measuring kernel inference time...
 │└─[ time = 7.82e-04 ± 9.67e-06 seconds ]
 └┬─Assessing results are the same as 'onnxruntime' module...
  └─[ o ]

//////////////////////////////
// new output
Loading modules...
 ├───onnxruntime [ ok ]
 ├───aidge_backend_cpu [ ok ]
 ├───torch [ ok ]
 ├───aidge_backend_cuda [ ok ]
 └───onnxruntime [ ok ]

Creating result directory at:  benchmark_results

Starting tests...
▷ feature_map_size -- 10
 ├─┬─aidge_backend_cpu
 │ ├───time [ 1.90e-05 ± 1.36e-06 ] (seconds)
 │ └───comp [ ok ]
 ├─┬─torch
 │ ├───time [ 2.54e-05 ± 1.11e-06 ] (seconds)
 │ └───comp [ ok ]
 ├─┬─aidge_backend_cuda
 │ ├───time [ 4.64e-05 ± 2.59e-06 ] (seconds)
 │ └───comp [ ok ]
 └─┬─onnxruntime
   ├───time [ 1.39e-05 ± 1.63e-06 ] (seconds)
   └───comp [ ok ]

Major modifications

• add a number of configuration files and make the configuration file directory a package directory that is scanned along the relative path to search for config files
• upd: make benchmark.py a module script to be easily accessed
• add: possibility to run the benchmark on several modules at once instead generating the ONNX for each of them. This speeds up the benchmarking time greatly
• add: a default directory benchmark_results for the benchmark instead of having a required argument. If the directory does not exist it is created
• add:
  • tree_structure.py to dislay benchmark elements in a tree-like structure
  • manage_json.py to handle json loading and scanning functions
  • logger.py to display colored outputs

Future update

Here is what remains to be done:

• Update the configuration file format to accept specific values for input Tensor. This is ESSENTIAL to test Operators such as Slice whom inputs like axis or start behave like attributes and cannot be chosen at random.
• Update the configuration file format to accept Aidge operator descriptions, not just ONNX operator descriptions
• Add an api for benchmark.py to be used in unit-tests. The main() function should be splitted into smaller functions