Include the management of full models
Description
Extend model integration in the benchmarking system to support fully specified models — those defined independently of operation-based configuration files. This includes models provided in standard formats such as ONNX or AIDGE (produced by aidge_export_aidge), along with their associated input specifications.
- Supported Formats:
-
ONNX - Model file
- Input specs (optional or inferred)
-
AIDGE - Model file
- Input specs (as exported by aidge_export_aidge)
-
These models must be compatible with the BenchmarkScheme
tree structure, allowing seamless integration alongside configuration defined tests.
Example of API
# Load existing benchmarking configuration
B, _ = BenchmarkScheme.from_config("conv2d.json")
# Create a new model configuration
new_config = ModelWithSpecs("path/to/model", format="onnx") # contains path to support lazy loading
# Insert into the benchmark tree under a label
B["a-label.a-sub-label"] = new_config
# Access model and input specifications
model = new_config.model # Triggers lazy loading
# or manually trigger loading beforehand:
# new_config.load_model()
input_specs = new_config.input_specs # Inferred or provided explicitly
for i in input_specs:
i.init_values(dtype=np.float32) # Initialize inputs
Other specifications from prototype interface
- Implement
ModelWithSpecs
class or similar abstraction-
Support lazy and manual loading -
Handle format-specific parsing -
Extend compatibility with BenchmarkScheme -
Allow user-provided or inferred input specifications -
Write unit tests for ONNX and AIDGE use cases -
Document usage
-