Skip to content

Include the management of full models

Description

Extend model integration in the benchmarking system to support fully specified models — those defined independently of operation-based configuration files. This includes models provided in standard formats such as ONNX or AIDGE (produced by aidge_export_aidge), along with their associated input specifications.

  • Supported Formats:
    • ONNX
      • Model file
      • Input specs (optional or inferred)
    • AIDGE
      • Model file
      • Input specs (as exported by aidge_export_aidge)

These models must be compatible with the BenchmarkScheme tree structure, allowing seamless integration alongside configuration defined tests.

Example of API

# Load existing benchmarking configuration
B, _ = BenchmarkScheme.from_config("conv2d.json")

# Create a new model configuration
new_config = ModelWithSpecs("path/to/model", format="onnx")  # contains path to support lazy loading

# Insert into the benchmark tree under a label
B["a-label.a-sub-label"] = new_config

# Access model and input specifications
model = new_config.model                  # Triggers lazy loading
# or manually trigger loading beforehand:
# new_config.load_model()

input_specs = new_config.input_specs      # Inferred or provided explicitly
for i in input_specs:
    i.init_values(dtype=np.float32)       # Initialize inputs

Other specifications from prototype interface

  • Implement ModelWithSpecs class or similar abstraction
    • Support lazy and manual loading
    • Handle format-specific parsing
    • Extend compatibility with BenchmarkScheme
    • Allow user-provided or inferred input specifications
    • Write unit tests for ONNX and AIDGE use cases
    • Document usage