Skip to content

Enhance dropout

Houssem ROUIS requested to merge enhance_dropout into dev

Context

This Merge Request adds full support for the ONNX Dropout operator (opset ≥ 12), based on the ONNX specification. It introduces three key features:

  1. training_mode input (optional)

    • A boolean scalar input that controls whether dropout is applied (True) or bypassed (False).
    • If omitted, it defaults to inference mode (False), meaning dropout is not applied.
    • Enables dynamic control of training vs. inference behavior at runtime.
  2. mask output (optional second output)

    • A boolean tensor of the same shape as the input.
    • Indicates which elements were retained (True) or dropped (False) during dropout.
    • Useful for debugging, backpropagation, or custom downstream logic.
  3. seed attribute (optional)

    • A fixed integer value used to seed the random number generator for dropout.
    • Ensures reproducibility of the dropout mask across runs when provided.
    • If omitted, a random seed is used (non-deterministic behavior).

resolves issue aidge_core#321

Implementation Details

  • Added parsing and validation for the optional training_mode input:

    • Defaults to False (inference mode) if not provided
    • Enables dropout logic when training_mode=True
  • Implemented generation of the mask tensor when the second output is requested:

    • Mask is a boolean tensor indicating retained elements
    • Same mask is used to compute the final output
  • Supported the optional seed attribute for deterministic behavior:

    • When provided, it seeds the dropout RNG
    • Guarantees reproducible masks across runs
  • Updated operator registry and unit tests accordingly

Edited by Houssem ROUIS

Merge request reports

Loading