Skip to content

Draft: Add InstanceNorm operator for 3DUnet support

matthieu marchal requested to merge feat/3d-unet-instance-norm into dev

This MR is linked to aidge#308 and aims to add the missing operator InstanceNormalization (InstanceNorm) for 3D-Unet support.

InstanceNorm - Core Framework Implementation

Core Framework Files

  • InstanceNorm.hpp (202 lines) - Main operator header with epsilon attribute and comprehensive mathematical documentation
  • InstanceNorm.cpp (81 lines) - Operator implementation with constructor, dimension forwarding, and factory function
  • pybind_InstanceNorm.cpp (128 lines) - Python bindings with LaTeX documentation and parameter defaults
  • Test_InstanceNorm_Op.cpp (169 lines) - Unit tests for attributes, dimension handling, and functionality

Core Operator Architecture

Header Implementation (InstanceNorm.hpp)

  • Attribute System: StaticAttributes<InstanceNormAttr> with single Epsilon parameter
  • Class Definition: InstanceNorm_Op inherits from OperatorTensorWithImpl<InstanceNorm_Op>
  • Documentation: Extensive LaTeX formulations for forward/backward passes
  • API Design: Epsilon getter and factory function with learnable parameters

Source Implementation (InstanceNorm.cpp)

  • Constructor: Epsilon initialization with three input categories (Data, Param, Param)
  • Dimension Forwarding: Multi-dimensional tensor shape inference and parameter validation
  • Factory Function: Creates nodes with attached scale/bias parameters via addProducer()

Python Bindings (pybind_InstanceNorm.cpp)

  • Class Binding: Python class with attribute access and comprehensive docstrings
  • Function Binding: Factory with defaults (nb_features, epsilon=1e-5, name="")
  • Registration: Automatic operator registration for Python interface

Unit Testing (Test_InstanceNorm_Op.cpp)

  • Coverage: 3D/4D tensors, constructors, attributes, factory function, and I/O names
  • Validation: Epsilon handling, dimension forwarding, and parameter attachment

Key Implementation Details

  • Single Attribute: Epsilon (float, default 1e-5) for numerical stability
  • Input Structure: data_input, scale (γ), bias (β) - latter two are learnable per-channel parameters
  • Normalization: Per-instance, per-channel across spatial dimensions
  • Mathematical Support: Complete forward/backward formulations with gradient computation

External References

Edited by matthieu marchal

Merge request reports

Loading