Skip to content

Add Normalize MetaOperator

Description

Missing normalization operation.

This operation can be decomposed as follows:

flowchart LR

input((input)):::tensor -->|"→ 0\n(B,C,H,W)"| norm["Norm\n(order = 2, dim = 1)"] -->|"0 → 0\n(B,H,W)"| unsqueeze[Unsqueeze] -->|"0 → 1\n(B,1,H,W)"| div[Div] -->|"0 →\n(B,C,H,W)"| output((output)):::tensor
input -->|"→ 0\n(B,C,H,W)"| div

classDef tensor fill:#ffbe98, color:#000

The Norm operation itself could be further down decomposed like so:

flowchart LR

input((input)):::tensor -->|"→ 0\n(B,C,H,W)"| pow["Pow\n(order = 2)"] -->|"→ 0\n(B,C,H,W)"| sum["Sum\n(dim = 1)"] -->|"0 → 0\n(B,H,W)"| pow2["Pow\n(order = 1/2)"] -->|"0 → 0\n(B,H,W)"| output((output)):::tensor

classDef tensor fill:#ffbe98, color:#000

Introduction of this feature brings the oportunity to talk about many core aidge choices that should be clarified now that the framework is more mature.

  • Operators that are special case of more generic Operators (here, Pow(order = 0.5) could be replaced with Sqrt().*)
  • Operators that are a composition of other Operators (here Norm, but it works for FC too)
  • Missing basic Operators (here: Sum). May be it is time to have a list

Proposal

  • Add Sum Operator
  • Add Normalize Operator (dicussion about "basic" Operators and MetaOperators in another issue)