Add Normalize MetaOperator
Description
Missing normalization operation.
This operation can be decomposed as follows:
flowchart LR
input((input)):::tensor -->|"→ 0\n(B,C,H,W)"| norm["Norm\n(order = 2, dim = 1)"] -->|"0 → 0\n(B,H,W)"| unsqueeze[Unsqueeze] -->|"0 → 1\n(B,1,H,W)"| div[Div] -->|"0 →\n(B,C,H,W)"| output((output)):::tensor
input -->|"→ 0\n(B,C,H,W)"| div
classDef tensor fill:#ffbe98, color:#000
The Norm operation itself could be further down decomposed like so:
flowchart LR
input((input)):::tensor -->|"→ 0\n(B,C,H,W)"| pow["Pow\n(order = 2)"] -->|"→ 0\n(B,C,H,W)"| sum["Sum\n(dim = 1)"] -->|"0 → 0\n(B,H,W)"| pow2["Pow\n(order = 1/2)"] -->|"0 → 0\n(B,H,W)"| output((output)):::tensor
classDef tensor fill:#ffbe98, color:#000
Introduction of this feature brings the oportunity to talk about many core aidge choices that should be clarified now that the framework is more mature.
- Operators that are special case of more generic Operators (here,
Pow(order = 0.5)could be replaced withSqrt().*) - Operators that are a composition of other Operators (here Norm, but it works for FC too)
- Missing basic Operators (here:
Sum). May be it is time to have a list
Proposal
- Add
SumOperator - Add
NormalizeOperator (dicussion about "basic" Operators and MetaOperators in another issue)