Graph begining by a residual are hard to express with Aidge

I wanted to create a MetaOperator for the swish operator:

$swish(x)={{x}\over{1+e^{-\beta x}}}$

This will translate to the following graph:

graph TD
    Input --> Div
    Input --> Mul --> Pow --> Add --> Div
    Beta("-Beta") --> Mul
    e --> Pow
    one("1") --> Add

However, doing this is not trivial as the graph view has to know that it has one Input and not two (the Input node for Mul and Div are the same).

Maybe we will need an identity node to represent this ?

changed the description

auto g = Residual({Mul, Pow, Add, Div});?

auto g = Residual({Parallel({Identity, -Beta}), Parallel({Mul, e}), Parallel({Pow, 1}), Add, Div});?

I have tried the following:

import aidge_core
import numpy as np

from math import exp

e_prod = aidge_core.Producer(aidge_core.Tensor(np.array([exp(1)])), "exp")

one_prod = aidge_core.Producer(aidge_core.Tensor(np.array([1])), "one")

beta = 1
beta_prod = aidge_core.Producer(aidge_core.Tensor(np.array([-beta])), "beta")

mul_op = aidge_core.Mul()
pow_op = aidge_core.Pow()
add_op = aidge_core.Add()

print("mul")
beta_prod.add_child(mul_op, 0, 1)
print("pow")
e_prod.add_child(pow_op, 0, 1)
print("add")
one_prod.add_child(add_op, 0, 1)

swish = aidge_core.sequential([
    aidge_core.residual([
            mul_op,
            pow_op,
            add_op
        ]),
    aidge_core.Div()
])

swish.save("swish")

But the residual raises an error because mul has two input (it does not check for free inputs). The problem I see with this is that the graph wiew would be like:

graph TD
    FreeInput1 --> Div
    FreeInput2 --> Mul 
    Mul --> Pow --> Add --> Div
    Beta("-Beta") --> Mul
    e --> Pow
    one("1") --> Add

Furthermore the order of inputs is very important for the Div operator which make the graph harder to describe.

The main problem comes from Mul, Pow, Add, Div not having one input data and one attribute but 2 input data. Hence by not including manually beta_prod, e_prod, one_prod in the GraphView, each Node will be considered in inputNode. (if Attribute, they would be included automatically and Residual({Mul, Pow, Add, Div}) could work...)

The Residual function doesn't know which Node to choose and fails.

auto g = Residual({Parallel({Identity, -Beta}), Parallel({Mul, e}), Parallel({Pow, 1}), Add, Div});

Would not work as Residual works like the Sequential function. How does it handle the sub-graph Parallel({Mul, e}) with two inputs and two outputs in a generic maner? What are the connections? It is simply not possible to know.

changed the description

changed milestone to %v0.1.0

added 1 deleted label

assigned to @cmoineau

added StatusWork in Progress label and removed 1 deleted label

added Missing label

Implemented by commit: !50 (c994acd5) in the merge request: !50 (c994acd5)

This feature is used for aidge!8 (merged)

closed