How to handle fold of producer with scaling + casting

After PTQ the graph is as follow:

flowchart TD
	Input --> Conv
	Weights --> Scaling --> Conv

or

flowchart TD
	Input --> Conv
	Weights --> Scaling --> Cast --> Conv

If we are in true quantize.

For export we will want to fold the weights. To do so we need to set Producers to constant.

A current implementation has been proposed by @noamzerah that set every Producer to constant then apply constant folding: https://gitlab.eclipse.org/eclipse/aidge/aidge_quantization/-/blob/dev/src/PTQ/PTQ.cpp?ref_type=heads#L270-282

Another approache was proposed by @lucaslopez !41 (comment 3112148) In which he walk back the parents of a certain nodes ["FC","Conv1D", "Conv2D", "Conv3D","ConvDepthWise1D", "ConvDepthWise2D", "ConvDepthWise3D"] until he finds a producer.

After some reflexion, I wonder if we could not "just" use graph Matching to set to constant only the producer that are followed by a Scaling node.

What are you thougts @noamzerah @bhalimi @lucaslopez @olivierbichler @pineapple ?

added Discussion 💬 label

changed title from Freeze producers to fold producer and scaling + casting to How to handle fold of producer with scaling + casting

Hello @cmoineau !

In the current PTQ refactor I'm doing in !50 (merged), I fold only the parameters quantizers. Here is the code i use :

void foldProducerQuantizers(std::shared_ptr<GraphView> graphView)
{
    std::vector<std::shared_ptr<Node>> producerQuantizers;
    for (std::shared_ptr<Node> node : graphView->getNodes())
        if (hasAttr(node, "isProducerScaling"))
            producerQuantizers.push_back(node);

    for (std::shared_ptr<Node> quantizer : producerQuantizers)
    {
        // Set the param producer to be constant

        auto paramProducer = quantizer->getParent(0);
        auto paramProducerOp = std::static_pointer_cast<Producer_Op>(paramProducer->getOperator());
        paramProducerOp->constant() = true;

        // Set the internal producers of the quantizer to be constant

        auto quantizerOp = std::static_pointer_cast<MetaOperator_Op> (quantizer->getOperator());
    
        auto microGraph = quantizerOp->getMicroGraph();
        
        for (auto producer : microGraph->getNodes())
            if (producer->type() == "Producer")
            {
                auto producerOp = std::static_pointer_cast<Producer_Op>(producer->getOperator());
                producerOp->constant() = true;  
            }   

        expandMetaOp(quantizer); 
    }

    constantFolding(graphView);
}

Ah nice I didn't know you were working on this