Skip to content
Snippets Groups Projects
Commit 59b3d4e5 authored by Maxence Naud's avatar Maxence Naud
Browse files

Merge branch 'dev' into 'main' 

version 0.2.2

See merge request eclipse/aidge/aidge!60
parents e3655e61 cec0f7a9
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
Showing
with 69 additions and 51 deletions
aidge_backend_cpu @ d1114932
Compare a7666da2...d1114932
Subproject commit a7666da2e654db9e3b9be786c61f905069c7a325
Subproject commit d111493246ae4835681c47510cb08031950b3790
aidge_backend_cuda @ c18f99dd
Compare fa57d1d1...c18f99dd
Subproject commit fa57d1d18fb54fcfc5753ddf1b9bfbc862bf74f2
Subproject commit c18f99dd639635c4b2668664e087e50871bb17e3
aidge_backend_opencv @ ac47767e
Compare fd7466df...ac47767e
Subproject commit fd7466dfffdb85cd8f8f3c7143059e7895c4360c
Subproject commit ac47767e9016014e95387480a3ebe731b1f4b1e8
aidge_core @ 28fe9a43
Compare 928cb238...28fe9a43
Subproject commit 928cb2389b931be9c8d76daa3f3feef0b6537eb0
Subproject commit 28fe9a4344bf556949974ea2ad84248f30b9caf7
aidge_export_cpp @ c30ca760
Compare 64d65c9b...c30ca760
Subproject commit 64d65c9b68f86816b13b252b0d08c537776b0cf7
Subproject commit c30ca76080b0aeb43183525b08fe585c1f38bc40
aidge_learning @ 671b8440
Compare 74739318...671b8440
Subproject commit 7473931866b481bd95b3f6a5593ffd5f70fb89d6
Subproject commit 671b844081457d387ae17c437da76f1d54965b06
aidge_onnx @ f14ce434
Compare 2f103867...f14ce434
Subproject commit 2f1038678353f5e7744717374ffea0c0b642a21d
Subproject commit f14ce43460b0078379519f4b1a0f3c0f8c0da23f
aidge_quantization @ 3b7cc254
Compare 5c93e990...3b7cc254
Subproject commit 5c93e9908b8ef9b69d88b2d65eaaba9a870b50d6
Subproject commit 3b7cc2547c0a79f3ab71f48835fe28bec058e7c9
clean_all.sh 0 → 100755
+ 3
− 0
View file @ 59b3d4e5
#!/bin/bash
find . -maxdepth 1 -type f -name "clean_*" ! -name "clean_all.sh" -exec {} \;
clean_builds.sh
+ 1
− 1
View file @ 59b3d4e5
......@@ -13,7 +13,7 @@ BUILDPATHS=()
while IFS= read -r -d '' dir; do
# Add each directory to the BUILDPATHS list
BUILDPATHS+=("$dir")
done < <(find . -type d -name "build_bundle_cpp" -print0)
done < <(find . -type d -name "build*" -print0)
## Print and remove the list of build paths
for path in "${BUILDPATHS[@]}"; do
......
clean_graphs.sh 0 → 100755
+ 12
− 0
View file @ 59b3d4e5
#!/bin/bash
# Get the directory in which the script is located
script_directory="$(pwd)"
# Use find to locate files with the .mmd extension and print the list
files_to_remove=$(find "$script_directory" -type f -name "*.mmd")
echo "$files_to_remove"
find "$script_directory" -type f -name "*.mmd" -exec rm -f {} +
echo "Files with .mmd extension removed in $(pwd)"
clean_pip.sh 0 → 100755
+ 3
− 0
View file @ 59b3d4e5
#!/bin/bash
pip list | grep 'aidge' | cut -d ' ' -f 1 | xargs pip uninstall -y
examples/tutorials/101_first_step/load_and_run.ipynb
+ 11
− 11
View file @ 59b3d4e5
%% Cell type:markdown id: tags:
# Aidge demonstration
Aidge is a collaborative open source deep learning library optimized for export and processing on embedded devices. With Aidge, you can create or import a Computational Graph from common Frameworks, apply editing on its structure, train it and export its architecture on many embedded devices. Aidge provides optimized functions for inference as well as training and many custom functionalities for the target device.
This notebook put in perspective the tool chain to import a Deep Neural Network from ONNX model and support its Inference in Aidge. The tool chain demonstrated is :
![pipeline(1)](./static/pipeline_1.png)
In order to demonstrate this toolchain, the MNIST digit recognition task is used.
![MNIST](./static/MnistExamples.png)
%% Cell type:markdown id: tags:
## Setting up the notebook
%% Cell type:markdown id: tags:
### (if needed) Download the model
If you don't have git-lfs, you can download the model and data using this piece of code
%% Cell type:code id: tags:
``` python
import os
import requests
def download_material(path: str) -> None:
if not os.path.isfile(path):
response = requests.get("https://gitlab.eclipse.org/eclipse/aidge/aidge/-/raw/dev/examples/tutorials/101_first_step/"+path+"?ref_type=heads")
if response.status_code == 200:
with open(path, 'wb') as f:
f.write(response.content)
print("File downloaded successfully.")
else:
print("Failed to download file. Status code:", response.status_code)
# Download onnx model file
download_material("MLP_MNIST.onnx")
# Download input data
download_material("input_digit.npy")
# Download output data for later comparison
download_material("output_digit.npy")
```
%% Cell type:markdown id: tags:
### Define mermaid visualizer function
Aidge save graph using the mermaid format, in order to visualize the graph live in the notebook, we will setup the following function:
%% Cell type:code id: tags:
``` python
import base64
from IPython.display import Image, display
import matplotlib.pyplot as plt
def visualize_mmd(path_to_mmd):
with open(path_to_mmd, "r") as file_mmd:
graph_mmd = file_mmd.read()
graphbytes = graph_mmd.encode("utf-8")
base64_bytes = base64.b64encode(graphbytes)
base64_string = base64_bytes.decode("utf-8")
display(Image(url=f"https://mermaid.ink/img/{base64_string}"))
```
%% Cell type:markdown id: tags:
## Import Aidge
In order to provide a colaborative environnement in the plateform, the structure of Aidge is built on a core library that interfaces with multiple modules binded to python libraries.
- ``aidge_core`` is the core library and offers all the basic functionnalities to create and manipulate the internal graph representation
- ``aidge_backend_cpu`` is a C++ module providing a generic C++ implementations for each component of the graph
- ``aidge_onnx`` is a module allowing to import ONNX to the Aidge framework
- ``aidge_export_cpp`` is a module dedicated to the generation of optimized C++ code
This way, ``aidge_core`` is free of any dependencies and the user can install what he wants depending on his use case.
%% Cell type:code id: tags:
``` python
import aidge_core
# Conv2D Operator is available but no implementation has been loaded
print(f"Available backends:\n{aidge_core.get_keys_ConvOp2D()}")
# note: Tensor is a special case as 'cpu' backend is provided in the core
# module to guarantee basic functionalities such as data accesss
print(f"Available backends for Tensor:\n{aidge_core.Tensor.get_available_backends()}")
```
%% Cell type:markdown id: tags:
As you can see, no backends are availables for the class ``Conv2D``.
We need to import the ``aidge_backend_cpu`` module which will register itself automatically to ``aidge_core``.
%% Cell type:code id: tags:
``` python
import aidge_backend_cpu
print(f"Available backends:\n{aidge_core.get_keys_ConvOp2D()}")
```
%% Cell type:markdown id: tags:
For this tutorial, we will need to import ``aidge_onnx`` in order to load ONNX files, numpy in order to load data and matplotlib to display images.
%% Cell type:code id: tags:
``` python
import aidge_onnx
import numpy as np
import matplotlib.pyplot as plt
```
%% Cell type:markdown id: tags:
## ONNX Import
Import an ONNX model into Aidge internal graph representation.
![pipeline(2)](./static/pipeline_2.png)
%% Cell type:code id: tags:
``` python
model = aidge_onnx.load_onnx("MLP_MNIST.onnx")
```
%% Cell type:markdown id: tags:
As you can see in the logs, aidge imported a Node as a ``GenericOperator``:
```
- /Flatten_output_0 (Flatten | GenericOperator)
```
This is a fallback mechanism which allow aidge to load ONNX graph without failing even when encountering a node which is not available.
The ``GenericOperator`` act as a stub retrieving node type and attributes from ONNX. This allow to provide an implementation in a user script or as we will see to remove/replace them using aidge recipes.
You can visualize the graph using the ``save`` method and the mermaid visualizer we have setup.
%% Cell type:code id: tags:
``` python
model.save("myModel")
visualize_mmd("myModel.mmd")
```
%% Cell type:markdown id: tags:
## Graph transformation
![pipeline(3)](./static/pipeline_3.png)
In order to support the graph for inference we need to support all operators.
The imported model contains ```Flatten``` before the ```Gemm``` operator. The ```aidge.FC``` operator already supports the flatten operation.
Graph transformation is required to support the graph for inference, i.e. remove the ```Flatten``` operator.
Aidge graph transformation toolchain is the following process :
**1. Describe the graph pattern**
In order to find specific patterns inside a graph, there is first a need to describe those patterns. Aidge introduces an innovative way to describe graph patterns, **Graph Regular Expression**, inspired by regular expression from the formal language theory.
In this example the GraphRegEx used would be simple:
```
"Flatten->FC;"
```
%% Cell type:code id: tags:
``` python
graph_regex = aidge_core.GraphRegex()
graph_regex.set_node_key("Flatten", "getType($) =='Flatten'")
graph_regex.set_node_key("FC", "getType($) =='FC'")
graph_regex.add_query("Flatten -> FC")
```
%% Cell type:markdown id: tags:
**2. Match the described pattern**
Once the graph pattern is described with a graph regular expression, we apply an innovative graph matching algorithm to find patterns corresponding to the description.
This alogrithm will return all the matched patterns described with a graph regular expression in a [match](https://eclipse-aidge.readthedocs.io/en/latest/source/API/Core/graphMatching.html#match) class. One matched pattern is the combinaison of the graph pattern start nodes and all the nodes in the matched pattern (including the start nodes).
%% Cell type:code id: tags:
``` python
all_match = graph_regex.match(model)
print('Number of match : ', len(all_match))
```
%% Cell type:markdown id: tags:
In this case, we have one match :
- List of one list containing the start node : [[Flatten node]]
- List of one set containing all the matched nodes : [{Flatten node, FC node}]
Let's visualize the match :
%% Cell type:code id: tags:
``` python
print('The start node : ')
for match in all_match:
print('\t', match.get_start_node()[0].type())
print('All the matched nodes for', match.get_query() , ':')
for n in match.get_all():
print('\t', n.type())
```
%% Cell type:markdown id: tags:
**3. Apply graph transformations on the matched patterns**
Now that we have matched the desired patterns we can apply graph transformation on it. The main graph transformation functions (currently under dev) are :
- Replace the current GraphView with a set of given Nodes if possible : [replace](https://eclipse-aidge.readthedocs.io/en/latest/source/API/Core/graph.html#aidge_core.GraphView.replace)
- Insert a node (newParentNode) as a parent of the passed node (childNode) : [insert_parent](https://eclipse-aidge.readthedocs.io/en/latest/source/API/Core/graph.html#_CPPv4N5Aidge9GraphView12insertParentE7NodePtr7NodePtr9IOIndex_t9IOIndex_t9IOIndex_t)
- Remove a node : remove()
In this example we remove the ```Flatten``` operator from the graph using replace.
%% Cell type:code id: tags:
``` python
g = aidge_core.GraphView()
g.add(next(iter(all_match)).get_start_node()[0])
aidge_core.GraphView.replace(g.get_nodes(), set())
```
%% Cell type:markdown id: tags:
The flatten is removed, let's visualize the model :
%% Cell type:code id: tags:
``` python
model.save("mySupportedModel")
visualize_mmd("mySupportedModel.mmd")
```
%% Cell type:markdown id: tags:
All of these steps are embedded inside ``recipes`` functions. These recipes are available in ``aidge_core``, some recipes are:
- *fuse_batchnorm*: Fuse BatchNorm inside Conv or FC operator;
- *fuse_mul_add*: Fuse MatMul and Add operator into a FC operator;
- *remove_flatten*: Remove Flatten if it is before an FC operator.
Let's do it again with the *remove_flatten* recipie :
%% Cell type:code id: tags:
``` python
# Import model again
model = aidge_onnx.load_onnx("MLP_MNIST.onnx")
# Use remove_flatten recipie
aidge_core.remove_flatten(model)
```
%% Cell type:markdown id: tags:
This time the flatten is removed with the recipie, let's visualize the model :
%% Cell type:code id: tags:
``` python
model.save("mySupportedModel")
visualize_mmd("mySupportedModel.mmd")
```
%% Cell type:markdown id: tags:
## Inference
We now have a graph fully supported by aidge, we are ready to do some inference !
![pipeline(4)](./static/pipeline_4.png)
### Create an input tensor & its node in the graph
In order to perform an inferencewe will load an image from the MNIST dataset using Numpy.
%% Cell type:code id: tags:
``` python
## Load input data & its output from the MNIST_model
digit = np.load("input_digit.npy")
plt.imshow(digit[0][0], cmap='gray')
```
%% Cell type:markdown id: tags:
And in order to validate the result our model will provide, we will also load the output the PyTorch model povided for this image
%% Cell type:code id: tags:
``` python
output_model = np.load("output_digit.npy")
print(output_model)
```
%% Cell type:markdown id: tags:
Thanks to the Numpy interoperability we can create an Aidge ``Tensor`` using directly the numpy array storing the image.
%% Cell type:code id: tags:
``` python
input_tensor = aidge_core.Tensor(digit)
print(f"Aidge Input Tensor dimensions: \n{input_tensor.dims()}")
```
%% Cell type:markdown id: tags:
To add an input to the graph we can create a ``Producer`` node, insert it in the ``GraphView`` and set its output with the ``Tensor`` we have just created, or data can simply be fed to the ``GraphView`` via the ``scheduler`` ```forward()``` call.
%% Cell type:markdown id: tags:
### Configure the model for inference
%% Cell type:markdown id: tags:
At the moment the model has no implementation, it is only a datastructure. To set an implementation we will set a dataype and a backend.
%% Cell type:code id: tags:
``` python
# Configure the model
model.compile("cpu", aidge_core.DataType.Float32, dims=[[1,1,28,28]])
model.compile("cpu", aidge_core.dtype.float32, dims=[[1,1,28,28]])
# equivalent to set_datatype(), set_backend() and forward_dims()
```
%% Cell type:markdown id: tags:
### Create a scheduler and run inference
The graph is ready to run ! We just need to schedule the execution, to do this we will create a ``Scheduler`` object, which will take the graph and generate an optimized scheduling using a consummer producer heuristic.
%% Cell type:code id: tags:
``` python
# Create SCHEDULER
scheduler = aidge_core.SequentialScheduler(model)
# Run inference !
scheduler.forward(data=[input_tensor])
```
%% Cell type:code id: tags:
``` python
# Assert results
for outNode in model.get_output_nodes():
output_aidge = np.array(outNode.get_operator().get_output(0))
print(output_aidge)
print('Aidge prediction = ', np.argmax(output_aidge[0]))
assert(np.allclose(output_aidge, output_model,rtol=1e-04))
```
%% Cell type:markdown id: tags:
It is possible to save the scheduling in a mermaid format using:
%% Cell type:code id: tags:
``` python
scheduler.save_scheduling_diagram("schedulingSequential")
visualize_mmd("schedulingSequential.mmd")
```
%% Cell type:markdown id: tags:
## Export
Now that we have tested the imported graph we can look at one of the main feature of Aidge, the export of computationnal graph to an hardware target using code generation.
![pipeline(5)](./static/pipeline_5.png)
### Generate an export in C++
In this example we will generate a generic C++ export.
This export is not based on the `cpu` backend we have set before.
In this example we will create a standalone export which is abstracted from the Aidge platform.
%% Cell type:code id: tags:
``` python
! rm -r myexport
```
%% Cell type:code id: tags:
``` python
!ls myexport
```
%% Cell type:markdown id: tags:
Generating a ``cpu`` export recquires the ``aidge_export_cpp`` module.
Once the module is imported you just need one line to generate an export of the graph.
%% Cell type:code id: tags:
``` python
import aidge_export_cpp
# Freeze the model by setting constant to parameters producers
for node in model.get_nodes():
if node.type() == "Producer":
node.get_operator().set_attr("Constant", True)
node.get_operator().attr.constant = True
# Create Producer Node for the Graph
input_node = aidge_core.Producer([1, 1, 28, 28], "input")
input_node.add_child(model)
model.add(input_node)
# Configuration for the model + forward dimensions
model.compile("cpu", aidge_core.DataType.Float32)
model.compile("cpu", aidge_core.dtype.float32)
# Export the model in C++ standalone
aidge_export_cpp.export("myexport", model, scheduler)
```
%% Cell type:markdown id: tags:
The export function will generate :
- **dnn/layers** layers configuration;
- **dnn/parameters** folder with parameters;
- **dnn/include/dnn.h** API to use the export;
- **dnn/include/network_functions.h** header file for kernels;
- **dnn/memory** memory management information;
- **dnn/src** kernel source code + forward function;
- **main.cpp** This file is an export of the scheduler, it allows
- **Makefile** To compile the main.cpp
%% Cell type:code id: tags:
``` python
!tree myexport
```
%% Cell type:markdown id: tags:
### Generate an input file for tests
To test the export we need to provide data, to do so we will export the numpy array using:
%% Cell type:code id: tags:
``` python
aidge_export_cpp.generate_input_file(array_name="inputs", array=digit.reshape(-1), export_folder="myexport")
```
%% Cell type:markdown id: tags:
### Compile the export
%% Cell type:code id: tags:
``` python
!cd myexport && make
```
%% Cell type:markdown id: tags:
### Run the export
%% Cell type:code id: tags:
``` python
!./myexport/bin/run_export
```
......
examples/tutorials/Backend_cuda_tutorial/Backend_cuda.ipynb
+ 4
− 4
View file @ 59b3d4e5
%% Cell type:markdown id: tags:
# Backend CUDA Inference
This tutorial demonstrate the inference of LeNet model using Aidge Backend CUDA. <br>
For the sake of simplicity, we will not train the model. Feel free to replace the onnx model with one already trained. <br>
- we sart by creating a LeNet model:
%% Cell type:code id: tags:
``` python
import torch
import torch.nn as nn
import torch.onnx
class LeNet(nn.Module):
def __init__(self, num_classes):
super(LeNet, self).__init__()
self.conv1 = nn.Conv2d(1, 6, kernel_size=5)
self.conv2 = nn.Conv2d(6, 16, kernel_size=5)
self.fc1 = nn.Linear(16 * 4 * 4, 120)
self.fc2 = nn.Linear(120, 84)
self.fc3 = nn.Linear(84, num_classes)
def forward(self, x):
x = torch.relu(self.conv1(x))
x = torch.max_pool2d(x, kernel_size=2, stride=2)
x = torch.relu(self.conv2(x))
x = torch.max_pool2d(x, kernel_size=2, stride=2)
x = x.view(-1, 16 * 4 * 4)
x = torch.relu(self.fc1(x))
x = torch.relu(self.fc2(x))
x = self.fc3(x)
return x
# Instantiate the model
num_classes = 10 # Assuming you're working with MNIST dataset
model = LeNet(num_classes)
# Set the model to evaluation mode
model.eval()
# Example input shape (batch_size, channels, height, width)
dummy_input = torch.randn(1, 1, 28, 28)
# Export the model to ONNX
torch.onnx.export(model, dummy_input, "lenet.onnx", verbose=True)
```
%% Cell type:markdown id: tags:
- import the needed libraries
%% Cell type:code id: tags:
``` python
import aidge_core
import aidge_backend_cuda
import aidge_onnx
import numpy as np
```
%% Cell type:markdown id: tags:
- load onnx model on Aidge
%% Cell type:code id: tags:
``` python
model = aidge_onnx.load_onnx("lenet.onnx")
aidge_core.remove_flatten(model)
# Configure the model
model.set_datatype(aidge_core.DataType.Float32)
model.set_datatype(aidge_core.dtype.float32)
model.set_backend("cuda")
```
%% Cell type:markdown id: tags:
- add input
%% Cell type:code id: tags:
``` python
# Create an input node
input = np.random.randn(1, 1, 28, 28).astype(np.float32)
input_tensor = aidge_core.Tensor(input)
input_node = aidge_core.Producer(input_tensor, "input")
input_node.get_operator().set_datatype(aidge_core.DataType.Float32)
input_node.get_operator().set_datatype(aidge_core.dtype.float32)
input_node.get_operator().set_backend("cuda")
# Link node to model
input_node.add_child(model)
model.add(input_node)
```
%% Cell type:markdown id: tags:
- create a scheduler and run inference
%% Cell type:code id: tags:
``` python
# Define the scheduler
scheduler = aidge_core.SequentialScheduler(model)
# Run inference !
scheduler.forward()
```
%% Cell type:markdown id: tags:
- get the ouput: <br>
Before getting the output we need to set it to backend cpu
%% Cell type:code id: tags:
``` python
for outNode in model.get_output_nodes():
outNode.get_operator().get_output(0).set_backend('cpu')
output_aidge = np.array(outNode.get_operator().get_output(0))
print("Aidge output: {}".format(output_aidge))
# Make sure to set the output back to "cuda" otherwise the model will not be usable
outNode.get_operator().get_output(0).set_backend('cuda')
```
......
examples/tutorials/Database_DataProvider_tutorial/Database.ipynb
+ 10
− 10
View file @ 59b3d4e5
%% Cell type:markdown id: tags:
# Database MNIST
This tutorial demonstrate the usage of databases and data providers to perform an evaluation on a model using Aidge.
%% Cell type:markdown id: tags:
## Installation and Requirements
- Python packages : aidge_core, aidge_backend_cpu, aidge_backend_opencv
- Download MNIST database
- Download MLP onnx model from git-lfs
- Define model visualization function and top-1 accuracy metric
%% Cell type:code id: tags:
``` python
!pip install git-lfs
!git lfs pull
```
%% Cell type:code id: tags:
``` python
import os
import urllib.request
import gzip
import shutil
mnist_dir = 'MNIST_test'
os.makedirs(mnist_dir, exist_ok=True)
base_url = 'https://ossci-datasets.s3.amazonaws.com/mnist/'
files = ['t10k-images-idx3-ubyte.gz', 't10k-labels-idx1-ubyte.gz']
for file in files:
url = base_url + file
file_path = os.path.join(mnist_dir, file)
decompressed_file_path = os.path.splitext(file_path)[0]
if not os.path.exists(decompressed_file_path):
print("Downloading", file)
urllib.request.urlretrieve(url, file_path)
print("Download complete")
decompressed_file_path = os.path.splitext(file_path)[0]
print("Decompressing", file)
raw = gzip.open(file_path, 'rb').read()
open(decompressed_file_path, 'wb').write(raw)
print("Decompression complete")
else:
print(f"{file} already exists. Skipping download and decompression.")
```
%% Cell type:code id: tags:
``` python
import base64
from IPython.display import Image, display
import matplotlib.pyplot as plt
def visualize_mmd(path_to_mmd):
with open(path_to_mmd, "r") as file_mmd:
graph_mmd = file_mmd.read()
graphbytes = graph_mmd.encode("ascii")
base64_bytes = base64.b64encode(graphbytes)
base64_string = base64_bytes.decode("ascii")
display(Image(url=f"https://mermaid.ink/img/{base64_string}"))
```
%% Cell type:code id: tags:
``` python
def top1_accuracy(predictions, labels):
total = len(predictions)
predicted_class = predictions.argmax(axis=1)
correct_pred = (predicted_class == labels).sum()
accuracy = correct_pred / total
return accuracy
```
%% Cell type:markdown id: tags:
## Perform an evaluation of the LeNet-like on Aidge
- Import Aidge libraries
- Import ONNX model
- Configure the model for inference
- Create the Database and DataProvider
- Perform the evaluation
%% Cell type:code id: tags:
``` python
import aidge_core
import aidge_backend_opencv
import aidge_backend_cpu
import aidge_onnx
import numpy as np
```
%% Cell type:code id: tags:
``` python
model = aidge_onnx.load_onnx("../101_first_step/MLP_MNIST.onnx")
aidge_core.remove_flatten(model)
model.save("mySupportedModel")
visualize_mmd("mySupportedModel.mmd")
```
%% Cell type:code id: tags:
``` python
# Configure the model
model.set_datatype(aidge_core.DataType.Float32)
model.set_datatype(aidge_core.dtype.float32)
model.set_backend("cpu")
# Define the scheduler
scheduler = aidge_core.SequentialScheduler(model)
```
%% Cell type:code id: tags:
``` python
val_mnist = aidge_backend_opencv.MNIST(dataPath="./MNIST_test",
train=False,
load_data_in_memory=False)
val_dataprovider = aidge_core.DataProvider(val_mnist,
batch_size=200,
shuffle=True,
drop_last=False)
```
%% Cell type:code id: tags:
``` python
val_acc = 0
for i, (data_batch, lbl_batch) in enumerate(val_dataprovider):
data_batch.set_datatype(aidge_core.DataType.Float32)
lbl_batch.set_datatype(aidge_core.DataType.Float32)
data_batch.set_datatype(aidge_core.dtype.float32)
lbl_batch.set_datatype(aidge_core.dtype.float32)
# Run inference !
scheduler.forward(data=[data_batch])
# Get output and label in a numpy array
output_aidge = np.array(list(model.get_output_nodes())[0].get_operator().get_output(0))
lbl = np.array(lbl_batch)
# Compute the top-1 accuracy
val_acc += top1_accuracy(output_aidge, lbl.flatten())
val_acc = val_acc / len(val_dataprovider)
print(val_acc)
```
......
examples/tutorials/GraphRegex_tutorial/GraphRegex.ipynb
+ 5
− 5
View file @ 59b3d4e5
%% Cell type:markdown id: tags:
# Graph Regular Expression
Graph Regular expression is a powerful tool inspired by regular expressions. It is designed to express complex graph structures, much like regular expressions do for character strings.
This tutorial demonstrate the usage of the Graph Regular Expression to perform graph matching in Aidge.
It is organized as follows :
- Section Requirements covers the requirements to run this tutorial
- Section Graph Regex Flow explains the user pipeline to use Graph Regex
- Section Query presents how to describe different graph patterns
%% Cell type:markdown id: tags:
## Requirements
- Python packages : aidge_core, matplotlib, numpy
- Define model visualization function
%% Cell type:code id: tags:
``` python
!pip install --quiet matplotlib
!pip install --quiet numpy
```
%% Cell type:code id: tags:
``` python
import base64
from IPython.display import Image, display
import matplotlib.pyplot as plt
def visualize_mmd(path_to_mmd):
with open(path_to_mmd, "r") as file_mmd:
graph_mmd = file_mmd.read()
graphbytes = graph_mmd.encode("ascii")
base64_bytes = base64.b64encode(graphbytes)
base64_string = base64_bytes.decode("ascii")
display(Image(url=f"https://mermaid.ink/img/{base64_string}"))
```
%% Cell type:markdown id: tags:
## Graph Regex Flow
![pipeline_graph_matching](./static/pipeline_gm.PNG)
The GraphRegex class enable to perform graph transformations. It takes as inputs :
- A GraphView
- Queries : express one or more graph pattern descriptions to find in the GraphView
To have more information on the query, i.e. graph descriptions, please refer to the following slides to open in your browser :
- Basic concepts of [Graph Regular Expression](./static/GraphRegularExpression.html)
- [Quantifier](./static/Quantifier.html)
For more information, please refer to the User Guide on [graph transformation](https://eclipse-aidge.readthedocs.io/en/latest/source/UserGuide/transformGraph.html).
%% Cell type:markdown id: tags:
## Query
This section presents several examples to express different graph patterns : sequential, parallel, and advanced node testing.
### Sequential graph
- Create a sequential graph
- Describe a pattern with graph Regular Expression
- Match the describe pattern
%% Cell type:code id: tags:
``` python
import aidge_core
model = aidge_core.sequential([aidge_core.MatMul(name="MatMul0"),
aidge_core.Add(2, name="Add0"),
aidge_core.ReLU(name="ReLU0"),
aidge_core.MatMul(name="MatMul1"),
aidge_core.Add(2, name="Add1"),
aidge_core.ReLU(name="ReLU1")])
model.save("mySequentialModel")
visualize_mmd("mySequentialModel.mmd")
```
%% Cell type:code id: tags:
``` python
graph_regex = aidge_core.GraphRegex()
graph_regex.set_node_key("A", "getType($) =='Add'")
graph_regex.set_node_key("B", "getType($) =='ReLU'")
graph_regex.add_query("A -> B")
all_match = graph_regex.match(model)
print('Number of match : ', len(all_match))
```
%% Cell type:markdown id: tags:
In this case, we have two matches. Each match contains :
- The associated query. In this example : "A->B"
- The list containing the start nodes. In this example : [Add0]
- The set containing all the matched nodes. In this example : {Add0, ReLU0}
Let's visualize the matches :
%% Cell type:code id: tags:
``` python
for i, match in enumerate(all_match):
print('Match ', i ,' associated to query : ', match.get_query())
print('The start node :', match.get_start_node()[0].name())
print('All the matched nodes :')
for n in match.get_all():
print('\t', n.name())
```
%% Cell type:markdown id: tags:
### Parallel graph
- Create a graph with parallel branches
- Describe a pattern with graph Regular Expression
- Match the describe pattern
%% Cell type:code id: tags:
``` python
import aidge_core
model = aidge_core.sequential([aidge_core.MatMul(name="MatMul0"),
aidge_core.Add(2, name="Add0"),
aidge_core.ReLU(name="ReLU0"),
aidge_core.parallel([aidge_core.Conv2D(1, 1, [3, 3], name="Conv0"), aidge_core.Conv2D(1, 1, [3, 3], name="Conv1")]),
aidge_core.Add(2, name="Add1"),
aidge_core.ReLU(name="ReLU1")])
model.save("mySequentialModel")
visualize_mmd("mySequentialModel.mmd")
```
%% Cell type:markdown id: tags:
When applying regular expression concepts to graphs, we find ourselves dealing with parallel branches.
To address this, we express the graph by decomposing it into sequences.
This decomposition involves defining two types of nodes:
- common nodes, which reference the same node across multiple sequences : token "#"
- unique nodes, as the name suggests, that are exclusive to a particular sequence.
Let's try to match the ReLU followed by the two parallel convolutions :
%% Cell type:code id: tags:
``` python
graph_regex = aidge_core.GraphRegex()
graph_regex.set_node_key("A", "getType($) =='ReLU'")
graph_regex.set_node_key("B", "getType($) =='Conv'")
graph_regex.add_query("A# -> B; A# -> B")
all_match = graph_regex.match(model)
print('Number of match : ', len(all_match))
for i, match in enumerate(all_match):
print('Match ', i ,' associated to query : ', match.get_query())
print('The start node :', match.get_start_node()[0].name())
print('All the matched nodes :')
for n in match.get_all():
print('\t', n.name())
```
%% Cell type:markdown id: tags:
### Quantifiers
Quantifiers are tokens that specify the number of occurrences of a preceding element in the pattern. The supported quantifiers in graph regular expressions are :
- "+" : one or more
- "*" : zero or more
Using the quantifiers, the graph regular expression can represent an ensemble of patterns.
For example "Conv+" represents sequences of at least one convolution.
In this subsection :
- Create a sequential graph
- Describe a pattern with graph Regular Expression using quantifier 'one or more' : '+'
- Match the describe pattern
%% Cell type:code id: tags:
``` python
import aidge_core
model = aidge_core.sequential([aidge_core.MatMul(name="MatMul0"),
aidge_core.Add(2, name="Add0"),
aidge_core.ReLU(name="ReLU0"),
aidge_core.Conv2D(1, 1, [3, 3], name="Conv0"),
aidge_core.Conv2D(1, 1, [3, 3], name="Conv1"),
aidge_core.Conv2D(1, 1, [3, 3], name="Conv2")])
model.save("mySequentialModel")
visualize_mmd("mySequentialModel.mmd")
```
%% Cell type:markdown id: tags:
Let's try to match the ReLU0 followed by at least one convolution :
%% Cell type:code id: tags:
``` python
graph_regex = aidge_core.GraphRegex()
graph_regex.set_node_key("A", "getType($) =='ReLU'")
graph_regex.set_node_key("B", "getType($) =='Conv'")
graph_regex.add_query("A -> B+")
all_match = graph_regex.match(model)
print('Number of match : ', len(all_match))
for i, match in enumerate(all_match):
print('Match ', i ,' associated to query : ', match.get_query())
print('The start node :', match.get_start_node()[0].name())
print('All the matched nodes :')
for n in match.get_all():
print('\t', n.name())
```
%% Cell type:markdown id: tags:
### Advanced node testing
So far, the described graph patterns were based on the topology of the graph and the type of nodes.
Graph Regular expression allows to perform any test on the node by supporting the function calls of the given node.
- Create a sequential graph
- Describe a pattern with graph Regular Expression using advanced testing on the dimension of the convolution kernels.
- Match the describe pattern
%% Cell type:code id: tags:
``` python
import aidge_core
conv = aidge_core.Conv2D(1, 1, [7, 7], name="C")
print(conv.get_operator().get_attr("KernelDims") == [7,7])
print(conv.get_operator().attr.kernel_dims == [7,7])
print(conv.type())
model = aidge_core.sequential([aidge_core.Conv2D(1, 1, [7, 7], name="Conv0"),
aidge_core.Conv2D(1, 1, [5, 5], name="Conv1"),
aidge_core.Conv2D(1, 1, [3, 3], name="Conv2"),
aidge_core.Conv2D(1, 1, [5, 5], name="Conv3"),
aidge_core.Conv2D(1, 1, [7, 7], name="Conv4")])
model.save("mySequentialModel")
visualize_mmd("mySequentialModel.mmd")
```
%% Cell type:markdown id: tags:
Let's try to find the convolutions with kernel size 3 or 5.
First define your own custom testing fonctions :
%% Cell type:code id: tags:
``` python
def test_kernel_3(node):
b = False
if (node.type() == "Conv") :
b = node.get_operator().get_attr("KernelDims") == [3,3]
b = node.get_operator().attr.kernel_dims == [3,3]
return b
def test_kernel_5(node):
b = False
if (node.type() == "Conv") :
b = node.get_operator().get_attr("KernelDims") == [5,5]
b = node.get_operator().attr.kernel_dims == [5,5]
return b
```
%% Cell type:markdown id: tags:
Then register it in the graph regex and add the associated queries :
%% Cell type:code id: tags:
``` python
graph_regex = aidge_core.GraphRegex()
# Register the custom testing functions
graph_regex.set_node_key("testk3", test_kernel_3)
graph_regex.set_node_key("testk5", test_kernel_5)
graph_regex.set_node_key("Convk3", "testk3($)==true")
graph_regex.set_node_key("Convk5", "testk5($)==true")
#Add the associated queries
graph_regex.add_query("Convk3")
graph_regex.add_query("Convk5")
all_match = graph_regex.match(model)
print('Number of match : ', len(all_match))
for i, match in enumerate(all_match):
print('Match ', i ,' associated to query : ', match.get_query())
print('The start node :', match.get_start_node()[0].name())
print('All the matched nodes :')
for n in match.get_all():
print('\t', n.name())
```
......
examples/tutorials/Hardmax_operator/operator_adding.ipynb
+ 1
− 1
View file @ 59b3d4e5
%% Cell type:markdown id: tags:
# <ins>Adding an operator on Aidge</ins>
In this tutorial, we'll be adding the operator 'HardMax' to the Aidge plateform.<br>
The HardMax operator computes hardmax values for the given input along a given axis: <br>
Hardmax(element in input, axis) = 1 if the element is the first maximum value along the specified axis, 0 otherwise. <br>
Let's start by creating an onnx model with just the hardmax operator:
%% Cell type:code id: tags:
``` python
import onnx
from onnx import helper
# Create an ONNX graph
input_tensor_onnx = helper.make_tensor_value_info('input', onnx.TensorProto.FLOAT, [1, 10])
# Create Hardmax node
hardmax_node = onnx.helper.make_node(
'Hardmax',
inputs=['input'],
outputs=['output'],
axis=1 # Constant value for axis
)
# Define output tensor
output_tensor_onnx = helper.make_tensor_value_info('output', onnx.TensorProto.FLOAT, [1, 10])
# Create the graph
graph = helper.make_graph(
[hardmax_node],
'hardmax_model',
[input_tensor_onnx],
[output_tensor_onnx],
)
# Create the model with opset version 10
model = helper.make_model(graph, producer_name='onnx-hardmax-generator', opset_imports=[helper.make_opsetid("", 10)])
# Save the ONNX model to a file
onnx.save(model, 'hardmax_model.onnx')
```
%% Cell type:markdown id: tags:
Now let's try to load the saved model with aidge:
%% Cell type:code id: tags:
``` python
import aidge_core
import aidge_backend_cpu
import aidge_onnx
aidge_model = aidge_onnx.load_onnx('hardmax_model.onnx')
```
%% Cell type:markdown id: tags:
Loading the model on Aidge plateform will create a Generic Opertor for all operators that are not supported or not yet implemented.
In order to support the HardMax Operator, we'll have to add it to the Aidge plateform. This requires going through three modules:
- aidge_core
- aidge_backend
- aidge_onnx
%% Cell type:markdown id: tags:
### <ins>Aidge_core</ins>
[Aidge Core](https://eclipse-aidge.readthedocs.io/en/latest/source/API/Core/index.html) is the main API for the aidge plateform, it is the carcase where we define the objects that will be handled in the platform like operators, recipes, nodes ...<br>
In this module we will need to add the operator class and its python binding:
- aidge_core/include/aidge/operator/Hardmax.hpp<br>
In this file, we need to define the class Hardmax_Op. What we need to keep in mind is that Hardmax_Op is operated on 1 input and it has 1 Int attribute (Axis) so the constructors of the class will be as follows:
%% Cell type:code id: tags:
``` python
!tail -n +11 Core_files/Hardmax.hpp
```
%% Cell type:markdown id: tags:
After creating the header, add it in aidge_core/include/aidge/aidge.hpp<br>
We also need to add the cpp file where we define the Type attribute and we override methods if needed like computeOutputDims() if the output doesn't have the same shape as the input. <br>
- aidge_core/src/operator/Hardmax.cpp<br>
%% Cell type:code id: tags:
``` python
!tail -n +11 Core_files/Hardmax.cpp
```
%% Cell type:markdown id: tags:
- aidge_core/python_binding/operator/pybind_Hardmax.cpp<br>
In this file, we need to define the init_Harmax function as follows:
%% Cell type:code id: tags:
``` python
!tail -n +11 Core_files/pybind_Hardmax.cpp
```
%% Cell type:markdown id: tags:
Make sure to add all attributes as arguments for m.def() for Hardmax we only have "axis".<br>
Finally update aidge_core/python_binding/pybind_core.cpp
```C++
// add the init of the operator
void init_Hardmax(py::module&);
// also inside the function init_Aidge(py::module& m)
init_Hardmax(m);
```
%% Cell type:markdown id: tags:
### <ins>Aidge_backend</ins>
[Aidge_backend](https://eclipse-aidge.readthedocs.io/en/latest/source/API/BackendCPU/index.html) is the module where we implement what the object created in Aidge_core actually do. In the case of Operators, the main thing we will add in this module is the kernel. There are currently two backends available: Aidge_backend_cpu and Aidge_backend_gpu. In this tutorial we will go through the cpu backend.<br>
The files we need to add in this module in order to implement Hardmax Operator are:<br>
- aidge_backend_cpu/include/aidge/backend/cpu/operator/HardmaxImpl.hpp<br>
In this file, we start by declaring the forward and backward kernel registry
%% Cell type:code id: tags:
``` python
!tail -n +11 Backend_cpu_files/HardmaxImpl.hpp
```
%% Cell type:markdown id: tags:
Make sure to include the header in aidge_backend_cpu/include/aidge/backend/cpu.hpp<br>
- aidge_backend_cpu/include/aidge/backend/cpu/operator/HardmaxImpl_forward_kernels.hpp<br>
This is the main file of the backend, in here we will code the main function of the operator
%% Cell type:code id: tags:
``` python
!tail -n +11 Backend_cpu_files/HardmaxImpl_forward_kernels.hpp
```
%% Cell type:markdown id: tags:
- aidge_backend_cpu/src/operator/HardmaxImpl.cpp<br>
The last file to add is the cpp for the implementation in which we call the kernal with the right arguments
%% Cell type:code id: tags:
``` python
!tail -n +11 Backend_cpu_files/HardmaxImpl.cpp
```
%% Cell type:markdown id: tags:
It is also a good practice to add unit tests for the operator under ``aidge_backend_cpu/unit_tests/operator``, you can find an example of operator unit test [here](https://gitlab.eclipse.org/eclipse/aidge/aidge_backend_cpu/-/blob/dev/unit_tests/operator/Test_PowImpl.cpp?ref_type=heads).
%% Cell type:markdown id: tags:
### <ins>Aidge_onnx</ins>
[Aidge Onnx](https://eclipse-aidge.readthedocs.io/en/latest/source/API/Onnx/index.html) is the API with the library ONNX. In this module we mainly handle imports and exports from and to onnx.<br>
What we need to do for adding the HardMax Operator is to add two files:
- aidge_onnx/aidge_onnx/node_import/onnx_converters/hardmax.py<br>
In this file we need to write the function that converts an onnx_node into an aidge_node.<br>
What differs in imorting from one operator to another are mainly the attributes as shown in the [onnx_doc](https://onnx.ai/onnx/operators/onnx__Hardmax.html#l-onnx-doc-hardmax), the hardmax operator has one attribute which is the axis so we need to get this attributes in order to create the aidge_core.Hardmax node with the correct value of Axis.<br>
%% Cell type:code id: tags:
``` python
!tail -n +11 Onnx_files/import_converter/hardmax.py
```
%% Cell type:markdown id: tags:
- aidge_onnx/aidge_onnx/node_export/aidge_converters/hardmax.py<br>
In this file we need to write the function that converts an aidge node into an onnx node.<br>
In the export we also need to handle the attrbutes when converting. Therefore we call helper.make_attribute() to create the axis attribute and append it to the node with onnx_node.attribute.append()
%% Cell type:code id: tags:
``` python
!tail -n +11 Onnx_files/export_converter/hardmax.py
```
%% Cell type:markdown id: tags:
### <ins>Building the project</ins>
Follow the [tutorial](https://eclipse-aidge.readthedocs.io/en/latest/source/GetStarted/install.html#build-on-linux) to build the three modified modules.<br>
```
$ pip install ./aidge_core
$ pip install ./aidge_backend_cpu
$ pip install ./aidge_onnx
```
%% Cell type:markdown id: tags:
### <ins>Test and Validation</ins>
Now we can test importing the operator on aidge and run an inference using the scheduler.
%% Cell type:code id: tags:
``` python
import aidge_core
import aidge_backend_cpu
import aidge_onnx
aidge_model = aidge_onnx.load_onnx('hardmax_model.onnx')
```
%% Cell type:markdown id: tags:
Now that we've added the HardMax Operator, loading the model no longer creates Generic Operator.<br>
We can even run inference to check if there are no problems in the implementation of the operator.
%% Cell type:code id: tags:
``` python
import numpy as np
# Setup input
input_tensor = aidge_core.Tensor(np.random.randn(1, 10).astype(np.float32))
# Setup model's features: data type, backend and input dimensions
aidge_model.compile("cpu", aidge_core.DataType.Float32, dims=[[1,10]])
aidge_model.compile("cpu", aidge_core.dtype.float32, dims=[[1,10]])
# Set up the scheduler
scheduler = aidge_core.SequentialScheduler(aidge_model)
# Run inference !
scheduler.forward(data = [input_tensor]) # provide an input
# Check results
print("Input: {}".format(input))
for outNode in aidge_model.get_output_nodes():
output_aidge = np.array(outNode.get_operator().get_output(0))
print("Aidge output: {}".format(output_aidge))
```
%% Cell type:markdown id: tags:
We can also compare the output of the inference of the model with onnx's runtime inference to make sure the implementation is correct.
%% Cell type:code id: tags:
``` python
import onnxruntime
import numpy as np
# Load the ONNX model
onnx_model_path = 'hardmax_model.onnx'
session = onnxruntime.InferenceSession(onnx_model_path)
# Run inference
output = session.run(['output'], {'input': input})
# Display the result
print("ONNX Output: {}".format(output[0]))
```
......
examples/tutorials/Learning/learn.ipynb
+ 4
− 4
View file @ 59b3d4e5
%% Cell type:markdown id: tags:
# Training a simple Neural Network
This tutorial introduces the basics of learning with the framework Aidge.
What you will learn:
1. creating a model using Aidge API
2. create and import a dataset
3. train a model
The folowinfg modules will be required:
%% Cell type:code id: tags:
``` python
import aidge_core
import aidge_backend_cpu
import aidge_learning
import numpy as np
# required to load CIFAR10 dataset
import torchvision
import torchvision.transforms as transforms
```
%% Cell type:markdown id: tags:
## Creating Aidge model
In this example, we will create a simple perceptron model. For this we will use the helper function ``sequential`` which will connect the nodes in a sequential manner and will return the reuslting ``GraphView``.
%% Cell type:code id: tags:
``` python
model = aidge_core.sequential([
aidge_core.FC(in_channels=32*32*3, out_channels=512),
aidge_core.ReLU(),
aidge_core.FC(in_channels=512, out_channels=256),
aidge_core.ReLU(),
aidge_core.FC(in_channels=256, out_channels=128),
aidge_core.ReLU(),
aidge_core.FC(in_channels=128, out_channels=10),
])
```
%% Cell type:markdown id: tags:
Once the model is created, we can set its backend, datatype and initialize the values of the parameters.
We will initialize ``FC`` weights with the He filler and set all biases to ``0.01``.
%% Cell type:code id: tags:
``` python
# Set backend and datatype
model.set_backend("cpu")
model.set_datatype(aidge_core.DataType.Float32)
model.set_datatype(aidge_core.dtype.float32)
# Initialize parameters (weights and biases)
for node in model.get_nodes():
if node.type() == "Producer":
prod_op = node.get_operator()
value = prod_op.get_output(0)
tuple_out = node.output(0)[0]
# No conv in current network
if tuple_out[0].type() == "Conv" and tuple_out[1]==1:
# Conv weight
aidge_core.xavier_uniform_filler(value)
elif tuple_out[0].type() == "Conv" and tuple_out[1]==2:
# Conv bias
aidge_core.constant_filler(value, 0.01)
elif tuple_out[0].type() == "FC" and tuple_out[1]==1:
# FC weight
aidge_core.he_filler(value)
elif tuple_out[0].type() == "FC" and tuple_out[1]==2:
# FC bias
aidge_core.constant_filler(value, 0.01)
else:
pass
```
%% Cell type:markdown id: tags:
**Note:** We could have initialized producers using graph matching.
## Aidge database
Now that the model is ready we need to prepare a database. For this we will use the possiiblity to create a custom database using ``aidge_core.Database``.
We will use the framework PyTorch to load CIFAR10 and then write a custom database to transform the tensors to one hot encoded Aidge tensor.
%% Cell type:code id: tags:
``` python
def one_hot_encoding(cls, nb_cls):
values = [float(0.0)] * nb_cls
values[cls] = float(1.0)
t = aidge_core.Tensor(values)
t.set_datatype(aidge_core.DataType.Float32)
t.set_datatype(aidge_core.dtype.float32)
return t
class aidge_cifar10(aidge_core.Database):
def __init__(self):
aidge_core.Database.__init__(self)
transform = transforms.Compose(
[transforms.ToTensor(), transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])
self.trainset = torchvision.datasets.CIFAR10(root='./data', train=True,
download=True, transform=transform)
def get_item(self, idx):
data, label = self.trainset.__getitem__(idx)
return [aidge_core.Tensor(data.numpy()),
one_hot_encoding(label, 10)]
def len(self):
return len(self.trainset)
def get_nb_modalities(self):
return 2
```
%% Cell type:markdown id: tags:
Using this, we can now create a dataprovider that we will use to send data to the network.
%% Cell type:code id: tags:
``` python
aidge_database = aidge_cifar10()
BATCH_SIZE = 64
aidge_dataprovider = aidge_core.DataProvider(aidge_database,
batch_size=BATCH_SIZE,
shuffle=True,
drop_last=True)
```
%% Cell type:markdown id: tags:
## Set up learning objects
We now have all the basic elements required to run the leanring. We just need to setup the object specific to the learning and we will be able to write our first training loop !
For propagation and backpropagation, Aidge use scheduler obejct, we will use the ``SequentialScheduler``.
%% Cell type:code id: tags:
``` python
# Set object for learning
scheduler = aidge_core.SequentialScheduler(model)
```
%% Cell type:markdown id: tags:
To update weights, we will use an optimizer, in this case SGD.
%% Cell type:code id: tags:
``` python
# setup optimizer
opt = aidge_learning.SGD()
learning_rates = aidge_learning.constant_lr(0.01)
opt.set_learning_rate_scheduler(learning_rates)
opt.set_parameters(list(aidge_core.producers(model)))
```
%% Cell type:markdown id: tags:
## Training loop
%% Cell type:code id: tags:
``` python
tot_acc = 0
for i, (input, label) in enumerate(aidge_dataprovider):
input.init_grad()
# input.init_grad()
scheduler.forward(data=[input])
# Really long line should be a faster way ...
pred = list(model.get_output_nodes())[0].get_operator().get_output(0)
opt.reset_grad()
loss = aidge_learning.loss.MSE(pred, label)
acc = np.sum(np.argmax(pred, axis=1) == np.argmax(label, axis=1))
tot_acc += acc
scheduler.backward()
opt.update()
print(f"Nb samples {(i+1)*BATCH_SIZE}, loss: {loss[0]}, acc:{(acc/BATCH_SIZE)*100}%, tot_acc:{(tot_acc/((i+1)*BATCH_SIZE))*100}%")
```
......
examples/tutorials/Onnx_tutorial/onnx_tuto.ipynb
+ 4
− 4
View file @ 59b3d4e5
%% Cell type:markdown id: tags:
# Aidge ONNX tutorial
In this tutorial, we will see how to extensivly use the aidge ONNX module.
The following points will be covered:
- How to load an ONNX;
- How to add the support for an ONNX operator;
- How to support an unsupported operator for export purposes;
- How to add an implementation to an unknown operator.
For this tutorial, we will need the following libraries:
%% Cell type:code id: tags:
``` python
import aidge_core
import aidge_onnx
import aidge_backend_cpu # Required for Producer implementation
import numpy as np # Required to load data
```
%% Cell type:markdown id: tags:
## Setting up the notebook
### Retrieve the onnx model
In order to run this tutorial, we will use a simple ONNX composed of a single operator ``Swish``. This operator is not supported by ONNX and is often decomposed in multiple operators.
Pull this onnx model using git-lfs :
%% Cell type:code id: tags:
``` python
!pip install git-lfs
!git lfs pull
```
%% Cell type:markdown id: tags:
## Importing an ONNX
Importing an ONNX using Aidge is done using the function: [aidge_onnx.load_onnx()](https://eclipse-aidge.readthedocs.io/en/latest/source/API/Onnx/index.html#aidge_onnx.load_onnx).
%% Cell type:code id: tags:
``` python
graph = aidge_onnx.load_onnx("test_swish.onnx")
```
%% Cell type:markdown id: tags:
The swish operator is not supported and thus is loaded as a [GenericOperator](https://eclipse-aidge.readthedocs.io/en/latest/source/API/Core/operator.html#generic-operator). This mechanism allow to load unsupported graph into the framework.
The ``aidge_onnx`` library has a coverage report tools in order to check how well the graph loaded is supported:
%% Cell type:code id: tags:
``` python
if not aidge_onnx.has_native_coverage(graph):
print("The graph is not fully supported by aidge !\n")
aidge_onnx.native_coverage_report(graph)
```
%% Cell type:markdown id: tags:
However, this does not mean we cannot work with this graph !
## Working with generic operator
Indeed, using the python library, we can work with GenericOperator !
For this we will begin with retrieving the operator:
%% Cell type:code id: tags:
``` python
swish_node = graph.get_nodes().pop() # get_nodes return a set
swish_op = swish_node.get_operator() # Retrieving operator from node
```
%% Cell type:markdown id: tags:
### Computing output dimensions
In order to generate a scheduling, we need to specify how the operator will modify the data. This is required so that Aidge can propagate in/out dimensions.
Generating a scheduling is necessary to generate an export or make inference in the framework.
We can set a function to compute the dims using the ``set_forward_dims()`` method.
In our case, the swish function does not modify the dimensions so we can just send the same dimensiosn as the input. For this we will create an identity lambda function.
%% Cell type:code id: tags:
``` python
swish_op.set_forward_dims(lambda x: x)
```
%% Cell type:markdown id: tags:
### Providing an implementation
If we want to run an inference, we need to provide an implementation. Which means define the forward function.
The swish function is defined as: $swish(x)={{x}\over{1+e^{-\beta x}}}$.
So we can create a simple implementation using the numpy library:
%% Cell type:code id: tags:
``` python
from functools import reduce
class SwishImpl(aidge_core.OperatorImpl): # Inherit OperatorImpl to interface with Aidge !
def __init__(self, op: aidge_core.Operator):
aidge_core.OperatorImpl.__init__(self, op, "swish") # Required to avoid type error with C++ binding !
self.op = op # Reference to the Aidge operator to retrieve attributes, inputs, outputs ..
def forward(self):
data_input = np.array(self.op.get_input(0))
beta = np.array(self.op.get_attr("beta")) # Attribute name is the same as the one in the ONNX
output = (data_input / (1 + np.exp(-data_input*beta)))
self.op.set_output(0, aidge_core.Tensor(output)) # setting operator output
```
%% Cell type:markdown id: tags:
This implementation can then be set using:
%% Cell type:code id: tags:
``` python
swish_op.set_impl(SwishImpl(swish_op)) # Setting implementation
```
%% Cell type:markdown id: tags:
Once this is done, we can run an inference.
Let's first create an input:
%% Cell type:code id: tags:
``` python
numpy_tensor = np.random.randn(1, 10).astype(np.float32)
in_tensor = aidge_core.Tensor(numpy_tensor)
print(f"Random input:\n{numpy_tensor}")
```
%% Cell type:markdown id: tags:
Then we can create a scheduler and run the inference:
%% Cell type:code id: tags:
``` python
graph.compile("cpu", aidge_core.DataType.Float32, dims=[[1,10]])
graph.compile("cpu", aidge_core.dtype.float32, dims=[[1,10]])
scheduler = aidge_core.SequentialScheduler(graph)
scheduler.forward(data=[in_tensor])
for outNode in graph.get_output_nodes():
output_aidge = np.array(outNode.get_operator().get_output(0))
print('Aidge prediction = ', output_aidge)
```
%% Cell type:markdown id: tags:
## Updating ONNX import
We have seen how to handle GenericOperator in order to generate an export or run inference. However, this is not the only approach we can have to support an unsupported operator.
As stated above, the Swish function is the composition of an ``Exp``, ``Add`` and ``Div``. In this section, we will see how we can interact with the ``aidge_onnx`` library in order to add the support for new operators. This section will also showcase the use of MetaNodes.
### Creating a MetaNode
The first step is to reproduce the swish operation using a MetaOperator.
For this we will need to create a Producer Node for each constant: ``exp``, ``1`` and ``beta``.
Then define each function ``Exp``, ``Add`` and ``Div``. And then create a GraphView that we will embedded in a MetaOperator.
> Note: The swish computation graph begin with a branch split, so to have one input I use the operator Identity.
%% Cell type:code id: tags:
``` python
from math import exp
def gen_swish_metaop(nb_chan, name):
# Declaring constant values
e_prod = aidge_core.Producer(aidge_core.Tensor(np.array([exp(1)]*nb_chan, dtype=np.float32)), "exp")
one_prod = aidge_core.Producer(aidge_core.Tensor(np.array([1]*nb_chan, dtype=np.float32)), "one")
beta = 0.1
beta_prod = aidge_core.Producer(aidge_core.Tensor(np.array([-beta]*nb_chan, dtype=np.float32)), "beta")
# Declaring operators
mul_op = aidge_core.Mul(name=f"{name}_MUL")
pow_op = aidge_core.Pow(name=f"{name}_POW")
add_op = aidge_core.Add(2, name=f"{name}_ADD")
div_op = aidge_core.Div(name=f"{name}_DIV")
input_op = aidge_core.Identity(f"{name}_Input")
# Declaring Connectors
x = aidge_core.Connector(input_op)
b = aidge_core.Connector(beta_prod)
e = aidge_core.Connector(e_prod)
o = aidge_core.Connector(one_prod)
# Graph creation using functionnal declaration
y = div_op(x, add_op(pow_op(e, mul_op(x, b)), o))
swish_micro_graph = aidge_core.generate_graph([y])
# Saving micrograph for visualisation
swish_micro_graph.save("swish_micro")
# Embedding GraphView in a MetaOperator
swish_op = aidge_core.meta_operator(
"Swish",
swish_micro_graph,
name
)
return swish_op
# Testing swich metaop
_ = gen_swish_metaop(10, "Test")
```
%% Cell type:markdown id: tags:
We can then visualize the micro graph of the macro operator swish using mermaid:
%% Cell type:code id: tags:
``` python
import base64
from IPython.display import Image, display
import matplotlib.pyplot as plt
def visualize_mmd(path_to_mmd):
with open(path_to_mmd, "r") as file_mmd:
graph_mmd = file_mmd.read()
graphbytes = graph_mmd.encode("ascii")
base64_bytes = base64.b64encode(graphbytes)
base64_string = base64_bytes.decode("ascii")
display(Image(url=f"https://mermaid.ink/img/{base64_string}"))
visualize_mmd("swish_micro.mmd")
```
%% Cell type:markdown id: tags:
We have successfully created a function which can create a MetaOperator for the Swish function !
%% Cell type:markdown id: tags:
We have successfully created a function which can create a MetaOperator for the Swish function !
The next step is to register this function so that it is called by the ONNX import library.
### Registering new node import
Registering a new node to the ONNX import library can be easily done using the decorator function ``@aidge_onnx.node_import.auto_register_import``.
This decorator will register the function to the dictionary of import function [aidge_onnx.node_converter.ONNX_NODE_CONVERTER_](https://eclipse-aidge.readthedocs.io/en/latest/source/API/Onnx/index.html#aidge_onnx.node_converter.ONNX_NODE_CONVERTER_). Note that the key you should use is the ONNX name of the operator in lowercase.
%% Cell type:code id: tags:
``` python
NB_CHAN = 10 # TODO: Find a way to infer nb channel later ...
@aidge_onnx.node_import.auto_register_import("swish")
def import_swish(onnx_node, input_nodes, opset=None):
node_name = onnx_node.output[0]
return gen_swish_metaop(NB_CHAN, node_name)
```
%% Cell type:markdown id: tags:
Once this is done, you can use ``aidge_onnx.node_import.supported_operators()`` and check that swish is part of the supported operators:
%% Cell type:code id: tags:
``` python
aidge_onnx.node_import.supported_operators()
```
%% Cell type:markdown id: tags:
Since swish is supported we can load again the onnx:
%% Cell type:code id: tags:
``` python
supported_graph = aidge_onnx.load_onnx("test_swish.onnx")
```
%% Cell type:markdown id: tags:
Since we have decomposed the Swish operation in atomic operator supported by Aidge, we don't need to provide an implementation and instead we can just use the ``aidge_backend_cpu`` implementation to run an inference:
%% Cell type:code id: tags:
``` python
data_input = aidge_core.Producer(aidge_core.Tensor(np.arange(NB_CHAN, dtype=np.float32)+1.0), "data")
data_input.add_child(supported_graph)
supported_graph.add(data_input)
data_input.get_operator().set_datatype(aidge_core.DataType.Float32)
data_input.get_operator().set_datatype(aidge_core.dtype.float32)
data_input.get_operator().set_backend("cpu")
supported_graph.set_datatype(aidge_core.DataType.Float32)
supported_graph.set_datatype(aidge_core.dtype.float32)
supported_graph.set_backend("cpu")
# Create SCHEDULER
scheduler = aidge_core.SequentialScheduler(supported_graph)
# Run inference !
scheduler.forward()
for outNode in supported_graph.get_output_nodes():
output_aidge = np.array(outNode.get_operator().get_output(0))
print("MetaOperator output:")
print(output_aidge)
x = np.arange(NB_CHAN, dtype=np.float32)+1.0
beta = 0.1
print("Reference output:")
print(x / (1. + np.exp(-beta*x)))
```
......
examples/tutorials/PTQ_tutorial/ptq_tuto.ipynb
+ 3
− 3
View file @ 59b3d4e5
%% Cell type:markdown id: tags:
## Post Training Quantization with AIDGE
#### What is Network Quantization ?
Deploying large Neural Network architectures on embedded targets can be a difficult task as they often require billions of floating operations per inference.
To address this problem, several techniques have been developed over the past decades in order to reduce the computational load and energy consumption of those inferences. Those techniques include Pruning, Compression, Quantization and Distillation.
In particular, Post Training Quantization (PTQ) consists in taking an already trained network, and replacing the costly floating-point MADD by their integer counterparts. The use of Bytes instead of Floats also leads to a smaller memory bandwidth.
While this process can seem trivial, the naive approach consisting only in rounding the parameters and activations doesn't work in practice. Instead, we want to normalize the network in order to optimize the ranges of parameters and values propagated inside the network, before applying quantization.
#### The Quantization Pipeline
The PTQ algorithm consists in a 3 steps pipeline:
- First we optimize the parameter ranges by propagating the scaling coefficients in the network.
- Secondly, we compute the activation values over an input dataset, and insert the scaling nodes.
- Finally, we quantize the network by reconfiguring the scaling nodes according to the desired precision.
![alt text](./static/ptq_diagram.png)
#### Doing the PTQ with AIDGE
This notebook shows how to perform PTQ of a Convolutional Network, trained on the MNIST dataset.
The tutorial is constructed as follows :
- Setup of the AIDGE environment
- Loading of the model and example inferences
- Evaluation of the trained model accuracy
- Post Training Quantization and test inferences
- Evaluation of the quantized model accuracy
As we will observe in this notebook, we get zero degradation of the accuracy for a 8-bits PTQ.
Let's begin !
%% Cell type:markdown id: tags:
### (if needed) Download the model
If you don't have git-lfs, you can download the model and data using this piece of code
%% Cell type:code id: tags:
``` python
import os
import requests
def download_material(path: str) -> None:
if not os.path.isfile(path):
response = requests.get("https://gitlab.eclipse.org/eclipse/aidge/aidge/-/raw/dev/examples/tutorials/PTQ_tutorial/"+path+"?ref_type=heads")
if response.status_code == 200:
with open(path, 'wb') as f:
f.write(response.content)
print("File downloaded successfully.")
else:
print("Failed to download file. Status code:", response.status_code)
# Download onnx model file
download_material("ConvNet.onnx")
# Download data sample
download_material("mnist_samples.npy.gz")
# Download data label
download_material("mnist_labels.npy.gz")
```
%% Cell type:markdown id: tags:
#### Environment setup ...
We need numpy for manipulating the inputs, matplotlib for visualization purposes, and gzip to uncompress the numpy dataset.
Then we want to import the aidge modules :
- the core module contains everything we need to manipulate the graph.
- the backend module allows us to perform inferences using the CPU.
- the onnx module allows us to load the pretrained model (stored in an onnx file).
- the quantization module encaplsulate the Post Training Quantization algorithm.
%% Cell type:code id: tags:
``` python
import gzip
import numpy as np
import matplotlib.pyplot as plt
import aidge_core
import aidge_onnx
import aidge_backend_cpu
import aidge_quantization
print(" Available backends : ", aidge_core.Tensor.get_available_backends())
```
%% Cell type:markdown id: tags:
Then, let's define the configurations of this script ...
%% Cell type:code id: tags:
``` python
NB_SAMPLES = 100
NB_BITS = 8
```
%% Cell type:markdown id: tags:
Now, let's load and visualize some samples ...
%% Cell type:code id: tags:
``` python
samples = np.load(gzip.GzipFile('mnist_samples.npy.gz', "r"))
labels = np.load(gzip.GzipFile('mnist_labels.npy.gz', "r"))
```
%% Cell type:code id: tags:
``` python
for i in range(10):
plt.subplot(1, 10, i + 1)
plt.axis('off')
plt.tight_layout()
plt.imshow(samples[i], cmap='gray')
```
%% Cell type:markdown id: tags:
#### Importing the model in AIDGE ...
%% Cell type:code id: tags:
``` python
aidge_model = aidge_onnx.load_onnx("ConvNet.onnx", verbose=False)
aidge_core.remove_flatten(aidge_model) # we want to get rid of the 'flatten' nodes ...
```
%% Cell type:markdown id: tags:
#### Setting up the AIDGE scheduler ...
In order to perform inferences with AIDGE we need to setup a scheduler. But before doing so, we need to create a data producer node and connect it to the network.
%% Cell type:code id: tags:
``` python
# Insert the input producer
input_node = aidge_core.Producer([1, 1, 28, 28], "XXX")
input_node.add_child(aidge_model)
aidge_model.add(input_node)
# Set up the backend
aidge_model.set_datatype(aidge_core.DataType.Float32)
aidge_model.set_datatype(aidge_core.dtype.float32)
aidge_model.set_backend("cpu")
# Create the Scheduler
scheduler = aidge_core.SequentialScheduler(aidge_model)
```
%% Cell type:markdown id: tags:
#### Running some example inferences ...
Now that the scheduler is ready, let's perform some inferences. To do so we first declare a utility function that will prepare and set our inputs, propagate them and retreive the outputs.
%% Cell type:code id: tags:
``` python
def propagate(model, scheduler, sample):
# Setup the input
sample = np.reshape(sample, (1, 1, 28, 28))
input_tensor = aidge_core.Tensor(sample)
input_node.get_operator().set_output(0, input_tensor)
# Run the inference
scheduler.forward()
# Gather the results
output_node = model.get_output_nodes().pop()
output_tensor = output_node.get_operator().get_output(0)
return np.array(output_tensor)
print('\n EXAMPLE INFERENCES :')
for i in range(10):
output_array = propagate(aidge_model, scheduler, samples[i])
print(labels[i] , ' -> ', np.round(output_array, 2))
```
%% Cell type:markdown id: tags:
#### Computing the model accuracy ...
%% Cell type:code id: tags:
``` python
def compute_accuracy(model, samples, labels):
acc = 0
for i, x in enumerate(samples):
y = propagate(model, scheduler, x)
if labels[i] == np.argmax(y):
acc += 1
return acc / len(samples)
accuracy = compute_accuracy(aidge_model, samples[0:NB_SAMPLES], labels)
print(f'\n MODEL ACCURACY : {accuracy * 100:.3f}%')
```
%% Cell type:markdown id: tags:
#### Quantization dataset creation ...
We need to convert a subset of our Numpy samples into AIDGE tensors, so that they can be used to compute the activation ranges.
%% Cell type:code id: tags:
``` python
tensors = []
for sample in samples[0:NB_SAMPLES]:
sample = np.reshape(sample, (1, 1, 28, 28))
tensor = aidge_core.Tensor(sample)
tensors.append(tensor)
```
%% Cell type:markdown id: tags:
#### Applying the PTQ to the model ...
Now that everything is ready, we can call the PTQ routine ! Note that after the quantization we need to update the scheduler.
%% Cell type:code id: tags:
``` python
aidge_quantization.quantize_network(aidge_model, NB_BITS, tensors)
scheduler = aidge_core.SequentialScheduler(aidge_model)
```
%% Cell type:markdown id: tags:
#### Running some quantized inferences ...
Now that our network is quantized, what about testing some inferences ? Let's do so, but before, we need not to forget that our 8-bit network expect 8-bit inputs ! We thus need to rescale the input tensors ...
%% Cell type:code id: tags:
``` python
scaling = 2**(NB_BITS-1)-1
for i in range(NB_SAMPLES):
samples[i] = np.round(samples[i] * scaling)
```
%% Cell type:markdown id: tags:
We can now perform our quantized inferences ...
%% Cell type:code id: tags:
``` python
print('\n EXAMPLE QUANTIZED INFERENCES :')
for i in range(10):
input_array = np.reshape(samples[i], (1, 1, 28, 28))
output_array = propagate(aidge_model, scheduler, input_array)
print(labels[i] , ' -> ', np.round(output_array, 2))
```
%% Cell type:markdown id: tags:
### Computing the quantized accuracy ...
Just as we've done for the initial network, we can compute the quantized model accuracy ...
%% Cell type:code id: tags:
``` python
accuracy = compute_accuracy(aidge_model, samples[0:NB_SAMPLES], labels)
print(f'\n QUANTIZED MODEL ACCURACY : {accuracy * 100:.3f}%')
```
%% Cell type:markdown id: tags:
#### Work is done !
We see that a 8-bit PTQ does not affect the accuracy of our model ! This result shows that a proper quantization algorithm can be used to deploy a Neural Network on very small devices, where manipulating bytes is optimal. We encourage you to run this notebook again with even more aggressive quantization values !
%% Cell type:code id: tags:
``` python
print('That\'s all folks !')
```
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment

Copyright © Eclipse Foundation, Inc. All Rights Reserved.     Privacy Policy | Terms of Use | Copyright Agent