Multiple fixes to enable multi-GPUs forward execution
Some of these fix allow multi-GPU forward execution of cloned graphs.
-
Correctly escape HTML special characters in GraphView::save()
; -
Added const
version ofrefCastFrom()
; -
Make GraphView::add()
from another GraphView preserve ordered inputs/outputs; -
Operators attributes were not properly cloned (should fix aidge!105 (merged)).
This now works:
aidge_model0 = aidge_onnx.load_onnx("mymodel.onnx")
aidge_model1 = aidge_model0.clone()
aidge_model0.set_backend("cuda", 0)
aidge_model1.set_backend("cuda", 1)
aidge_models = aidge_core.GraphView()
aidge_models.add(aidge_model0)
aidge_models.add(aidge_model1)
scheduler = aidge_core.ParallelScheduler(aidge_models)
scheduler.forward(True, [in_model0, in_model1])
The models aidge_model0
and aidge_model1
will be run in parallel on 2 GPUs, yet in a synchronized fashion (the same layers are executed at the same time). This should be generalizable to any number of device/backend combinations!
This also works:
aidge_model0 = aidge_onnx.load_onnx("mymodel.onnx")
aidge_model1 = aidge_model0.clone()
aidge_model0.set_backend("cuda", 0)
aidge_model1.set_backend("cuda", 1)
scheduler0 = aidge_core.SequentialScheduler(aidge_model0)
scheduler1 = aidge_core.SequentialScheduler(aidge_model1)
import concurrent.futures
with concurrent.futures.ThreadPoolExecutor(max_workers=2) as e:
e.submit(scheduler0.forward, True, [in_model0])
e.submit(scheduler1.forward, True, [in_model1])
Here models aidge_model0
and aidge_model1
are run in parallel fully asynchronously.
Edited by Olivier BICHLER
Merge request reports
Activity
Filter activity
changed milestone to %aidge v0.6.0
added Fix 🔥🔥 label
assigned to @olivierbichler
added StatusReview Ready label
mentioned in merge request aidge!105 (merged)
Please register or sign in to reply