Skip to content
Snippets Groups Projects

Multiple fixes to enable multi-GPUs forward execution

Merged Olivier BICHLER requested to merge htmlescape into dev

Some of these fix allow multi-GPU forward execution of cloned graphs.

  • Correctly escape HTML special characters in GraphView::save();
  • Added const version of refCastFrom();
  • Make GraphView::add() from another GraphView preserve ordered inputs/outputs;
  • Operators attributes were not properly cloned (should fix aidge!105 (merged)).

This now works:

aidge_model0 = aidge_onnx.load_onnx("mymodel.onnx")
aidge_model1 = aidge_model0.clone()
aidge_model0.set_backend("cuda", 0)
aidge_model1.set_backend("cuda", 1)

aidge_models = aidge_core.GraphView()
aidge_models.add(aidge_model0)
aidge_models.add(aidge_model1)

scheduler = aidge_core.ParallelScheduler(aidge_models)
scheduler.forward(True, [in_model0, in_model1])

The models aidge_model0 and aidge_model1 will be run in parallel on 2 GPUs, yet in a synchronized fashion (the same layers are executed at the same time). This should be generalizable to any number of device/backend combinations!

This also works:

aidge_model0 = aidge_onnx.load_onnx("mymodel.onnx")
aidge_model1 = aidge_model0.clone()
aidge_model0.set_backend("cuda", 0)
aidge_model1.set_backend("cuda", 1)

scheduler0 = aidge_core.SequentialScheduler(aidge_model0)
scheduler1 = aidge_core.SequentialScheduler(aidge_model1)

import concurrent.futures

with concurrent.futures.ThreadPoolExecutor(max_workers=2) as e:
    e.submit(scheduler0.forward, True, [in_model0])
    e.submit(scheduler1.forward, True, [in_model1])

Here models aidge_model0 and aidge_model1 are run in parallel fully asynchronously.

Edited by Olivier BICHLER

Merge request reports

Merge request pipeline #70465 passed

Merge request pipeline passed for 7957949f

Test coverage 46.75% (0.45%) from 2 jobs

Merged by Olivier BICHLEROlivier BICHLER 1 month ago (Apr 14, 2025 7:26am UTC)

Loading

Pipeline #70667 passed

Pipeline passed for 7957949f on dev

Test coverage 46.75% (0.45%) from 2 jobs

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
Please register or sign in to reply
Loading