Skip to content

Dataloader

Thibault Allenet requested to merge dataloader into dev

This MR is for data handling. Issue aidge#31 (closed) explain the added features. In aidge_core, the features added are :

  1. Stimuli and StimuliImpl - A class wrapping a data sample. Stimuli has two functioning modes. The first mode enables to load data samples from a dataPath and optionnaly store the data in-memory. The second mode enables to store a data sample that was already loaded in memory. StimuliImpl provides the implementation to load a specific file extension.
  2. Database - An abstract class representing a map from a key to several Stimulis (for example a stimuli image and a stimuli label). All databases should inherit from this class. All subclasses should overwrite :cpp:function:Database::getItem to fetch data from a given index.
  3. DataProvider - Compose batches by fetching data from a given database.
  4. Database and DataProvider python biding
  5. Connection of tensors to the GraphView handled by Scheduler::connectInputs

ConnectInputs uses the OrderedInputs feature of the graphView.

In the case where data is passed during the call of Scheduler::forward, the call of GraphView::forwardDims (before forward) does not have the input dimensions. Thus GraphView::forwardDims have now an optional argument to pass the dimension of the input tensors.


Implement the solution presented in issue aidge_backend_opencv#3 (closed).

  • Change the constructor of TensorImpl to take the dimensions of the tensor as argument instead of the number of elements. For tensorImpl and tensorImpl_cpu.
  • Adapt pybind_tensor with dimensions instead of number of elements
Edited by Thibault Allenet

Merge request reports