Skip to content
Snippets Groups Projects

Draft: [Issue #251] : WIP: Softmax Backward implementation for cpu

Open Adam Maroni requested to merge adamaroni/aidge_backend_cpu:feat_operator_SoftMax into dev
2 unresolved threads

Context

Through this Merge-Request is illustrated the code for the backward cpu implementation of Softmax operator (find related issue here).

The current code is only preliminary, it is likely not mature enough. Thus, the purpose of this MR is only to get feedback for further development. In addition, some implementation choice have been made through the issue, choice that might not be appropriate. Please validate or invalidate them.

Modified files

  • include/aidge/backend/cpu/operator/SoftmaxImpl.hpp, include/aidge/backend/cpu/operator/SoftmaxImpl_kernels.hpp , src/operator/SoftmaxImpl.cpp : add backward implementation for the operator
  • unit_tests/operator/Test_SoftmaxImpl.cpp : Add unit tests for softmax backward (once the implementation validated, additional tests cases will be added)

TODO

  • Backward Implementation validated
  • Add other test cases

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
114 }
115
116
117
118 TEST_CASE("[cpu/operator] Softmax(backward)", "[Softmax][CPU]") {
119 SECTION("1D Tensor") {
120 std::shared_ptr<Softmax_Op> op = std::make_shared<Softmax_Op>(0);
121 op->setDataType(DataType::Float32);
122 op->setBackend("cpu");
123
124 std::shared_ptr<Tensor> softMaxForwardInputTensor =
125 std::make_shared<Tensor>(Array1D<float,3> { {3.0, 1.0, 0.2} });
  • You should test your implementation with a greater number of dimensions in your input Tensor. With this 1D Tensor you have no way to know if "pre" and "post" dimensions are well handled.

  • Please register or sign in to reply
  • 89 {
    90 const O* softmaxOut = static_cast<const O*>(softmaxOut_);
    91 const O* target = static_cast<const O*>(target_);
    92 I* dL = static_cast<I*>(gradientLoss_);
    93
    94 // Compute the number of elements after the softmax axis (post-axis size)
    95 std::size_t postAxisElems = 1;
    96 for (std::size_t i = axisIdx + 1; i < inputDims.size(); ++i) {
    97 postAxisElems *= inputDims[i];
    98 }
    99
    100 // Compute the number of elements after the softmax axis (pre-axis size)
    101 std::size_t preAxisElems = 1;
    102 for (std::size_t i = 0; i < axisIdx; ++i) {
    103 preAxisElems *= inputDims[i];
    104 }
    • Comment on lines +95 to +104

      By passing down a pointer to input and output Tensors instead of raw pointers, you wouldn't need to compute "post" and "pre" strides at each function calls.

    • Please register or sign in to reply
  • Adam Maroni added 1 commit

    added 1 commit

    • e713e7df - [Issue #251] : WIP: Softmax Backward implementation for cpu

    Compare with previous version

  • Adam Maroni added 1 commit

    added 1 commit

    • f57cdbda - [Issue #251] : Softmax Backward implementation for cpu

    Compare with previous version

  • Please register or sign in to reply
    Loading