Draft: [Issue #251] : WIP: Softmax Backward implementation for cpu
2 unresolved threads
Context
Through this Merge-Request is illustrated the code for the backward cpu implementation of Softmax operator (find related issue here).
The current code is only preliminary, it is likely not mature enough. Thus, the purpose of this MR is only to get feedback for further development. In addition, some implementation choice have been made through the issue, choice that might not be appropriate. Please validate or invalidate them.
Modified files
-
include/aidge/backend/cpu/operator/SoftmaxImpl.hpp
,include/aidge/backend/cpu/operator/SoftmaxImpl_kernels.hpp
,src/operator/SoftmaxImpl.cpp
: add backward implementation for the operator -
unit_tests/operator/Test_SoftmaxImpl.cpp
: Add unit tests for softmax backward (once the implementation validated, additional tests cases will be added)
TODO
-
Backward Implementation validated -
Add other test cases
Merge request reports
Activity
Filter activity
assigned to @adamaroni
requested review from @pineapple
mentioned in issue aidge#251
114 } 115 116 117 118 TEST_CASE("[cpu/operator] Softmax(backward)", "[Softmax][CPU]") { 119 SECTION("1D Tensor") { 120 std::shared_ptr<Softmax_Op> op = std::make_shared<Softmax_Op>(0); 121 op->setDataType(DataType::Float32); 122 op->setBackend("cpu"); 123 124 std::shared_ptr<Tensor> softMaxForwardInputTensor = 125 std::make_shared<Tensor>(Array1D<float,3> { {3.0, 1.0, 0.2} }); 89 { 90 const O* softmaxOut = static_cast<const O*>(softmaxOut_); 91 const O* target = static_cast<const O*>(target_); 92 I* dL = static_cast<I*>(gradientLoss_); 93 94 // Compute the number of elements after the softmax axis (post-axis size) 95 std::size_t postAxisElems = 1; 96 for (std::size_t i = axisIdx + 1; i < inputDims.size(); ++i) { 97 postAxisElems *= inputDims[i]; 98 } 99 100 // Compute the number of elements after the softmax axis (pre-axis size) 101 std::size_t preAxisElems = 1; 102 for (std::size_t i = 0; i < axisIdx; ++i) { 103 preAxisElems *= inputDims[i]; 104 } added Feature 🚀 StatusWork in Progress labels
added 1 commit
- e713e7df - [Issue #251] : WIP: Softmax Backward implementation for cpu
added 1 commit
- f57cdbda - [Issue #251] : Softmax Backward implementation for cpu
Please register or sign in to reply