CI: Attempt to throttle load caused by builds better

Use the support that both make and ninja have to hold back on starting new parallel jobs if load is higher than a set number.

Use the $CI_ONIRO_NUM_CPUS variable if set, or the output of nproc command to set sane values.

For most cases, this means that CI runners don't have to set anything, as nproc output should be fine.

Closes eclipse/oniro-core/meta-openharmony#33

Signed-off-by: Esben Haabendal esben.haabendal@huawei.com

requested review from @pastanki, @zyga, and @robertd

assigned to @esben

approved this merge request

approved this merge request

approved this merge request

enabled an automatic merge when the pipeline for c36243fd succeeds

I created issue related to this. https://gitlab.eclipse.org/eclipse/oniro-core/meta-openharmony/-/issues/33

aborted the automatic merge because source branch was updated

added 1 commit

• 2864b938 - CI: Apply parallel job limits to ohos test jobs also

Compare with previous version

added 1 commit

• d91f1fec - CI: Allow use with shallow clone CI setup

Compare with previous version

enabled an automatic merge when the pipeline for d91f1fec succeeds

changed title from CI: Attempt to throttle load caused builds better to CI: Attempt to throttle load caused by builds better

changed the description

marked this merge request as draft

I think that the load based limit set by this change is hurting us a lot, as it is evaluated against the total system load average, so mostly causing make and ninja to not do anything.

aborted the automatic merge because source branch was updated

added 7 commits

• 02fe8d71 - CI: Fix problem with handling updates to mirror in China
• 6ce9d78b - CI: Improve parallel jobs handling
• 3c49407b - CI: Apply parallel job limits to ohos test jobs also
• a1d242ed - CI: Allow use with shallow clone CI setup
• 7603b0b2 - CI: Add support for $CI_PARALLEL_BUILD_MAX_JOBS
• 603a49fb - CI: Add support for $CI_ONIRO_PARALLEL_BUILD_MAX_LOADAVG
• 40987c58 - prebuilts: Fix support for multiple --ninja-args arguments

Compare with previous version

added 4 commits

• 3a2253bc - CI: Allow use with shallow clone CI setup
• ca708ccf - CI: Add support for $CI_PARALLEL_BUILD_MAX_JOBS
• be87006e - CI: Add support for $CI_ONIRO_PARALLEL_BUILD_MAX_LOADAVG
• 696f12df - prebuilts: Fix support for multiple --ninja-args arguments

Compare with previous version

mentioned in merge request !37 (closed)

added 5 commits

• c84539d0 - CI: Add support for $CI_ONIRO_PARALLEL_BUILD_MAX_JOBS
• 8503494b - CI: Add support for $CI_ONIRO_PARALLEL_BUILD_MAX_LOADAVG
• 176b858e - prebuilts: Fix support for multiple --ninja-args arguments
• 0c39610b - manifest: Bump oniro and oe-core
• 2ddee140 - CI: Set MAXLOAD_NINJA environment variable from runner setting

Compare with previous version

marked this merge request as ready

enabled an automatic merge when the pipeline for 2ddee140 succeeds

mentioned in commit c282b9e0

merged