Draft: .oniro-ci: use CI_ONIRO_RUNNER_EPHEMERAL_STORAGE

When the variable is defined it ought to point to ephemeral storage that is "better" than the default storage provided by the runner.

This may be, for example, memory-backed file system designed to seed things up and avoid wearing down persistent storage.

I've tested this with a local runner using 100GB tmpfs partition as ephemeral storage. Due to package accumulation that amount fills up to about 60GB for oniro-image-base x86-64 build.

With DEPLOY_DIR on separate non-ram filesystem, tmpfs usage is very good, easily allowing 2-4 concurrent builds on the same 100GB tmpfs.

Signed-off-by: Zygmunt Krynicki zygmunt.krynicki@huawei.com

.oniro-ci/build-generic.yaml

@@ -28,7 +28,7 @@
     # Create scratch space, being careful not to pollute the working directory.
     # This is done so that we are not attached to the contents of
     # $CI_PROJECT_DIR which contains something that GitLab CI prepared for us.
-    - SCRATCH_DIR="$(mktemp -p /tmp -d workspace.XXXXXXXXXX)"
+    - SCRATCH_DIR="$(mktemp -p "${CI_ONIRO_RUNNER_EPHEMERAL_STORAGE:-/tmp}" -d workspace.XXXXXXXXXX)"
     - echo "$SCRATCH_DIR" > "$CI_PROJECT_DIR"/.scratch-dir-name
     - cd "$SCRATCH_DIR"
 

[Esben Haabendal]

Why is this changed from CI_ONIRO_RUNNER_PERSISTENT_STORAGE (which is what we still use in meta-openharmony) ?

[Zygmunt Krynicki]

I don't know why you would want to create the whole workspace there. The idea is that instead of using the default docker storage, you create the workspace in tmpfs, and in result, perform all the builds there, for a huge speedup.

If you really use persistent storage for the repo workspace in meta-openharmony, can you explain how is that useful and how do you deal with concurrency issues (as that storage is shared by many builds)

[Esben Haabendal]

Would it be possible to get away with mounting tmpfs in /build ?

[Zygmunt Krynicki]

It depends on how the jobs are written. The .workspace job does not really use the default build dir. In addition, in practice, tuning is useful (unless you have ridiculous amount of ram on hand). In my case I've defined the tmpfs for tmp/ but moved tmp/deploy back to docker storage as that only grows and is huge, eating lots of memory for almost no gain.