Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

The .ostc-ci directory becomes .oniro-ci

The various CI variables names are now all uniformly named CI_ONIRO_.
Some specific variables were further trimmed to avoid using "CI" twice.

Remaining uses of OSTC and OHOS are for URLs, container images and
branch names.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

Zygmunt Krynicki authored 3 years ago and Stefan Schmidt committed 3 years ago

The job will now generate an image artifact. The images will be consumed
by the SystemOTA CI solution shortly.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

This reverts commit 86a144eb.

As mentioned in the commit this was a temporary workaround until the
needed change could land in the manifest repository. This has happend
now:

https://git.ostc-eu.org/OSTC/OHOS/manifest/-/commit/8d604fb7cd63f6247c3071989005d251f0a068bc



Signed-off-by: Stefan Schmidt <stefan.schmidt@huawei.com>

Our QEMU builds were using the .build-image job, which archives all of
the deploy/image directory. This directory started to inflate recently,
in ways I did not explore.

Given that this breaks CI and is split across repositories the
.build-wic-image job was pushed into the manifest repository. Now with
the new job available we can switch the x86 and x86_64 QEMU image builds
to use the new .build-wic-image job, which saves a subset of the
artifacts that CI actually consumes downstream.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

For the images renames to fully pass CI, we need to have a change in the
manifest repo. As these two MR would block each other before we could
merge them, we override the image names here, land the rename, do the
actual change in the manifest repo and finally clean this up again.

To be reverted when this change lands in the manifest repository.

Signed-off-by: Stefan Schmidt <stefan.schmidt@huawei.com>

Signed-off-by: Stefan Schmidt <stefan.schmidt@huawei.com>

Adjust to the new DISTRO names introduced in the repo.

Signed-off-by: Stefan Schmidt <stefan.schmidt@huawei.com>

With the introduction of $OHOS_RUNNER_TAG on the manifest repository, we
can now reliably add a custom tag to all bitbake builds that do not
customize the tag list.

Use the new tag on the two jobs which provide a custom list to be able
to redirect them to a special runner as well.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

With the .build-linux-matrix job now available from the manifest
repository, we can now switch linux-seco-intel-b68,
linux-seco-imx8mm-c61 and linux-raspberrypi4-64 to build the base and
extras images separately, as distinct jobs.

In the end the exact same thing happens but we have more parallelism
since the jobs can be picked up by separate workers.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

With the recent improvements to the CI infrastructure we are seeing
relatively fast build times for most of our jobs. The exception to this
rule are the relatively-larger blueprint build jobs.

Assign the set of tags: "bitbake-friendly", large-disk (legacy) and the
new "heavy" tag. The tag will route the job to a runner with
significantly more resources. This shoudl help even out the total build
time.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

This build is reliably failing to fetch when invoked without download
cache constructed earlier. I've filed an issue to track this
but this blocks the auto-scaling CI system from going to production

Related-To: https://git.ostc-eu.org/OSTC/OHOS/meta-ohos/-/issues/132


Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

Signed-off-by: Chase Qi <chase.qi@linaro.org>

This reverts commit f7eb3cb0.

Signed-off-by: Chase Qi <chase.qi@linaro.org>

LAVA jobs are AFAIK not useful since the read-only images landed. Since
we've disabled publishing image artifacts earlier, let's properly
disable the LAVA jobs as well.

This commit should be reverted when LAVA jobs are working and the
slowness issue was debugged.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

There are indications that upload pipe between GitLab workers and the
OSTC GitLab instance is slowing down CI, making an otherwise fast
machine idle while it waits for network traffic.

Since the images were only consumed by LAVA and for the moment, LAVA is
disabled, switch jobs using .build-image to .build-recipe. The only
difference is that now artifacts are discarded immediately after the
build.

This is a temporary workaround that can be reverted once the situation
is debugged further.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

To avoid build breaks to stay unnoticed (as happened recently) a simple
image build for the blueprint image is added.

Fixes https://git.ostc-eu.org/OSTC/OHOS/meta-ohos/-/issues/111



Signed-off-by: Stefan Schmidt <stefan.schmidt@huawei.com>

Signed-off-by: Chase Qi <chase.qi@linaro.org>

Hidden job .lava-test and .lava-report should have same set of rules
as the .bitbake-workspace job (which has the rules overridden for this
project).

Signed-off-by: Stevan Radaković <stevan.radakovic@linaro.org>

For LAVA testing purposes we need the built images available as
artifacts.

Signed-off-by: Stevan Radaković <stevan.radakovic@linaro.org>

Add Qemu test jobs.
Add images for Qemu builds in order to boot them with LAVA.
Also fix a problem where CI jobs where not added to the pipeline on
changes within .ostc-ci dir.

Signed-off-by: Stevan Radaković <stevan.radakovic@linaro.org>

Chase Qi authored 3 years ago and Stevan Radaković committed 3 years ago

Signed-off-by: Chase Qi <chase.qi@linaro.org>

Andrei Gherzan authored 3 years ago and Stefan Schmidt committed 3 years ago

Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>

Signed-off-by: Nagesh Shamnur <nagesh.shamnur@huawei.com>

The OS will start to depend on a specific partition table and having to
maintain multiple qemu setups might not prove useful. Also, we have
encountered issues trying to boot wic images (disk images as opposed to
root filesystems) with qemu ARM machines.

Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>

Scheduled builds will allow us to re-test what's in HEAD without opening
a pull request. Making drafts start in manual mode will conserve
resources without limiting one's ability to iterate on selective
targets.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

Include the machines-and-flavours file explicitly, from the local
repository.  It is already included via the build-generic.yaml
project-include above, but including it again here allows GitLab to pick
up the changes to the definition straight away. The include done via the
manifest repository is using the default branch, so it cannot pick up
changes to this file as it is being modified by incoming branches.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

Raspberry Pi 4 was added a while ago but is not a part of CI.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

Signed-off-by: Wojciech Zmuda <wojciech.zmuda@huawei.com>

Currently machines and flavours are defined in the meta-ohos repository
but their CI pipelines are defined in the manifest repository. Move the
definitions of the various build jobs here to be included from the
manifest repository internally.

This way the public interface is unchanged but patches such as the
recent Raspberry Pi 4 support can enable a new MACHINE configuration and
a new CI job all in one go, to the extent that in-progress CI will pick
it up.

Closes: https://git.ostc-eu.org/OSTC/OHOS/meta-ohos/-/issues/44


Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

The manifest repository now offers new hidden jobs, that make
customization and reuse easier. Those allow us to remove bulk of the
custom logic, again, reducing everything to mere configuration.

The build-docs and update-docs jobs are entirely replaced by .build-docs
and .aggregate-docs (name change without logic change).

Custom .build job is split into smaller changes to the .workspace and
.bitbake-workspace jobs. Lastly, the variable OHOS_CI_GIT_REPO_PATH
is no longer relative to sources/ but includes it explicitly.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

There are two separate issues here:

- CI_BRANCH_NAME doesn't exist, we want CI_COMMIT_BRANCH instead.
- The rules engine doesn't handle quotes in the normal sense.
Quotes are only allowed on literals, not variable references.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

CI_BRANCH_NAME doesn't exist

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

The rules engine doesn't handle quotes in the normal sense.
Quotes are only allowed on literals, not variable references.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

Set rule to run our build pipeline for git tags as well.

Signed-off-by: Stefan Schmidt <stefan.schmidt@huawei.com>

When documentation changes, trigger the docs pipeline to update the
aggregated documentation view.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

Zygmunt Krynicki authored 4 years ago and Andrei Gherzan committed 4 years ago

The implicit test stage is a scheduled to run after all the jobs from
the build stage succeed. This introduces a unneeded dependency between
building the code and building the documentation.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

The manifest repository no longer contains distinct default.xml and
develop.xml manifest files. That role has been delegated to distinct git
branches.

Switch to the default manifest so that the deprecated compatibility
module which has the same effect, can be removed.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

It seems that `**` matches only the directory parts, `*` is needed to
match files contained therein.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

CI process in the meta-ohos repository is rather heavy, with large
number of long-running build jobs. The pipeline system allows
restricting job startup to a set of rules, and to further restrict each
rule to trigger only when one of a set of given files is modified.

This should allow efficient modification pipeline that only touches the
documentation or the meta-layers, without needlessly running unrelated
testing.

This is documented at https://docs.gitlab.com/ee/ci/yaml/#ruleschanges

Closes: https://git.ostc-eu.org/OSTC/OHOS/meta-ohos/-/issues/34


Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>