The parent of all zephyr jobs is now unified to enable easier
extensibility later on.
Update documentation.

Signed-off-by: Davide Gardenal <davide.gardenal@huawei.com>

Unqualified jobs, which rely on the default compiler, are gone.
Qualified jobs explicitly configure either clang or gcc and use the
appropriate suffix. Qualified jobs have needs/depends set to empty to
both avoid waiting for previous stages to start and to avoid pulling in
artifacts from the previous stage.

Qualified jobs inhabit two new lanes: build-gcc and build-clang, to
visually break the already long column of "build" jobs.

Job ordering within a lane is not exactly matching, gcc-clang. This is
noticeable with the jobs using the "extra" suffix. The reason for this
is unclear.

Some jobs are marked as broken as they do not build with clang. Allow
them to fail and make them start manually until more patches arrive.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

Chase Qi authored 2 years ago and Stevan Radaković committed 2 years ago

Signed-off-by: Chase Qi <chase.qi@linaro.org>

Signed-off-by: Chase Qi <chase.qi@linaro.org>

Signed-off-by: Chase Qi <chase.qi@linaro.org>

Introduce variable CI_ONIRO_JOB_ARTIFACTS to allow build job to
customize what artifacts it needs to deploy.

Signed-off-by: Chase Qi <chase.qi@linaro.org>

Signed-off-by: Chase Qi <chase.qi@linaro.org>

After splitting, also ci need to build separate MACHINE variables.

Signed-off-by: Ettore Chimenti <ettore.chimenti@seco.com>

Signed-off-by: Chase Qi <chase.qi@linaro.org>

Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>

Blueprints will have a new home:
https://booting.oniroproject.org/distro/meta-oniro-blueprints



Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>

Ensure we test for the newly created distro and image.

Signed-off-by: Stefan Schmidt <stefan.schmidt@huawei.com>

With the move to booting.oniroproject.org there is no public download
and sstate-cache.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

Andrei Gherzan authored 3 years ago and Stefan Schmidt committed 3 years ago

Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

Zephyr and FreeRTOS jobs do not require 16GB of RAM to build and
constitute the bulk of our CI (by number) quota usage. By moving them to
smaller instances, with 4GB of RAM, we can use the remaining quota
better.

On the infrastructure side we now have a new auto-scaling pool, with
s3.large.2 instances. Those join the existing s3.xlarge.4 (formerly
known as "heavy") and s3.large.8 (formerly known as "light").

The .workspace job uses the s2.large.8 by default, making a safe (but
expensive) default. Specific jobs deviate as follows:

 - zephyr and freertos jobs use s3.large.2
 - two blueprint jobs with more recipes use s3.xlarge.4

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

The issue is marked as closed now. We should be able to work without
allow_failure here again.

Signed-off-by: Stefan Schmidt <stefan.schmidt@huawei.com>

Most things are moved unchanged. The relatively few changes are to the
documentation, as the split between the meta-ohos and manifest is gone,
as well as to the actual CI definitions for the workspace directory,
since the sources/ sub-directory is gone, and there was some
backwards-compatibility code that no longer makes sense to maintain.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

The .ostc-ci directory becomes .oniro-ci

The various CI variables names are now all uniformly named CI_ONIRO_.
Some specific variables were further trimmed to avoid using "CI" twice.

Remaining uses of OSTC and OHOS are for URLs, container images and
branch names.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

Zygmunt Krynicki authored 3 years ago and Stefan Schmidt committed 3 years ago

The job will now generate an image artifact. The images will be consumed
by the SystemOTA CI solution shortly.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

This reverts commit 86a144eb.

As mentioned in the commit this was a temporary workaround until the
needed change could land in the manifest repository. This has happend
now:

https://git.ostc-eu.org/OSTC/OHOS/manifest/-/commit/8d604fb7cd63f6247c3071989005d251f0a068bc



Signed-off-by: Stefan Schmidt <stefan.schmidt@huawei.com>

Our QEMU builds were using the .build-image job, which archives all of
the deploy/image directory. This directory started to inflate recently,
in ways I did not explore.

Given that this breaks CI and is split across repositories the
.build-wic-image job was pushed into the manifest repository. Now with
the new job available we can switch the x86 and x86_64 QEMU image builds
to use the new .build-wic-image job, which saves a subset of the
artifacts that CI actually consumes downstream.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

For the images renames to fully pass CI, we need to have a change in the
manifest repo. As these two MR would block each other before we could
merge them, we override the image names here, land the rename, do the
actual change in the manifest repo and finally clean this up again.

To be reverted when this change lands in the manifest repository.

Signed-off-by: Stefan Schmidt <stefan.schmidt@huawei.com>

Signed-off-by: Stefan Schmidt <stefan.schmidt@huawei.com>

Adjust to the new DISTRO names introduced in the repo.

Signed-off-by: Stefan Schmidt <stefan.schmidt@huawei.com>

With the introduction of $OHOS_RUNNER_TAG on the manifest repository, we
can now reliably add a custom tag to all bitbake builds that do not
customize the tag list.

Use the new tag on the two jobs which provide a custom list to be able
to redirect them to a special runner as well.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

With the .build-linux-matrix job now available from the manifest
repository, we can now switch linux-seco-intel-b68,
linux-seco-imx8mm-c61 and linux-raspberrypi4-64 to build the base and
extras images separately, as distinct jobs.

In the end the exact same thing happens but we have more parallelism
since the jobs can be picked up by separate workers.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

With the recent improvements to the CI infrastructure we are seeing
relatively fast build times for most of our jobs. The exception to this
rule are the relatively-larger blueprint build jobs.

Assign the set of tags: "bitbake-friendly", large-disk (legacy) and the
new "heavy" tag. The tag will route the job to a runner with
significantly more resources. This shoudl help even out the total build
time.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

This build is reliably failing to fetch when invoked without download
cache constructed earlier. I've filed an issue to track this
but this blocks the auto-scaling CI system from going to production

Related-To: https://git.ostc-eu.org/OSTC/OHOS/meta-ohos/-/issues/132


Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

Signed-off-by: Chase Qi <chase.qi@linaro.org>

This reverts commit f7eb3cb0.

Signed-off-by: Chase Qi <chase.qi@linaro.org>

There are indications that upload pipe between GitLab workers and the
OSTC GitLab instance is slowing down CI, making an otherwise fast
machine idle while it waits for network traffic.

Since the images were only consumed by LAVA and for the moment, LAVA is
disabled, switch jobs using .build-image to .build-recipe. The only
difference is that now artifacts are discarded immediately after the
build.

This is a temporary workaround that can be reverted once the situation
is debugged further.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

To avoid build breaks to stay unnoticed (as happened recently) a simple
image build for the blueprint image is added.

Fixes https://git.ostc-eu.org/OSTC/OHOS/meta-ohos/-/issues/111



Signed-off-by: Stefan Schmidt <stefan.schmidt@huawei.com>

Signed-off-by: Chase Qi <chase.qi@linaro.org>

For LAVA testing purposes we need the built images available as
artifacts.

Signed-off-by: Stevan Radaković <stevan.radakovic@linaro.org>

Add Qemu test jobs.
Add images for Qemu builds in order to boot them with LAVA.
Also fix a problem where CI jobs where not added to the pipeline on
changes within .ostc-ci dir.

Signed-off-by: Stevan Radaković <stevan.radakovic@linaro.org>

Chase Qi authored 3 years ago and Stevan Radaković committed 3 years ago

Signed-off-by: Chase Qi <chase.qi@linaro.org>

Andrei Gherzan authored 3 years ago and Stefan Schmidt committed 3 years ago

Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>

Signed-off-by: Nagesh Shamnur <nagesh.shamnur@huawei.com>