Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>

With the .build-linux-matrix job now available from the manifest
repository, we can now switch linux-seco-intel-b68,
linux-seco-imx8mm-c61 and linux-raspberrypi4-64 to build the base and
extras images separately, as distinct jobs.

In the end the exact same thing happens but we have more parallelism
since the jobs can be picked up by separate workers.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

With the recent improvements to the CI infrastructure we are seeing
relatively fast build times for most of our jobs. The exception to this
rule are the relatively-larger blueprint build jobs.

Assign the set of tags: "bitbake-friendly", large-disk (legacy) and the
new "heavy" tag. The tag will route the job to a runner with
significantly more resources. This shoudl help even out the total build
time.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

For now the gateway blueprint testing is limited to raspberrypi4-64. We
are not actively excluding other boards, but at least wifi and a USB
host for an OpenThread dongle is needed. Better state the tested board
only in the docs until we expand the testing.

Signed-off-by: Stefan Schmidt <stefan.schmidt@huawei.com>

Our download cache is temporarily out of order. Before it returns,
disabling it should speed up fresh builds, as the cache is gone and is
accessed repeatedly which wastes time.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

FreeRTOS switched to use main branch instead of master while our code
still looks for master. Overriding of bb class is needed for the change
thus it requires change in the order of layers.

Closes: https://git.ostc-eu.org/OSTC/OHOS/meta-ohos/-/issues/132


Signed-off-by: Pavel Zhukov <pavel.zhukov@huawei.com>
Upstream status: Backport
Upstream commit: d4bd9d748a4223856bf92dd35ca088d1f319d189

This build is reliably failing to fetch when invoked without download
cache constructed earlier. I've filed an issue to track this
but this blocks the auto-scaling CI system from going to production

Related-To: https://git.ostc-eu.org/OSTC/OHOS/meta-ohos/-/issues/132


Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

This should help making re-run of do_configure safe.

Signed-off-by: Esben Haabendal <esben.haabendal@huawei.com>

Eilís Ní Fhlannagáin authored 3 years ago and Eilís Ní Fhlannagáin committed 3 years ago

Looking at allscenarios-image-common.inc I see some devel/dbg packages
in there. These really should be pulled into a development or debug
image as images that endusers would build off of for a production
environment should only contain the minimal packages needed to run the system.

This patch removes gdbserver and packagegroup-net-tools (iperf3) from common
and puts them along with other useful EXTRA_IMAGE_FEATURES into two new images:

allscenarios-image-base-dev
allscenarios-image-extra-dev

Due to the size of the dbg-pkgs and dev-pkgs,
allscenarios-image-extra-dev needs a bump of it's ROOT_PARTITION_SIZE
else wic creation explodes.

Closes <issues 123>

Signed-off-by: Eilís Ní Fhlannagáin <elizabeth.flanagan@huawei.com>

Signed-off-by: Esben Haabendal <esben.haabendal@huawei.com>

It is preferable to allow checker function to run until bbfatal is executed
for a meaningful error message to be printed for a better user experience.

Signed-off-by: Robert Drab <robert.drab@huawei.com>

gn docs state that the output dir should be passed as a last argument of the
gn gen command without any preceding switch

Signed-off-by: Robert Drab <robert.drab@huawei.com>

There's instance level bug template file which is copy of the project
level one. Directory kept for future overriding if needed

Closes: https://git.ostc-eu.org/OSTC/OHOS/meta-ohos/-/issues/127


Signed-off-by: Pavel Zhukov <pavel.zhukov@huawei.com>

Signed-off-by: Chase Qi <chase.qi@linaro.org>

This reverts commit f7eb3cb0.

Signed-off-by: Chase Qi <chase.qi@linaro.org>

The code detects when boot assets would overwrite something already
deployed/installed in the rootfs. This helps to avoid breaking packaging
at runtime. On the other hand, if IMAGE_BOOT_FILES has duplicates, this
check will fail when trying to install the same file multiple times. To
avoid this, treat the install_task as a set.

Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>

This implements logic as a rootfs postprocess command where the set of
boot artifacts are provided as part of the root filesystem. The reason
is that the system update components assume only the root filesystem as
part of the update payload while boot files are also needed depending on
the BSP. For example, RaspberryPi update process would update the kernel
and dtbs on the boot partition too.

The boot files are copied to the path defined by
ROOTFS_BOOT_ARTIFACTS_PATH, relative to the root of the root filesystem.

SOme of the logic in this mechanism are based on the current
implementation of the bootimg-partition wic plugin.

Fixes https://git.ostc-eu.org/OSTC/OHOS/meta-ohos/-/issues/104



Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>

Signed-off-by: Stefan Schmidt <stefan.schmidt@huawei.com>

Bernhard Rosenkränzer authored 3 years ago and Pavel Zhukov committed 3 years ago

python3-pyelftools 0.26 can't handle DWARF-5, which is generated by default
by modern toolchains (clang 12+, gcc 11+).

Update it to 0.27 with an extra patch taken from upstream git to support
DWARF-5 so we can use it (e.g. in the Zephyr kernel test suite) without
switching the toolchains back to DWARF-4.

Signed-off-by: Bernhard Rosenkränzer <bernhard.rosenkraenzer.ext@huawei.com>

We first convert over qemuarm, qemuarm64, qemux86 and qemux86-64 to use
the common kernel.

Signed-off-by: Amit Kucheria <amit.kucheria.ext@huawei.com>

These will be similar to linux-yocto, but customized to the ASOS kernel
policies and support our reference HW.

qemuarm and qemuarm64 builds aren't yet supported by wic, but we can
still start building the kernels from a common base.

Signed-off-by: Amit Kucheria <amit.kucheria.ext@huawei.com>

This rename also better reflects the purpose of the file since it
reflects ASOS kernel policies.

Signed-off-by: Amit Kucheria <amit.kucheria.ext@huawei.com>

We have merged in the inclusion of NetworkManager in the reference
images so we can disable networkd to avoid runtime conflict.

Fixes https://git.ostc-eu.org/OSTC/OHOS/meta-ohos/-/issues/112



Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>

The recently introduced fixes for networkmanager introduced an implicit
dependency on meta-networking.
This change allows use of meta-ohos-staging without meta-networking layer,
without changing behaviour when used together with meta-networking.

Signed-off-by: Esben Haabendal <esben.haabendal@huawei.com>

This problem showed when building nodejs and resulted in an error of
malformed archives:

libv8_initializers.a: error adding symbols: malformed archive

There was a corner case where the fd's have not been closed on thin
archives and nodejs was heavy enough to be linked to show this problem.

Upstream report and analysis can be seen here:
https://sourceware.org/bugzilla/show_bug.cgi?id=28138



Fix is already upstream and just a backport for 2.37 added here.

Signed-off-by: Stefan Schmidt <stefan.schmidt@huawei.com>

The cache is relatively network-heavy, as it attempts to check if one of
myriad of cache files is available and is somewhat misleading, since the
cache only works for the small subset of configurations that have been
cleared for legal re-distribution and have a public cache available.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

Fix crash on startup with networkmanager built with musl

Signed-off-by: Bernhard Rosenkränzer <bernhard.rosenkraenzer.ext@huawei.com>

Andrei Gherzan authored 3 years ago and Bernhard Rosenkränzer committed 3 years ago

This provides a fully fledged network manager in our reference images.

Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>

Andrei Gherzan authored 3 years ago and Bernhard Rosenkränzer committed 3 years ago

Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>
Signed-off-by: Bernhard Rosenkraenzer <bernhard.rosenkraenzer.ext@huawei.com> [adapted to current meta-ohos]

Andrei Gherzan authored 3 years ago and Bernhard Rosenkränzer committed 3 years ago

On musl, build would fail with:

| In file included from
../NetworkManager-1.22.10/shared/systemd/src/basic/env-util.c:11:
|
../NetworkManager-1.22.10/shared/systemd/src/basic/alloc-util.h:100:35:
error: static declaration of 'reallocarray' follows non-static
declaration
|   100 | _alloc_(2, 3) static inline void *reallocarray(void *p, size_t
need, size_t size) {
|       |                                   ^~~~~~~~~~~~
| In file included from
../NetworkManager-1.22.10/shared/nm-default.h:184,
|                  from
../NetworkManager-1.22.10/shared/systemd/sd-adapt-shared/nm-sd-adapt-shared.h:9,
|                  from
../NetworkManager-1.22.10/shared/systemd/src/basic/env-util.c:3:
|
[...]/build-linux/tmp/work/aarch64-poky-linux-musl/networkmanager/1.22.10-r0/recipe-sysroot/usr/include/stdlib.h:148:7:
note: previous declaration of 'reallocarray' with type 'void *(void *,
size_t,  size_t)' {aka 'void *(void *, long unsigned
 int,  long unsigned int)'}
|   148 | void *reallocarray (void *, size_t, size_t);
|       |       ^~~~~~~~~~~~

This is because musl does't provide reallocarray as part of malloc.h but
only as part of stdlib.h. Configure script was modified to check for its
definition in stdlib.h as well (backported patch).

Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>
Signed-off-by: Bernhard Rosenkraenzer <bernhard.rosenkraenzer.ext@huawei.com> [adapt filename in .bbappend to match the actual patch filename]

Due to custom option parser implementation weston accepts only one
argument of given type. As the result if multiple modules add --module
only last will be used. This fix introduces wrapper around modules in
terms of weston-init to prepare proper modules argument for weston and
moves systemd-notify module into weston-init module instead of command
line argument

Based on Poky revision #bdd30be1a3815f70062d8febca91eaf042a77c3d
Downstream changes: Adding add_weston_module function into weston-start
script and modify xwayland plugin to use new interface.

Submitted upstream:
https://lists.openembedded.org/g/openembedded-core/topic/patch_weston_wrapper_for/85453995
Ref: https://git.ostc-eu.org/OSTC/OHOS/meta-ohos/-/issues/110



Signed-off-by: Pavel Zhukov <pavel.zhukov@huawei.com>

In version 2.0.11 a potential security bug (CVE-2021-34431) was fixed.

Reports:
https://nvd.nist.gov/vuln/detail/CVE-2021-34431
https://bugs.eclipse.org/bugs/show_bug.cgi?id=573191

Fix:
https://github.com/eclipse/mosquitto/commit/42163634c72d41a1f12d299f54e00adf14520eb2



Updating to the latest stable update 2.0.12 here for our reciep to
collect all the fixes. The notice.html file got renamed to NOTICE.md,
adjust accordingly.

Signed-off-by: Stefan Schmidt <stefan.schmidt@huawei.com>

LAVA jobs are AFAIK not useful since the read-only images landed. Since
we've disabled publishing image artifacts earlier, let's properly
disable the LAVA jobs as well.

This commit should be reverted when LAVA jobs are working and the
slowness issue was debugged.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

There are indications that upload pipe between GitLab workers and the
OSTC GitLab instance is slowing down CI, making an otherwise fast
machine idle while it waits for network traffic.

Since the images were only consumed by LAVA and for the moment, LAVA is
disabled, switch jobs using .build-image to .build-recipe. The only
difference is that now artifacts are discarded immediately after the
build.

This is a temporary workaround that can be reverted once the situation
is debugged further.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com>

We revert this workaround as the existing poky plugin,
bootimg-pcbios.py, includes the needed support as per the last poky bump
in the manifest repository.

PartOf: https://git.ostc-eu.org/OSTC/OHOS/meta-ohos/-/issues/97



Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>

Ref: https://git.ostc-eu.org/OSTC/planning/devops/-/issues/70


Signed-off-by: Pavel Zhukov <pavel.zhukov@huawei.com>

The wpantund package is only needed for scenarios where the OpenThread
device is operating in network co-processor (NCP) mode. We are not using
NCP, but radio co-processor (RCP instead). The later one works with
ot-br-posix out of the box and we have no need for wpantund, as of now.

Signed-off-by: Stefan Schmidt <stefan.schmidt@huawei.com>

The patch is pending in the upstream PR queue for a while now. Include
it here until its merged upstream and we can update.

Signed-off-by: Stefan Schmidt <stefan.schmidt@huawei.com>

The CVE issue CVE-2020-8916 has been fixed in the upstream repo for a
long time and our recipe has this commit already included.

https://cve.mitre.org/cgi-bin/cvename.cgi?name=2020-8916



Mark the CVE as such.

Signed-off-by: Stefan Schmidt <stefan.schmidt@huawei.com>

Using current meta-java in a project configured for a riscv32 machine gives
the following error:

    ERROR: ExpansionError during parsing /home/esben/ostc/openharmony/poky/meta-java/recipes-core/openjdk/openjdk-7_99b00-2.6.5.bb
    Traceback (most recent call last):
      File "Var <LLVM_CONFIGURE_ARCH>", line 1, in <module>
      File "/home/esben/ostc/openharmony/poky/meta-java/classes/openjdk-build-helper.bbclass", line 86, in openjdk_build_helper_get_llvm_configure_arch(d=<bb.data_smart.DataSmart object at 0x7fc729388940>):
             else:
        >        if 'shark' in d.getVar('PACKAGECONFIG').split():
                     bb.warn("%s does not support %s in Shark builds yet" % (d.getVar('PN'), arch) );
    bb.data_smart.ExpansionError: Failure expanding variable LLVM_CONFIGURE_ARCH, expression was ${@openjdk_build_helper_get_llvm_configure_arch(d)} which triggered exception AttributeError: 'NoneType' object has no attribute 'split'

Which is caused by the attempt to expand PACKAGECONFIG variable, which is only
set for selected architectures.  Defaulting it to empty is a sane fix to that.

A patch for meta-java doing the same have been submitted.

Signed-off-by: Esben Haabendal <esben.haabendal@huawei.com>