From 4a71d2b48ee1bafcabad3fcb234f31d3869389e9 Mon Sep 17 00:00:00 2001
From: Gururaj Shetty <gururaj.shetty@huawei.com>
Date: Mon, 21 Jun 2021 20:17:46 +0530
Subject: [PATCH] ci/device-testing: Add new topic on device testing
* Overview of on-device testing.
* Testing Infrastructure and how devices are added to the central system.
* What LAVA does.
* References for more details on LAVA and testing jobs.
This closes OSTC/planning/testing#50
Signed-off-by: Gururaj Shetty <gururaj.shetty@huawei.com>
---
ci/device-testing.rst | 82 +++++++++++++++++++++++++++++++++++++++++++
ci/index.rst | 1 +
2 files changed, 83 insertions(+)
create mode 100644 ci/device-testing.rst
diff --git a/ci/device-testing.rst b/ci/device-testing.rst
new file mode 100644
index 0000000..64a3b54
--- /dev/null
+++ b/ci/device-testing.rst
@@ -0,0 +1,82 @@
+.. SPDX-FileCopyrightText: Huawei Inc.
+..
+.. SPDX-License-Identifier: CC-BY-4.0
+
+On-device Testing
+=================
+
+Overview
+--------
+
+All Scenarios OS implements distributed device testing using Linaro Automation
+and Validation Architecture (LAVA). This architecture creates an environment
+where you can operate the necessary physical infrastructure responsible for
+testing development on real devices, like operating system boot-loader and
+kernel development, while sharing access to a project-specific software
+infrastructure used in the public cloud.
+
+How does the CI system work?
+----------------------------
+
+The system automatically performs a set of test jobs upon a new or
+modified pull request. Failed jobs stop the pipeline, allowing you to review
+build logs, reproduce and resolve the failure locally.
+The central system maintains a queue and schedules build and test jobs for the
+available workers. The workers may be auto-scaled, for example, virtual machines
+in the public cloud, or fixed, for example, a set of physical machines prepared for
+automated deployment and testing.
+
+Testing Infrastructure
+----------------------
+
+The testing infrastructure consists of a pool of devices physically located at
+a specific site. These devices are operated and maintained by partner companies
+and/or individuals. The device maintenance may involve resolving networking
+problems, swapping out a faulty storage medium, or configuring the device for
+initial provisioning to the pool.
+
+A site may operate as little as one device or as many as several dozen or
+hundred, depending on the test suites load and available
+resources. Sites can thus range from a single desk with a single device,
+a small rack with several devices in a corporate office, up to a dedicated
+testing lab with a large number of diverse devices.
+
+Each site is added to the central infrastructure by registering a
+software-specific service operating on-site and connecting it to the central
+system. There are two possible site configurations, depending on connectivity
+to the public cloud:
+
+* If the network connection is poor, building and downloading the images
+ locally is suggested.
+* If the network connection is robust, use the central build system directly to
+ build the images, where scalability is easy.
+
+Typically a micro-site that has limited throughput will be bound by the limited
+number of test devices and will be able to perform the builds locally much
+faster than being able to pull each new large image from the central system.
+Typically micro-sites will also see a more limited usage, for example, to
+support a bootstrap of a new project or preparing the process for automation
+for a new device.
+
+What does LAVA do?
+------------------
+
+1. The LAVA job definition template populates the required variables for
+ executing the test job using the values from the Gitlab CI.
+2. This job is submitted via REST API, and the LAVA executes the job.
+3. Using the callback system, LAVA triggers the last CI job in the loop, the
+ report job.
+4. The report job calls LAVA REST API to collect the result.
+5. The report is submitted back to Gitlab, and the developers can see the
+ report in their merge request.
+
+
+References
+----------
+
+* For more details on LAVA, see `Introduction to LAVA <https://docs.lavasoftware.org/lava/>`__.
+* For configuring and adding a new device, see `Adding your first devices <https://docs.lavasoftware.org/lava/first-devices.html>`__.
+* For more details on LAVA test job definition template, see `lava-test <https://allscenarios.readthedocs.io/en/latest/repos/manifest/ci/hidden-jobs/lava-test.html>`_.
+* For sample test job definition, see `sample jobs <https://git.lavasoftware.org/lava/lava/-/tree/master/tests/lava_scheduler_app/sample_jobs>`__.
+* For our CI test job definition, see `CI jobs <https://allscenarios.readthedocs.io/en/latest/repos/manifest/ci/hidden-jobs/lava-report.html>`__.
+* For LAVA supported device list, see `Supported Devices <https://git.lavasoftware.org/lava/lava/-/tree/master/etc/dispatcher-config/device-types>`__.
diff --git a/ci/index.rst b/ci/index.rst
index 8b20d76..8c06784 100644
--- a/ci/index.rst
+++ b/ci/index.rst
@@ -17,3 +17,4 @@ important elements are described below.
meta-ohos
xts-acts
docs
+ device-testing
--
GitLab