... | ... | @@ -34,7 +34,6 @@ We clone the required repositories in `/ort`. |
|
|
|
|
|
As root:
|
|
|
```
|
|
|
mkdir /ort
|
|
|
chown root:users /ort
|
|
|
chmod 775 /ort
|
|
|
```
|
... | ... | @@ -51,6 +50,13 @@ Now set the ORT repository to the latest tested commit. As of now it is `2d29748 |
|
|
git checkout 2d29748d945827a82e62313380af0a6960c1204e
|
|
|
```
|
|
|
|
|
|
Edit ORT Dockerfile to change the user ID from 1000 (default) to 1001 (genie.infra):
|
|
|
```
|
|
|
ARG USERNAME=ort
|
|
|
ARG USER_ID=1001
|
|
|
ARG USER_GID=$USER_ID
|
|
|
```
|
|
|
|
|
|
Build the docker image (it will take some time):
|
|
|
```
|
|
|
cd /ort/ort
|
... | ... | @@ -67,203 +73,68 @@ These are the exact steps committed when installing ORT on the VM. |
|
|
|
|
|
As root:
|
|
|
```
|
|
|
apt-get install prometheus-node-exporter htop
|
|
|
apt-get install openjdk-17-jdk nodejs git python3-virtualenv python3-pip npm subversion cvs mercurial
|
|
|
npm install --global yarn
|
|
|
```
|
|
|
|
|
|
## Install scancode
|
|
|
|
|
|
As root:
|
|
|
```
|
|
|
cd /ort
|
|
|
wget https://github.com/nexB/scancode-toolkit/releases/download/v30.1.0/scancode-toolkit-30.1.0_py39-linux.tar.xz
|
|
|
tar xJf scancode-toolkit-30.1.0_py39-linux.tar.xz
|
|
|
chown -R ort:users scancode-toolkit-30.1.0
|
|
|
apt-get install htop openjdk-17-jdk nodejs git python3-virtualenv python3-pip npm
|
|
|
```
|
|
|
|
|
|
As user ort:
|
|
|
```
|
|
|
cd /ort/scancode-toolkit-30.1.0
|
|
|
./scancode --help
|
|
|
```
|
|
|
|
|
|
## Install Conan
|
|
|
|
|
|
As root:
|
|
|
```
|
|
|
pip install conan
|
|
|
```
|
|
|
|
|
|
## Install ORT
|
|
|
|
|
|
As root:
|
|
|
```
|
|
|
cd /ort
|
|
|
git clone --recurse-submodules https://github.com/oss-review-toolkit/ort.git
|
|
|
chown -R ort:users ort
|
|
|
```
|
|
|
Result of `docker run ort requirements`:
|
|
|
|
|
|
As user ort:
|
|
|
```
|
|
|
cd /ort/ort
|
|
|
export JAVA_HOME=/lib/jvm/java-17-openjdk-amd64/
|
|
|
./gradlew installDist
|
|
|
./gradlew cli:run --args="requirements"
|
|
|
```
|
|
|
[SNIP]
|
|
|
12:32:40,475 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@6f0cb5a1 - Registering current configuration as safe fallback point
|
|
|
|
|
|
Add binaries to PATH:
|
|
|
```
|
|
|
cd /usr/local/bin/
|
|
|
ln -s /ort/scancode-toolkit-30.1.0/scancode .
|
|
|
ln -s /ort/ort/cli/build/install/ort/bin/ort .
|
|
|
```
|
|
|
______________________________
|
|
|
/ \_______ \__ ___/ The OSS Review Toolkit, version 2d29748d94-dirty.
|
|
|
| | | | _/ | |
|
|
|
| | | | | \ | | Running 'requirements' as 'ort' under Java 17.0.5 on Linux
|
|
|
\________/ |____|___/ |____| with 8 CPUs and a maximum of 8032 MiB of memory.
|
|
|
|
|
|
Check that they can be found:
|
|
|
```
|
|
|
ort@ort-vm1:/usr/local/bin$ ort requirements
|
|
|
________ _____________________
|
|
|
\_____ \\______ \__ ___/ the OSS Review Toolkit, version a056b54c72.
|
|
|
/ | \| _/ | |
|
|
|
/ | \ | \ | | Running 'requirements' under Java 17.0.2 on Linux with
|
|
|
\_______ /____|_ / |____| 8 CPUs and a maximum of 4004 MiB of memory.
|
|
|
\/ \/
|
|
|
Environment variables:
|
|
|
ORT_CONFIG_DIR = /home/ort/.ort/config
|
|
|
ORT_DATA_DIR = /home/ort/.ort
|
|
|
SHELL = /bin/bash
|
|
|
TERM = screen.xterm-256color
|
|
|
JAVA_HOME = /lib/jvm/java-17-openjdk-amd64/
|
|
|
|
|
|
Scanners:
|
|
|
- Askalono: Requires 'askalono' in no specific version. Tool not found.
|
|
|
- BoyterLc: Requires 'lc' in no specific version. Tool not found.
|
|
|
- Licensee: Requires 'licensee' in no specific version. Tool not found.
|
|
|
* ScanCode: Requires 'scancode' in no specific version. Found version 30.1.0.
|
|
|
|
|
|
[SNIP]
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
## Prepare for run
|
|
|
|
|
|
Clone the Eclipse ORT git repository:
|
|
|
```
|
|
|
git clone https://gitlab.eclipse.org/eclipsefdn/emo-team/eclipsefdn-ort.git
|
|
|
chown -R ort:users eclipsefdn-ort/
|
|
|
```
|
|
|
|
|
|
Add following lines to ~ort/.bashrc
|
|
|
```
|
|
|
export EDITOR=emacs
|
|
|
export JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64
|
|
|
export ORT_CONFIG_DIR=/ort/eclipsefdn-ort/conf
|
|
|
```
|
|
|
|
|
|
Increase allocated Java Memory by editing `ORT_HOME/cli/build/install/ort/bin/ort` and set DEFAULT_JVM_OPTS:
|
|
|
```
|
|
|
DEFAULT_JVM_OPTS="-Xmx12192M"
|
|
|
```
|
|
|
|
|
|
|
|
|
## Execute first run
|
|
|
|
|
|
Clone a batch of git repositories:
|
|
|
```
|
|
|
cd /data/projects/
|
|
|
git clone https://git.eclipse.org/r/jgit/jgit
|
|
|
git clone https://github.com/eclipse-theia/theia
|
|
|
git clone https://github.com/eclipse/tycho
|
|
|
git clone https://github.com/eclipse/che.git
|
|
|
```
|
|
|
|
|
|
Execute first run to check everything is in place:
|
|
|
```
|
|
|
ort@ort-vm1:/data/projects$ time bash /ort/eclipsefdn-ort/scripts/run_ort.sh /data/projects/che/ /data/results/
|
|
|
Start scan at 2022-02-24 10:40
|
|
|
- Using ort from [ort].
|
|
|
- Log written to [/data/results/che_202202241040.log].
|
|
|
- Working on project [/data/projects/che/].
|
|
|
|
|
|
* Start analyzer at 2022-02-24 10:40
|
|
|
10:40:36.525 [main] INFO org.ossreviewtoolkit.model.config.OrtConfiguration - Using ORT configuration file '/ort/eclipsefdn-ort/conf/ort.conf'.
|
|
|
|
|
|
[SNIP]
|
|
|
|
|
|
```
|
|
|
|
|
|
## Additional package managers
|
|
|
|
|
|
Install the following system packages:
|
|
|
- [ ] cargo
|
|
|
- [ ] composer
|
|
|
- [ ] Go (package golang-go on debian).
|
|
|
- [ ] Gradle
|
|
|
- [ ] Ruby (ruby-rubygems on debian)
|
|
|
|
|
|
Install (manually) the following tools:
|
|
|
- [ ] Go dep from https://golang.github.io/dep/docs/installation.html
|
|
|
- [ ] bower via npm: `npm install -g bower`
|
|
|
- [ ] sbt from https://www.scala-sbt.org/download.html
|
|
|
- [ ] stack (Haskell) from https://docs.haskellstack.org/en/stable/README/
|
|
|
- [ ] cocoapods from https://guides.cocoapods.org/using/getting-started.html
|
|
|
|
|
|
JAVA_HOME = /opt/java/openjdk
|
|
|
ANDROID_HOME = /opt/android-sdk
|
|
|
|
|
|
Result of `ort requirements`:
|
|
|
Looking for ORT configuration in the following file:
|
|
|
/home/ort/.ort/config/config.yml (does not exist)
|
|
|
|
|
|
```
|
|
|
Scanners:
|
|
|
- Askalono: Requires 'askalono' in no specific version. Tool not found.
|
|
|
- BoyterLc: Requires 'lc' in no specific version. Tool not found.
|
|
|
- Licensee: Requires 'licensee' in no specific version. Tool not found.
|
|
|
* ScanCode: Requires 'scancode' in no specific version. Found version 30.1.0.
|
|
|
- Askalono: Requires 'askalono' in no specific version. Tool not found.
|
|
|
- BoyterLc: Requires 'lc' in no specific version. Tool not found.
|
|
|
- Licensee: Requires 'licensee' in no specific version. Tool not found.
|
|
|
* ScanCode: Requires 'scancode' in no specific version. Found version 31.2.1.
|
|
|
|
|
|
PackageManagers:
|
|
|
* Bower: Requires 'bower' in version >=1.8.8. Found version 1.8.14.
|
|
|
* Cargo: Requires 'cargo' in no specific version. Found version 1.46.0.
|
|
|
* CocoaPods: Requires 'pod' in version >=1.11.0. Found version 1.11.3.
|
|
|
* Composer: Requires 'composer' in version >=1.5.0. Found version 2.0.9.
|
|
|
* Conan: Requires 'conan' in version >=1.18.0. Found version 1.45.0.
|
|
|
* GoDep: Requires 'dep' in no specific version. Found version 0.5.4.
|
|
|
* GoMod: Requires 'go' in no specific version. Found version 1.15.15.
|
|
|
* Npm: Requires 'npm' in version >=5.7.0 <7.21.0. Found version 7.5.2.
|
|
|
* Pip: Requires 'pip' in no specific version. Found version 22.0.3.
|
|
|
- Pipenv: Requires 'pipenv' in version >=2018.10.9. Tool not found.
|
|
|
- Pub: Requires 'dart' in version >=2.10.0. Tool not found.
|
|
|
* Sbt: Requires 'sbt' in version >=0.13.0. Found version 1.6.2.
|
|
|
* Stack: Requires 'stack' in version >=2.1.1. Found version 2.7.5.
|
|
|
* Yarn: Requires 'yarn' in version >=1.3.0 <1.23.0. Found version 1.22.17.
|
|
|
|
|
|
VersionControlSystems:
|
|
|
* Cvs: Requires 'cvs' in no specific version. Found version 1.12.13.
|
|
|
* Git: Requires 'git' in version >=2.29.0. Found version 2.30.2.
|
|
|
- GitRepo: Requires 'repo' in no specific version. Tool not found.
|
|
|
* Mercurial: Requires 'hg' in no specific version. Found version 5.6.1.
|
|
|
* Bower: Requires 'bower' in version >=1.8.8. Found version 1.8.12.
|
|
|
* Cargo: Requires 'cargo' in no specific version. Found version 1.64.0.
|
|
|
* CocoaPods: Requires 'pod' in version >=1.11.0. Found version 1.11.2.
|
|
|
* Composer: Requires 'composer' in version >=1.5.0. Found version 2.2.18.
|
|
|
* Conan: Requires 'conan' in version >=1.18.0. Found version 1.53.0.
|
|
|
* GoDep: Requires 'dep' in no specific version. Found version 0.5.4.
|
|
|
* GoMod: Requires 'go' in no specific version. Found version 1.18.3.
|
|
|
* Npm: Requires 'npm' in version >=6.0.0 <9.0.0. Found version 8.15.1.
|
|
|
* Pipenv: Requires 'pipenv' in version >=2018.10.9. Found version 2022.9.24.
|
|
|
* Pnpm: Requires 'pnpm' in version >=5.0.0 <8.0.0. Found version 7.8.0.
|
|
|
* Poetry: Requires 'poetry' in no specific version. Found version Poetry (version 1.2.2).
|
|
|
* Pub: Requires 'dart' in version >=2.10.0. Found version 2.18.4.
|
|
|
* Sbt: Requires 'sbt' in version >=0.13.0. Found version 1.6.1.
|
|
|
* Stack: Requires 'stack' in version >=2.1.1. Found version 2.9.3.
|
|
|
* Yarn: Requires 'yarn' in version >=1.3.0 <1.23.0. Found version 1.22.17.
|
|
|
|
|
|
Other tools:
|
|
|
* PythonVersion: Requires 'python3' in no specific version. Found version 3.9.2.
|
|
|
* VirtualEnv: Requires 'virtualenv' in version >=15.1.0. Found version 20.4.0+ds.
|
|
|
|
|
|
Prefix legend:
|
|
|
- The tool was not found in the PATH environment.
|
|
|
+ The tool was found in the PATH environment, but not in the required version.
|
|
|
* The tool was found in the PATH environment in the required version.
|
|
|
* CvsCommand: Requires 'cvs' in no specific version. Found version 1.12.13.
|
|
|
* MercurialCommand: Requires 'hg' in no specific version. Found version 6.3.2.
|
|
|
* Python: Requires 'python3' in no specific version. Found version 3.10.8.
|
|
|
* PythonInspector: Requires 'python-inspector' in version >=0.9.2. Found version 0.9.2.
|
|
|
|
|
|
ScanCode license texts found in '/ort/scancode-toolkit-30.1.0/src/licensedcode/data/licenses'.
|
|
|
```
|
|
|
|
|
|
## Configuration
|
|
|
|
|
|
Set the cache for the various build tools to a common location in `/data/cache/`.
|
|
|
|
|
|
```
|
|
|
$ mkdir -p /data/cache/{ort,yarn,gradle}
|
|
|
```
|
|
|
VersionControlSystems:
|
|
|
* Git: Requires 'git' in version >=2.29.0. Found version 2.34.1.
|
|
|
* GitRepo: Requires 'repo' in no specific version. Found version 2.30 (launcher).
|
|
|
|
|
|
For most tools this is achieved with an environment variable, but for others (e.g. yarn) we need to execute the following command for the user running ort.
|
|
|
Prefix legend:
|
|
|
- The tool was not found in the PATH environment.
|
|
|
+ The tool was found in the PATH environment, but not in the required version.
|
|
|
* The tool was found in the PATH environment in the required version.
|
|
|
|
|
|
```
|
|
|
$ yarn config set cache-folder /data/cache/yarn
|
|
|
ScanCode license texts not found.
|
|
|
```
|
|
|
|