CI Setup

changed the description

assigned to @rbiegel

changed milestone to %Release v0.8

added Work In Progress label

created branch 2-ci-setup to address this issue

mentioned in merge request !11 (merged)

added To Do label

added 1 deleted label and removed To Do label

Hey Micha,

Relating to https://gitlab.eclipse.org/eclipse/simopenpass/simopenpass/-/tree/2-ci-setup-windows-conan, I just checked the recipes for Linux (Debian) compatibility.

Just fixed the OSI recipe - it now also works under Linux
The patch for the FMI library is located in /utils/ci/scripts/patches, but I needed to fix two tabs to get it running. I've just shared the file for you, so please check it in, as soon as you've got time.
Boost works out of the box
Qt works out of the box

Unfortunatly, openpass fails - there still seem to be some issues with OSI and Protobuf:

openpass/0.8: Calling build()
-- The C compiler identification is GNU 10.2.1
-- The CXX compiler identification is GNU 10.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Protobuf: /home/developer/.conan/data/openpass/0.8/_/_/build/d593edb122dbf365cc66925435f37bd910346620/ThirdParty/lib/libprotobuf.so;-lpthread (found version "3.11.4") 
CMake Error at /usr/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
  Could NOT find OSI (missing: OSI_LIBRARY)
Call Stack (most recent call first):
  /usr/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE)
  cmake/FindOSI.cmake:70 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  cmake/global.cmake:77 (find_package)
  CMakeLists.txt:24 (include)


-- Configuring incomplete, errors occurred!
See also "/home/developer/.conan/data/openpass/0.8/_/_/build/d593edb122dbf365cc66925435f37bd910346620/CMakeFiles/CMakeOutput.log".
See also "/home/developer/.conan/data/openpass/0.8/_/_/build/d593edb122dbf365cc66925435f37bd910346620/CMakeFiles/CMakeError.log".
openpass/0.8: 
openpass/0.8: ERROR: Package 'd593edb122dbf365cc66925435f37bd910346620' build failed
openpass/0.8: WARN: Build folder /home/developer/.conan/data/openpass/0.8/_/_/build/d593edb122dbf365cc66925435f37bd910346620
ERROR: openpass/0.8: Error in build() method, line 45
        cmake.configure(defs={"CMAKE_PREFIX_PATH": "./ThirdParty"})
        ConanException: Error 1 while executing cd '/home/developer/.conan/data/openpass/0.8/_/_/build/d593edb122dbf365cc66925435f37bd910346620' && cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE="Release" -DCONAN_IN_LOCAL_CACHE="ON" -DCONAN_COMPILER="gcc" -DCONAN_COMPILER_VERSION="10" -DCONAN_CXX_FLAGS="-m64" -DCONAN_SHARED_LINKER_FLAGS="-m64" -DCONAN_C_FLAGS="-m64" -DCONAN_LIBCXX="libstdc++11" -DBUILD_SHARED_LIBS="ON" -DCMAKE_INSTALL_PREFIX="/home/developer/.conan/data/openpass/0.8/_/_/package/d593edb122dbf365cc66925435f37bd910346620" -DCMAKE_INSTALL_BINDIR="bin" -DCMAKE_INSTALL_SBINDIR="bin" -DCMAKE_INSTALL_LIBEXECDIR="bin" -DCMAKE_INSTALL_LIBDIR="lib" -DCMAKE_INSTALL_INCLUDEDIR="include" -DCMAKE_INSTALL_OLDINCLUDEDIR="include" -DCMAKE_INSTALL_DATAROOTDIR="share" -DCONAN_CMAKE_POSITION_INDEPENDENT_CODE="ON" -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY="ON" -DCONAN_EXPORTED="1" -Wno-dev -DCMAKE_PREFIX_PATH="./ThirdParty" '/home/developer/.conan/data/openpass/0.8/_/_/build/d593edb122dbf365cc66925435f37bd910346620'

Conan is responsible for dependency management

TODO InTech:

Test on Linux
Create Ticket for Installing Conan ("pip install conan --ignore-installed six ")

TODO ITK:

Script to build Deps by Conan in CI utils folder. (Build_prepare.sh)
Replace 15_.sh and 16_.sh with 15_Conan.sh
Docu for Conan

As Docu update for Conan has now its own Issue #62 (closed), it won´t be part of this Issue

closed

reopened

@rbiegel

I committed the script changes for conan. https://gitlab.eclipse.org/eclipse/simopenpass/simopenpass/-/commit/0429b34e5c31b35a1a801491d5201fd0038873ba

Can you please have a look at it. Naming suggestions and shell script changes are appreciated.

Issue: Is the Conan install Ticket closed? Jenkins says that conan is not installed.

Regarding the issue of conan not being found

Windows:

**Edit: ** Actually I'm not sure, in which environment conan got installed, we'll have to check. If it is MSYS, then maybe a python -m conan like described below for Windows will work.

You will have to call conan via the "Windows native" Python installation, like in the links I mentioned here, using the environment variable PYTHON_WINDOWS_EXE:

I'd expect something like ${PYTHON_WINDOWS_EXE} -m conan ... to work, but I found this (old) issue: https://github.com/conan-io/conan/issues/962#issuecomment-278314462, where a user reported ... -m conans.conan ... as working.

Linux:

You have to add conan to the pip install command in utils/docker/Dockerfile.

General notices

15_prepare_thirdParty.sh needs exec permissions (755)
Also for 15_prepare_thirdParty.sh: You dropped the lines

MYDIR="$(dirname "$(readlink -f $0)")"
cd "$MYDIR/../../../.."

I think WORKSPACE_ROOT some lines below won't resolve to the directory expected.

Other than that it looks good to me.

I think Jenkins has a Problem with: https://gitlab.eclipse.org/eclipse/simopenpass/simopenpass/-/blob/2-ci-setup-windows/utils/ci/Jenkinsfile#L104

I think he doesn´t like the space in PYTHON_WINDOWS_EXE = 'C:/Program Files/Python39/python.exe'

if it's possible to switch to a backslash notation, we might be able to use the windows %programfiles% instead (see https://stackoverflow.com/a/45923797/7063154) - or we create (or ask eclipse to create) a junction to Python39

mklink /J C:\python "C:\Program Files\Python39"

leading to

PYTHON_WINDOWS_EXE = 'C:/python/python.exe'

This definitely has already been working previously. I'll look into that.

Not sure if this would completely solve the issue, but ${PYTHON_WINDOWS_EXE} has to be quoted here: https://gitlab.eclipse.org/eclipse/simopenpass/simopenpass/-/blob/2-ci-setup-windows/utils/ci/scripts/15_prepare_thirdParty.sh#L23

i.e.

"${PYTHON_WINDOWS_EXE}" -m conans.conan profile new default --detect

This would also affect Linux environments if a command is executed like here (via an environment variable containing spaces in paths).

... and a switch has to be added to allow execution of the script in our Linux docker container as well, e.g. by setting another variable PYTHON_COMMAND from PYTHON_WINDOWS_EXE or just use python, depending on the environment.

example for a switch: https://gitlab.eclipse.org/eclipse/simopenpass/simopenpass/-/blob/2-ci-setup-windows/utils/ci/scripts/20_configure.sh#L34

Hi you two,

just yesterday I played around with Conan a little. In order to make it work under Linux, I had to do some changes. Also, I noticed small things like the absence of Arena allocation. Could this be helpful for you? If so, I would push a merge request today.

Best, Robin

Hi Robin,

I saw some activity some hours ago on https://gitlab.eclipse.org/eclipse/simopenpass/simopenpass/-/merge_requests/40 Did you already report your findings in the follow up issue https://gitlab.eclipse.org/eclipse/simopenpass/simopenpass/-/issues/67?

If not, please report what you've got ;) (@rbiegel - do you concur?)

Hello René,

jep, changes I have done are included in https://gitlab.eclipse.org/eclipse/simopenpass/simopenpass/-/merge_requests/40 and open points are noticed down in https://gitlab.eclipse.org/eclipse/simopenpass/simopenpass/-/issues/67.

mentioned in merge request !43 (merged)

@rbiegel Several MRs have been adressing this issue: Draft:

Resolve "CI Setup" !11 (merged) E2E_test for PCM !22 (merged) fix(Conan): Fix build from source and creation of packages !40 (merged) Resolve "CI Setup" !43 (merged)

Can this issue be closed?

closed

CI Setup

Designs

Child items ...

Activity