CBI: Sponsor resource packs of DLR to have macOS agent for Eclipse SUMO

Summary

As the representative of the German Aerospace Center (DLR), I would like to sponsor 2 additional resource packs to the ci instance of the Eclipse SUMO project, because SUMO needs a macOS runner for signing and notarizing the releases. In the near future, we will also need a Windows runner to build and sign the binaries as well.

Currently, two of the three CBI resource packs of the DLR are assigned to the JustJ project (@emerks). Please remove these resource packs from Eclipse JustJ and reassign those to SUMO to create the macOS runner (at the windows runner later).

Priority

• Urgent
• High
• Medium
• Low

Severity

• Blocker
• Major
• Normal
• Low

added IT-prioritymedium IT-severitynormal service:jenkins team:releng labels

SUMO needs a macOS runner for signing and notarizing the releases. In the near future, we will also need a Windows runner to build and sign the binaries as well.

Just to set expectations: we won't set up signing on individual build agents. Signing will happen through our existing signing infrastructure for macOS and Windows.

I understand. We will use the existing infrastructure for signing and notarization. The runner will be used for creating the unsigned binaries etc.

assigned to @fgurr

added statewip label

I've started to set up access to the macOS agent, but macOS is giving me a hard time. I will continue this after the Christmas break in early January.

I was able to fix it, so the macOS agent w8qlf-macos11-arm64 is connected to the Sumo Jenkins instance now. You can start to run builds on it.

In case something is still missing, we can fix this in January.

@emerks please start looking for a replacement sponsoring for the macOS and Windows machines on the JustJ Jenkins instance.

I will formerly reassign the resource packs in January.

Thank you for providing the macOS agent. I can confirm, that we are able to use the agent (instead of going to the k8 cluster as before).

We noticed, that the agent seems to be running macOS 11 (20.6.0 Darwin Kernel) - which is no longer supported officially by Apple - or by Homebrew. Do you see any chance to use a more recent version of macOS (e.g. GitHub offers macOS 13 and macOS 14 and macOS 15 as beta).

Also, is there a list of already installed software on that agent? Do we need to install homebrew or Apple CommandLine Tools from scratch?

We will set up a new agent based on a more recent macOS version.

Which tools/libraries/dependencies do you need on the agent?

Thank you - that would be great and is really appreciated.

With regards to the tools and libraries we need, well, we would like to replicate the macOS build on GitHub (https://github.com/eclipse-sumo/sumo/actions/workflows/build-macos.yml).

Essentially, we need:

• Apple CommandLineTools (clang compiler, git, ...)
• Python 3.9 (or newer)
• Homebrew (we will update and install dependencies during the build, like: xerces-c, proj, gdal, fox, gl2ps, ccache, googletest, fmt, swig, eigen, xquartz, java)
• cmake
• curl

Please ensure, that we can do "brew install ..." or (password-less) "sudo brew install ...".

Please ensure, that we can do "brew install ..." or (password-less) "sudo brew install ...".

Sorry, we can't provide that capability for security reasons. While root-less brew install is technically possible, it's unsupported and can easily break (see also https://superuser.com/questions/619498/can-i-install-homebrew-without-sudo-privileges).

Do I understand correctly, that we cannot have a system-wide installation of homebrew for which we can do "brew upgrade" and "brew install ..." as part of our build process in order to have the most recent versions of the dependencies we need? The link seems to suggest, that a root-less install requires a recompilation of all formulas - which is not a viable option for us.

What other options do we have in that case? Of course, in preparing the agent, you could pre-install all dependencies we need. But as the versions of the dependencies in homebrew change quite often, could you automatically update the system-wide homebrew installation - maybe once every week?

Do I understand correctly, that we cannot have a system-wide installation of homebrew for which we can do "brew upgrade" and "brew install ..." as part of our build process in order to have the most recent versions of the dependencies we need?

Yes.

Of course, in preparing the agent, you could pre-install all dependencies we need. But as the versions of the dependencies in homebrew change quite often, could you automatically update the system-wide homebrew installation - maybe once every week?

We did not have this specific requirement before, but this sounds feasible.

Sounds like a plan, what do you need from our side, except this list of dependencies:

• Apple CommandLineTools (clang compiler, git, ...)
• Homebrew (weekly updated) with:
  • python 3.11
  • xerces-c
  • proj
  • gdal
  • fox
  • gl2ps
  • ccache
  • googletest
  • fmt
  • swig
  • eigen
  • xquartz
  • cmake
  • curl
  • openjdk
  • maven

(and their transitive dependencies)

The list of dependencies is a good start. After we've set up the machine, the tools and the dependencies, we can iterate if something is still missing.

@pstankie please set up a macOS agent based on macOS14 and the lists above.

assigned to @pstankie

unassigned @fgurr

Dear @pstankie, could you give us a rough estimate on a timeframe for the availability of the new agent? We are planning to have a new SUMO release in January and would be very happy to have the agent running in order to deliver a proper signed installer ... Thank you!

@rhilbrich The machine is currently being provisioned and configured. It should be ready within max 2 days.

The machine has been provisioned and set at https://ci.eclipse.org/sumo/computer/rg34F%2Dmacos14%2Darm64/ Please verify it works ok.

@pstankie: Thank you for the setup. I was able to build SUMO successfully on the new macOS-14 agent, so that it looks fine for the moment. I will close the issue for now and re-open, if we stumble upon missing packages.