Additional Build-Node for Windows on ARM
Hello.
I'd like to help getting Eclipse and SWT ported to Windows on ARM by providing a build node for the project for a few months until a more permanent solution is available.
server description: Windows 11 22H2 22621, aarch64, 200 GB of available storage space, 10GB of memory
user name: runnerPlease add it to https://ci.eclipse.org/releng/job/eclipse.platform.swt/
I'll leave the name up do maintainers here. In GIMP, when I was helping that project in the past. They called it "Hernan's ARM runner"
Since this is not a beefy machine, I think we should let it do only one concurrent build for now.
See discussion at https://github.com/eclipse-platform/eclipse.platform.swt/issues/1019
Where can I download agent.jar?
Activity
What's the version of Java that the master node is running? I don't have access to https://ci.eclipse.org/releng/job/eclipse.platform.swt/ to be able to look at that information myself.
@hmartinez82 many thanks, the agent jar will be usually copied automatically by the Jenkins master if the connection is setup. For Java versions Java 17 will be needed.
If you want to do some preliminary tests, you can look here https://github.com/eclipse-platform/eclipse.platform.swt/blob/master/.github/workflows/maven.yml how a build is running currently on the additional Github nodes.
Edited by Christoph Laeubrich@claeubrich Sorry then I'm very lost here. I thought that this whole thing started by me downloading agent.jar and running it locally with the master node parameters. I have no knowledge of starting a Jenkins agent any other way
Do you have instructions how to connect to the master node? I already installed Java 17. That video with instructions also begins with downloading agent.jar from the master node.
Also, is there an IRC or Slack channel for SWT?
Edited by Hernan Martinez-
I created a user + added the master Jenkins SSH key and IT staff did the rest but as it was a linux machine the setup is probably different for windows, I'm sure you will soon receive some additional information about that from @fgurr .
added IT-prioritylow IT-severitylow service:jenkins statetodo team:releng labels
@hmartinez82, thank you for your willingness to support the SWT project. Your generosity and eagerness to contribute are greatly appreciated.
However, we must decline the offer to connect external machines to our production systems. As much as we value community contributions, the security of our projects and infrastructure is our top priority. Connecting an external machine, regardless of intent, poses significant security risks. This situation would not be very different from connecting a USB stick found in a coffee shop to a company laptop—a practice fraught with potential security vulnerabilities.
That said, your enthusiasm and readiness to help are truly appreciated. We encourage you to assist in making the necessary changes to build SWT on the Windows ARM architecture. Your contributions in this area would be beneficial to the project.
Meanwhile, @claeubrich, I suggest exploring the possibility of obtaining a Windows ARM agent by leveraging resources through an Eclipse Foundation member: https://github.com/eclipse-cbi/cbi/wiki#additional-resources.
Thank you all for your understanding.
No hard feelings. I understand.
https://www.eclipse.org/membership/showMember.php?member_id=1196 or https://www.eclipse.org/membership/showMember.php?member_id=823 should be able to get the project two or three Windows DevKits.
The GIMP project which is a much smaller project than Eclipse was able to get some :)
Edited by Hernan Martinez
-
@mbarbero I must confess I'm really confused here... adding a build node does not "connects" something to "the infrastructure", in fact it is the other way round, you give quite much control to the one you donate this as the Jenkins master is able to execute arbitrary commands on this machine while the reverse is not possible in any way. So @hmartinez82 should be concerned that EF is maybe burning is CPU for mining bitcoins...
Also please note that this was done before here:
Furthermore we are waiting literally for years but the EF is not capable of providing a single ARM node see:
- https://github.com/adoptium/adoptium-support/issues/616
- https://github.com/eclipse-platform/eclipse.platform.releng.aggregator/issues/597
On the other hand there is now a FORK(!) of the whole Eclipse platform driven by an individual to supply with (unofficial) Eclipse for Windows ARM:
so who should "donate" us such runner then even if we found any strategic member? Even worse as per your document
Those resources are assigned as a whole to a single project (i.e., we can’t split resources across multiple projects).
So it would mean we need a total of at least (if not more) 5 dedicated contributions from strategic members (SWT, Platform, Equinox, aggregator, adoptium)... this completely blocks adoption of Windows ARM platform for Eclipse IDE!
In any case if there are concerns, please then consider to take @hmartinez82 gentle offer and take the required actions to connect his machine as a Github self-hosted runner:
so we can at least compile the code so if we can't test and ship it at least we don't need to handle this by single persons and missing valuable discussions and improvements like this one:
The GIMP project which is a much smaller project than Eclipse was able to get some
So true I'm really stunned...
FYI @jograham the Eclipse IDE WG FYI @akurtakov as project lead of SWT project
Edited by Christoph Laeubrich @mbarbero I must confess I'm really confused here... adding a build node does not "connects" something to "the infrastructure", in fact it is the other way round, you give quite much control to the one you donate this as the Jenkins master is able to execute arbitrary commands on this machine while the reverse is not possible in any way. So @hmartinez82 should be concerned that EF is maybe burning is CPU for mining bitcoins...
Our concern, among others, is that the agent would have access to build secrets, thus granting it access to a lot of other systems.
Also please note that this was done before here:
Indeed, two differences:
- you are a committer on the project,
- that was almost 2 years ago. Since then, the security landscape has evolved. We are tightening the rules to better secure projects.
Furthermore we are waiting literally for years but the EF is not capable of providing a single ARM node see:
This comment is a bit disingenuous to EF, as those two tickets are not requests to EF to provide ARM nodes.
As described in the first ticket, Adoptium has access to ARM nodes with WG/PMC support, but it seems that the builds lack the polish needed for a GA release.
AFAICT, the second ticket has never been brought up as a request to EF before this very ticket.
On the other hand there is now a FORK(!) of the whole Eclipse platform driven by an individual to supply with (unofficial) Eclipse for Windows ARM:
Agreed, it would be better for this individual to contribute upstream, but this fact does not alleviate our legitimate concerns about connecting a machine from a random user on the internet (no offense, @hmartinez82) to a production system.
so who should "donate" us such runner then even if we found any strategic member?
A strategic member or the IDE WG should.
In any case if there are concerns, please then consider to take @hmartinez82 gentle offer and take the required actions to connect his machine as a Github self-hosted runner:
GitHub self-hosted runners bear exactly the same concerns as Jenkins agents. The security concerns are also described in the GH documentation
-
AFAICT, the second ticket has never been brought up as a request to EF before this very ticket.
Well because it was always claimed "we need a machine, can you offer one", now we have one offered and it reads "sorry we can't use it", so its bit of circularity :-)
Agreed, it would be better for this individual to contribute upstream, but this fact does not alleviate our legitimate concerns about connecting a machine from a random user on the internet (no offense, @hmartinez82) to a production system.
We are open to alternatives, so can the Eclipse IDE WG "buy" such a machine located at Eclipse somehow?
GitHub self-hosted runners bear exactly the same concerns as Jenkins agents. The security concerns are also described in the GH documentation
As far as I know we do not really use any secrets on Github but even if it seems more like something that needs to be solved (e.g. we can only do it if (1), (2), ... is done)
What really needs to be offered? I think its understandable that individual committers/contributors can't afford a full eclipse member ship (about €100k ... €500k a year) just for a few ARM Maschines, I suspect one can get a full farm of computers for this money if even it is not clear if we can get one at all (see #4289 (closed)).
The linked issues also mentioning some funding program from microsoft or even getting some on azure seems possible: https://azure.microsoft.com/en-us/updates/general-availability-armbased-vms-now-available-in-four-additional-azure-regions/
If I may. My intent was to offer my machine to run the ARM64 just for a few months until the project gets a more permanent solution. This is how I did with GIMP. They only used my runner for 2 months.
The idea was to use my runner to test the build process and the build output while resolving the many quirks that will for sure appear and never use my runner for production builds.
By the time the project gets DevKit or VMs from Microsoft or ARM then it will get ready to hit the floor with proper production ready builds.
Edited by Hernan MartinezI will ask the IDE WG Steering Committee to fund the needed runners - @mbarbero I will cc you on that because I think the first question will be what the cost is.
The IDE WG has funded a similar item in the past (paying for Git LFS support for Eclipse SWT #3076 (closed))
Before I go to the steering committee I just need to run it by the planning council. This link should have my message once it is archived in a few minutes.
It really should be @droy in the cc for anything cost related ;)
Bosch is willing to sponsor the build-node with one of our free CBI resource.
Which kind of resource is needed for this task?
- Resource Pack
- Dedicated Agent
- GitHub hosted large runner
I was wondering if there is a timeline for getting this dedicated agent spun up and added to https://ci.eclipse.org/releng/? The PR that adds the functionality to the code is progressing well.
@fgurr when you set up the new native agent can you please also add the corresponding label for the SWT build pipeline:
swt.natives-win32.win32.aarch64Just like for it is already done for the other agents.
Edited by Hannes Wellmannassigned to @fgurr
A new Windows 11 ARM64 build node with 2vCPUs and 8GB RAM has been added to the Releng Jenkins instance:
https://ci.eclipse.org/releng/computer/rie8t-win11-arm64/
So far, only JDK21 and Git have been installed. Please test the build agent and report back if other dependencies are missing.
Edited by Frederic GurrThank you @fgurr the agent itself looks good and can be used by the SWT build pipeline.
So far, only JDK21 and Git have been installed. Please test the build agent and report back if other dependencies are missing.
We definitely need the Microsoft Visual Studio C/C++ Compiler and also the Windows 10 SDK (I think otherwise the headers like windows.h are missing) as it is listed here: https://www.eclipse.org/swt/swt_win_native.php
I'm not certain, but I guess that a more recent version of MSVC should also work. However I cannot tell if the Windows 10 SDK can be installed/used on a Win11-machine (which the executor is running on, I assume from its name) or if the binaries produced with that Win11-machine and Win-11 SDK can run on Win-10 and I cannot test if that is backwards compatible.
For local builds of SWT I use
Visual Studio 2022 Developer Command Prompt v17.8.3and the build works fine. I didn't encounter problems so far but also didn't test them extensively. If we use a more recent version of MSVC for WoA we should probably also update the win33.x86_64 node to use the same MSVC version.
added statereview label and removed statewip label
-
@hmackamul Sponsorship API has been updated: https://api.eclipse.org/cbi/sponsorships
mentioned in issue #4444 (closed)
removed statereview label
mentioned in issue #5075 (closed)
