Sign Electron Application on MacOs

changed title from Sign Electron Application on Windows and MacOs to Sign Electron Application on MacOs

changed the description

added IT-prioritylow IT-severitylow service:signing-mac statetodo team:releng labels

Hi @heurtemattes , thanks for updating this issue. Could you look into the error and let us know how we can certify Mac Electron applications? As is explained to me, electron application contain multiple binaries, see and features a possiblity to sign MacOS applications, see https://www.electronjs.org/docs/latest/tutorial/code-signing .

With credentials on our side we would be able to complete the task, but with the current setup we do not know. Any support welcome, of course we are happy to discuss with you any questions.

Hopefully we can find a solution to complete https://github.com/eclipse-esmf/esmf-aspect-model-editor/issues/141 which will give our growing number of MacOS users a much better experience.

thanks!

@netomi any idea about this?

I will take a look how we can integrate our notarization service into the electron build.

When you wanna use the Eclipse Infrastructure to sign or notarize a Mac application you will have to declare a job on the Jenkins Build Server that has been provided to you. On this build server you will be able to connect to a service that actually performs the signing / notarization.

When looking at the electron documentation, performing signing during the build requires that certificate / key information must be present which is not possible with our existing signing / notarization service.

Your best option will be to build the electron application and afterwards use the service the sign / notarize the application.

We can help you to set this up ofc.

Hi @netomi,

We have successfully received a Jenkins instance from you and are in the process of developing an Electron application compatible with Debian, Mac, and Windows. This application uniquely incorporates a GraalVM binary internally.

We have achieved success in signing the build for our Windows version of the Electron application, confirming its functional integrity.

However, we have encountered a specific challenge with the Mac version, as previously mentioned. Based on your feedback, it seems we need to transition the entire Mac application development process to Jenkins, incorporating the signing operation during the build phase?

I must admit, the exact steps required for this transition are not entirely clear to me.

To provide some background: our initial approach involved creating/building the Electron application on GitHub, transferring the “.app” application into Jenkins for signing, which led to the error message we have encountered.

We would deeply appreciate any clarification or detailed guidance on the necessary steps to resolve this issue effectively.

So it was not clear to my how you got to the output in the description. From the error message it looks like the app that you tried to notarize was not properly signed,

see https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution/resolving_common_notarization_issues

Documentation about how to use the EF signing / notarization service from a jenkins build server can be found here:

The signing documentation is using the maven plugin that connects to our service, but you can use the service also directly I supposed, need to find details about it though.

Let me know if this is sufficient, or you need more help.

@netomi, I have tried several approaches to set this up. The latest attempt resulted in the following error message: We have created a .zip file that contains the contents of the Electron application, enabling us to proceed with these steps:

Creating the .zip file in GitHub Actions.
Notifying Jenkins that there is something to pull.
Retrieving the .zip file from the assets.

Jenkinsfile:

sh 'curl -L https://github.com/eclipse-esmf/esmf-aspect-model-editor/releases/download/v5.1.1/aspect-model-editor-v5.1.1-mac.zip -o aspect-model-editor-v5.1.1-mac.zip'

Sign .zip Application

Jenkinsfile:

sh 'curl -o signed-mac.zip -F file=@aspect-model-editor-v5.1.1-mac.zip -F entitlements=@entitlements.plist https://cbi.eclipse.org/macos/codesign/sign'

Create .dmg file

Jenkinsfile:

sh "mvn clean package -P mac-convert-to-dmg"

Pom.xml:

    <profile>
      <id>mac-convert-to-dmg</id>
      <build>
        <plugins>
          <plugin>
            <groupId>org.eclipse.cbi.maven.plugins</groupId>
            <artifactId>eclipse-dmg-packager</artifactId>
            <executions>
              <execution>
                <goals>
                  <goal>package-dmg</goal>
                </goals>
                <phase>package</phase>
                <configuration>
                  <source>${project.basedir}/signed-mac.zip</source>
                  <target>${project.basedir}/aspect-model-editor-v5.1.1-mac.dmg</target>
                  <continueOnFail>true</continueOnFail>
                  <timeoutMillis>600000</timeoutMillis>
                  <continueOnFail>${macSigner.forceContinue}</continueOnFail>
                  <sign>true</sign>
                </configuration>
              </execution>
            </executions>
          </plugin>
        </plugins>
      </build>
    </profile>

implementation of MacOS Notarization

Jenkinsfile:

def dmg = "aspect-model-editor-v5.1.1-mac.dmg"

def jsonOptions = "options={\"primaryBundleId\": \"org.eclipse.esmf\", \"staple\": true};type=application/json"

def response = sh(script: "curl -s -X POST -F file=@${dmg} -F '${jsonOptions}' https://cbi.eclipse.org/macos/xcrun/notarize", returnStdout: true).trim()

def jsonSlurper = new groovy.json.JsonSlurper()
def object = jsonSlurper.parseText(response)
def uuid = object.uuid
def status = object.notarizationStatus.status

while (status == 'IN_PROGRESS') {
            sleep(time: 1, unit: 'MINUTES')
            response = sh(script: "curl -s https://cbi.eclipse.org/macos/xcrun/${uuid}/status", returnStdout: true).trim()
            def inProgressResponse = jsonSlurper.parseText(response)
            uuid = inProgressResponse.uuid
            status = inProgressResponse.notarizationStatus.status
}

if (status != 'COMPLETE') {
            echo "Notarization failed: ${response}"
            error("Notarization failed.")
}

sh "rm '${dmg}'"

sh "curl -JO https://cbi.eclipse.org/macos/xcrun/${uuid}/download"

And this is exactly where we encounter the error message during the notarization process. Do you see any mistake we might have made, or do you have a suggestion on how to resolve this issue?

your steps look good.

The error that you still receive is because the built application contains binary files that are not signed.

e.g.

aspect-model-editor-v5.1.1-mac-14394387365538383039.dmg/Aspect-Model-Editor.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libEGL.dylib

You also seem to be using a Squirrel Framework that not not signed either, maybe the information there helps:

https://github.com/Squirrel/Squirrel.Mac/issues/234

I am not sure if ad-hoc signing will work when notarizing the app, but it might be worth a try. Otherwise, you would have to extract the frameworks in the app archive and sign them as well using the signing service.

Edit: actually the signing service should unpack zip files and sign binaries, not sure why it does not do that in your case, I will try it out myself.

I found this tool to sign electron apps: https://github.com/electron/osx-sign

it only works if you have the identity locally available, which will not be possible in our case, but it gives you an idea what you have to do: extract all binaries and sign them separately.

I guess the osx-sign tool could be easily modified to call our webservice instead of running codesign locally.

In the long run, our code sign should support to sign all binaries inside a dmg file. We have other projects that need to extract all binaries from an archive and sign them separately.

Here is also the link to a Jenkinsfile of the theia project that signs and notarizes their installer using the EF services: https://github.com/eclipse-theia/theia-blueprint/blob/master/Jenkinsfile

They also build an electron app, maybe thats helpful as well.

@michelu89 can you provide me with a zip version of the application? Looking at the release assets I can only find dmg files.

Doing the signing on the dmg file results in the same error as in your case. The signing service should be able to extract zip files, but I would like to try it out myself. If you can provide me with a link to a zip file containing the app inside it would be really helpful.

Thank you for providing the information @netomi . I will thoroughly analyze it and get back to you with the results.

In the meantime, here is the link to a fully functional Electron MacOS application, which has been compressed into a ".zip" file: https://github.com/eclipse-esmf/esmf-aspect-model-editor/releases/download/v5.3.0-rc2/aspect-model-editor-v5.3.0-rc2-mac.zip

Looking at the theia build I can see that they dont use electron-builder (though its an electron app) and the sign the dmg file using our service and it works. Maybe the electron-builder creates something that can not be signed correctly by our service?

assigned to @netomi

@netomi

I would like to meticulously review and sign this MACH-O binary file "https://github.com/eclipse-esmf/esmf-aspect-model-editor-backend/actions/runs/9063962578/artifacts/1497311136". Everything functions correctly until I attempt to notarize it, at which point I encounter issues.

https://ci.eclipse.org/esmf/job/SIGN_AME_BACKEND_WINDOWS_MACOS/57/console

In the log file I see the following error:

Error: ame-backend-DEV-SNAPSHOT-mac-14805323734713405692. must be a zip archive (.zip), flat installer package (.pkg), or UDIF disk image (.dmg)

from the pasted screenshot I see that you make a curl request passing in a file without extension. Afaict the filename as passed in the request is used to store the received file locally and pass it on to the notarytool which seems to have a simple detection mechanism based on the extension.

Could you try to change the filename and retest? This is certainly an area that could be improved in our service to return a more meaningful error message.

from the latest build it looks like the notarization was succesful: https://ci.eclipse.org/esmf/job/SIGN_AME_BACKEND_WINDOWS_MACOS/60/

Hey @netomi, yes it worked out for the Mach-O. Next I would certify the Electron app with the binary and see if that works too.

ty for your effort in this area, your feedback is also welcome how we can improve the documentation to make it more easy to use the services and add some better error handling.

Hey @netomi,

We've encountered an issue with the notarization of our Electron ".dmg" file. Could you please review the process and determine the exact problem?

https://ci.eclipse.org/esmf/job/SIGN_AME_FRONTEND_WINDOWS_MACOS/11/console

It appears that the file that you sent for notarization is a zip file containing the dmg file:

macmini-cbi2:pending-files outage4$ file -I aspect-model-editor-v5.6.0-rc2-mac-3321943818027938356.dmg 
aspect-model-editor-v5.6.0-rc2-mac-3321943818027938356.dmg: application/zip; charset=binary

macmini-cbi2:pending-files outage4$ unzip -l aspect-model-editor-v5.6.0-rc2-mac-3321943818027938356.dmg 
Archive:  aspect-model-editor-v5.6.0-rc2-mac-3321943818027938356.dmg
  Length      Date    Time    Name
---------  ---------- -----   ----
177497308  05-27-2024 09:17   aspect-model-editor-v5.6.0-rc2-mac.dmg

if you send the actual dmg file it should work imho.

I see in your workflow you download the artifact from GitHub, but they are always zipped afaict, so unzip them after download and then send the unzipped file for notarization.

@netomi

I have tried various methods to notarize Electron applications for macOS, including ".app", ".zip", and ".dmg" formats. Unfortunately, I keep encountering the same issue: https://ci.eclipse.org/esmf/job/SIGN_AME_FRONTEND_WINDOWS_MACOS/19/console.

The binaries generated by Electron are not being signed, and subsequently, they cannot be notarized.

In the Theia Blueprint project on GitHub: https://github.dev/eclipse-theia/theia-blueprint, I noticed that after building the Electron application, an "after-pack.js" script is executed, which might be signing and notarizing the Electron binaries (though I'm not yet certain).

To implement a similar process, I would need to build Electron for Mac on Jenkins and replicate all these steps. This means that we would require a Mac agent on our Jenkins instance to proceed, as otherwise, I cannot build Electron for Mac.

Here the "after-pack.js" from Theia: https://github.com/eclipse-theia/theia-blueprint/blob/master/applications/electron/scripts/after-pack.js

If this works, I would need the same setup as the storage configuration found in line 16 of this file: https://github.com/eclipse-theia/theia-blueprint/blob/master/applications/electron/scripts/sign.sh and also "ssh" and "scp" permissions.

If you look at the error message from the last job you can see that the notarization process complains about adjacent files in the Frameworks folder that are not signed. I guess that is the reason why Theia signs all relevant files inside the dmg file as you can see from the log here:

https://ci.eclipse.org/theia/job/Theia2/job/master/151/consoleFull

So their workaround seems to be to integrate into the electron builder process to have a hook called after package that will sign all necessary files inside the app using the codesign service.

You should have access from a jenkins build to the various webservices for signing and notarization, see also https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/IT_Infrastructure_Doc#web-service as you already do in your current job. I am not sure why they use this mechanism in theia-blueprint with ssh and the genie bot, but normally that is not needed afaict. We also have various monitoring jobs for our services that do it in the same way as you do in your existing workflow.

In the last time im using this it was not working: https://github.com/eclipse-theia/theia-blueprint/blob/bfa92d03e9bf6f572e4410066e856883df5b011f/Jenkinsfile#L120

I will try it again, now.

Given that I need to reconstruct the entire macOS mechanism, I will require a macOS agent to build the Electron application.

ok we need to check if your project has such an agent available, @heurtemattes can you check please?

an option would be also to extract the contents of the dmg file, sign files individually and build the dmg again. Something like this is done by adoptium for the java runtime for macos. That would avoid having to build the app on a macos jenkins agent and you could reuse the dmg file build on GitHub.

That would also be a possibility. I will consider the better and simpler option later, thanks for your input.

I know its a painful process, I will open a ticket such that the signing service supports signing the content of dmg files as well to avoid having to do this workaround yourself.

https://github.com/eclipse-cbi/org.eclipse.cbi/issues/461

@netomi Im ready to test the procedure and have set up the .app on GitHub. I plan to run it through the Jenkins pipeline, but I need the nodeJs plugin. Is it already activated on our system, or should it be installed? If possible, may I install it myself?

I could also build a DMG file on linux that shouldn't be a problem, but a Mac agent would of course make more sense to make sure that everything works.

@michelu89 nodejs plugin has been added to the jenkins instance.

@heurtemattes

Thank you for adding the 'NodeJs' plugin. Could you please direct me to any documentation on how to use it within the Eclipse Foundation?

I tried using this example from https://github.com/eclipse-theia/theia-blueprint/blob/bfa92d03e9bf6f572e4410066e856883df5b011f/Jenkinsfile#L140

nodejs(nodeJSInstallationName: 'node_20.x') {
   sh "node --version"
   sh "npm --version"
}

but it doesnt seem to work, and I couldnt find relevant information on your documentation page: Eclipse Jenkins Wiki.

Could you also confirm which version of the plugin I should use, and whether I can configure it exactly as specified?

Additionally, we inquired earlier about activating the MacOS agents. Are we able to enable them to address our specific needs?

@michelu89 I forgot to specify the nodejs configuration in jenkins. I have triggered a new build: https://ci.eclipse.org/esmf/job/SIGN_AME_FRONTEND_WINDOWS_MACOS/23

@heurtemattes

Thanks now it has worked, can you tell me approximately when the MacOs agent like https://github.com/eclipse-theia/theia-blueprint/blob/bfa92d03e9bf6f572e4410066e856883df5b011f/Jenkinsfile#L121 might be available so I can test the next steps?

Please note that macOS agents are not part of the resources that are provided to Eclipse projects by default. It would require an additional resource pack that needs to be sponsored by an eligible member organization. See also https://github.com/eclipse-cbi/cbi/wiki#whats-provided.

@fgurr Thanks for the information, I will pass it along and see what happens.

@netomi I have now rebuilt everything on a Linux basis, and inspired by https://github.com/eclipse-theia/theia-blueprint, I have modified the scripts so that each individual element of the Electron application is signed.

Unfortunately, the entire process is still considered unsigned by notarize and is aborted. Can you spot any errors in the service that could help me?

Here is the console output: https://ci.eclipse.org/esmf/job/SIGN_AME_FRONTEND_WINDOWS_MACOS/37/console. I even manually deleted and replaced the file after the Curl command to ensure that the overwrite function is working correctly, but unfortunately, it makes no difference.

I am not sure what else could be done here!

Looking at the log file and your after-pack.js it looks to me that you are also signing symlinks, which are then treated as binaries whose signature is wrong.

The original code from theia only walks binary files that need to be signed, you walk now all files and sign them afaict.

So ignoring symlinks should makes it work imho, all the remaining errors are related to such files afaict. For performance reasons I would also omit signing all files, as this takes a while and is not necessary afaict. Examples are images.

@michelu89 looks like only signing binary files resolved the notarization problems. Great work.

If you need further help with publishing let us know.

Hey @netomi,

Yes, that worked thanks for that. But if I now pack the whole thing into a ".tar.gz" and then into a ".dmg" and sign and notarise it again, I get a ‘Not signed’ error again.

Can you look into what exactly the problem is?

https://ci.eclipse.org/esmf/job/SIGN_AME_FRONTEND_WINDOWS_MACOS/58/console

from the log it looks like that after your notarized the zip file, you unzip the result and the try to build the dmg file from the unpack_mac_dir which is the place where you started before signing all binary files, so you end up generating a dmg file with all unsigned files and trying to notarize the result again.

My understanding of the process would be:

download app from GitHub
extract contents
sign all binary files using the signing service
pack all files together to a dmg file
sign the dmg file
notarize the signed dmg file

@netomi

I have now restructured everything to make it even clearer.

I download the "notarized.zip" file from "https://cbi.eclipse.org/macos/xcrun/05eb9171-de8d-4bff-ac07-3f86c27d2c4e/download".
I package this file into a "DMG" using the Maven plugin "eclipse-dmg-packager".
I then sign the "DMG" and attempt to notarize it again, but it fails because the contents are apparently not singed.

Could it be that the ‘notarised.zip’ from the service does not contain the ‘--xattrs’ attribute?

I do not follow why you sign the files, then create a zip archive, notarize that, unzip that again and then create a dmg that you sign and notarize again.

If should be signing the contents, then create the dmg file that you want to notarize with the signed content, then notarize it afaict but I am not an Apple expert.

Thats basically also what others do so I would be surprised if other steps are required. Maybe something in this whole process gets messed up and you copy over the wrong file while trying to notarize a second time which is certainly not necessary. The input to the first notarization process is correct, use that one to build the final package and notarize it.

To be honest, I followed the procedure outlined by Theia, assuming you had already developed an effective method.

In reality, all I need are the certificates and a Mac environment. According to the Electron documentation, this would allow me to sign and notarize everything quickly and efficiently.

What I can test is not to pack the whole thing into a ‘.dmg’ and test whether the ‘.app’ itself works.

The only difference I see is that Theia uses the "afterPack" functionality to sign and notarize everything after the Electron build, and then directly package it into a ".dmg". Unfortunately, we cannot do this because we would need a Mac instance for that.

Hey @netomi,

The following insights pertain to Mac signing and notarization:

When I use the .app, it is signed and notarized. However, when I use the Eclipse DMG Packager (https://www.eclipse.org/cbi/sitedocs/eclipse-dmg-packager/plugin-info.html) to sign and notarize, the same issues described at the beginning occur: the contents within the .dmg are not signed.

I tested an unsigned and non-notarized .dmg to understand its structure. After downloading, I could unpack the .app but not launch it, receiving an error stating that the .app is broken.

It seems there may be an issue with the Eclipse DMG Packager.

We have now decided to release it initially as a .app and may revisit the process in the near future.

ok ty for your feedback, we will also look into the dmg packager to see what is going wrong there. Glad that you have an app that is notarized and can be used though.

I'm going to close this task now, we'll have to sort out the ‘.dmg’ issue some other time.

Thanks for your help.

Created https://github.com/eclipse-cbi/org.eclipse.cbi/issues/473 will take a look hopefully next week.

btw. latest plugin documentation is now published to https://eclipse-cbi.github.io/org.eclipse.cbi/

Where did you find the other link: https://www.eclipse.org/cbi/sitedocs/eclipse-dmg-packager/plugin-info.html which is about an outdated version?

This was the first page Google showed me, but I used the latest version in the pipeline ->

ok ty, it has not been republished anymore to the original place. Will check how we can republish the latest released version though.

closed

reopened

@michelu89 the MacOS signing service has now been overhauled to use a different tool to do the actual signing.

A test version is deployed at https://cbi-staging.eclipse.org/macos/codesign/sign (the production URL is https://cbi.eclipse.org/macos/codesign/sign so you just need to change the domain from cbi.eclipse.org to cbi-staging.eclipse.org).

Could you give it a go with your application to verify that the result is the same and we can switch this to production?

Also this version will now do deep signing by default, thus you do not have to traverse binary files yourself and sign them separately, making the whole process simpler. Would it be possible for you to test that with your application as well? That would help me tremendously.

HI @netomi,

Thanks for the info, I could test the whole thing next week and let you know.

Do you maybe have some time to take a look?

In the meantime another project could validate the service, see #5417 and we would to check for any possible regressions before we can switch that service to production.

Also that would allow to simplify your signing process afaict as it supports now deep signing.

added Awaiting Feedback label

Hi @netomi , apologies for the delay. @michelu89 is not available for the rest of the year and I am reminding him to have a look as soon as possible.

ty for the feedback, its, I will try to test it myself using local testing, and it would be great if @michelu89 could test as well beginning of next year.

Hi @netomi, i will test the complete setup at the beginning of next year and would let you know.

Sign Electron Application on MacOs

Summary

What is the current bug behavior?

What is the expected correct behavior?

Relevant logs and/or screenshots

Priority

Severity

Impact

Designs

Child items ...

Activity