Skip to content
Snippets Groups Projects

IT_Infrastructure_Doc

Last edited by Frederic Gurr

IT Infrastructure Doc

Website

How do I set up my project website?

Project websites are stored in a git repository separate from the actual project code. Once a website repository has been created for your project, files you commit to the website repository are automatically published to eclipse.dev/xyz, where xyz is your project's short name. You are free to use any tools (eg: Hugo, Jekyl) to author and build your website, but the hosting service only supports plain HTML files (historically PHP was supported, but that is being deprecated here: #1934 (closed)).

The publishing process does not preform any build steps or similar operations, it simply pulls content from the designated repository.

Hosting a project website is normally done when the project proposal has been approved. If you suspect your files are not being checked out to the eclipse.dev website, simply commit a small change to one file. This is usually enough to trigger a website refresh.

Do you have an example site?

Yes, you can find a boilerplate website here: https://gitlab.eclipse.org/eclipsefdn/it/webdev/hugo-eclipsefdn-website-boilerplate#how-to-build-my-projects-website-with-jenkins .

It also covers how to actually build your site via a CI job hosted at the Eclipse Foundation.

How do I use the Solstice theme?

Please refer to this document for information on using Solstice.

Use a database for my website?

We currently do not offer database support.

I need to put a large file on my website. How should I do this?

Large (1 MB+) ZIP and JAR files must be put in the downloads area, using the Find A Mirror script to link to them. However, small files (less than 1 MB) can be put on the eclipse.dev/yourproject website directly without causing too much harm.

Remember to allow our mirrors at least 24 hours to sync up before using a transparent mirror redirect.

Use PHP on my website?

PHP support has been deprecated, and will be removed in the near future.

SSH

Shells

Downloads

Upload files to the download server?

Downloadable files must be placed in the downloads area (~/downloads, or /home/data/httpd/download.eclipse.org) so they can be mirrored to our mirror sites worldwide. Please ensure only pertinent, current files are in the downloads area, as we cannot store an eternity of nightly, integration and stable builds. Production releases can be kept forever; however, we ask that you move archived releases to archive.eclipse.org (see below).

To upload your files:

  • Use Jenkins to upload your files, see How do I deploy artifacts to download.eclipse.org?. (Formerly, SFTP or SCP client (in SFTP mode) was used to connect to build.eclipse.org using your committer account, however this is no longer supported).
  • Please ensure that the file permissions include world-readable (664; rw-rw-r--) and directory permissions allow for world-executable (775, rwxrwxr-x).
  • Although you can link directly to download.eclipse.org/yourfile.zip, you can also use the Find a Mirror script (info below). Using this script allows you to view download statistics and allows users to pick a nearby mirror site for their download.

Once your files are on the download.eclipse.org server, they are immediately available to the general public. However, for release builds, we ask that you wait at least four hours for our mirror sites to fetch the new files before linking to them. It typically takes a day or two for all the mirror sites to synchronize with us and get new files.

Please note that although we tolerate PHP, HTML and JPG/GIF files on download.eclipse.org, we encourage you to put such files on www.eclipse.org. Those files are not mirrored to public mirror servers.

SYMLINKS are not supported. We cannot ensure that all our mirror servers support and honour symlinks. For that reason, please avoid the usage of symlinks.

Move files to archive.eclipse.org?

Because our mirror sites don't have as much disk space for Eclipse files as we do, we have created an https://archive.eclipse.org site for you to store older release builds.

The archive.eclipse.org structure is similar to that of download.eclipse.org. To move your files, we recommend using a job on your project's Jenkins instance. Alternatively, you can navigate to https://download.eclipse.org/path/to/your/project. From download.eclipse.org, authenticated committers can Archive files and folders (the archive process maintains the directory structure). From https://archive.eclipse.org/path/to/your/project files and folders can be permanently deleted.

Some folders contain an index file - such as index.html, which will be shown instead of the directory contents. Append /listing to the URL and the contents will be shown. https://download.eclipse.org/path/to/your/projectdirectory/listing

Note: if you preserve the exact path and filename from download.eclipse.org to archive.eclipse.org, you don't need to change your links (although it is recommended). This works for p2 repos, direct links to https://download.eclipse.org and if your links use the Find a Mirror script.

This link will work if /path/to/a/file.zip is on download.eclipse.org, or if it gets moved to the same place on archive.eclipse.org

  https://www.eclipse.org/downloads/download.php?file=/path/to/a/file.zip

P2 repositories: P2 repositories are not normally accessed via the mirror selection script. Therefore, extra treatment is required when the move should be made transparently without affecting users who may still have the original URL.

Equinox/p2/p2.mirrorsURL#Moving_a_repo_to_archive.eclipse.org has a discussion how to achieve this (work in progress).

Use mirror sites/see which mirrors are mirroring my files?

Link to your download files like this:

   Acceptable: https://download.eclipse.org/path/to/a/file.zip    

   Preferred: https://www.eclipse.org/downloads/download.php?file=/path/to/a/file.zip

Parameters for above script:

  • file (Required): specify the filename, relative to the downloads home, starting with a "/". This file must exist in the downloads area. Although you can specify a directory name, your mirror list will be more accurate if you specify a file.
  • format (Optional): specify html (default) or xml. Useful for building the mirrors.xml for Update sites.
  • protocol (Optional): ftp or http: list only ftp or http mirrors only (both are the default)
  • r (DEPRECATED): specify 1 to automatically redirect to the best mirror (the one that would normally be at the top) without asking the user to choose.
  • nf (DEPRECATED): specify 1 to get an actual 404 Not Found error if the file doesn't exist (instead of a lovely page saying so).

The script will examine the Last Modified timestamp of the given file and return only those mirrors that have synchronized with Eclipse.org after that time.

Examples:

   All mirrors of the Lepido project, in XML format:
   https://www.eclipse.org/downloads/download.php?file=/technology/lepido/M1/content.jar&format=xml

   Get a file from a random mirror, without prompting
   https://download.eclipse.org/eclipse/downloads/drops/R-3.1-200506271435/eclipse-SDK-3.1-win32.zip

PLEASE NOTE: We have a list of excluded file patterns -- files that are *not* sent to our mirrors. Nightly and Integration builds are typically very large and don't get many downloads, therefore it's typically more costly (in terms of bandwidth) to mirror them than to support the few client downloads they generate. At time of writing, our exclusion list is:

  • .nfs*
  • apitools/
  • apidocs/
  • archive/
  • archives/
  • /athena
  • builds/N*
  • drops/I*
  • drops/N*
  • drops/M*
  • *.jpg
  • *.gif
  • callisto/*
  • compilelogs/
  • eclipse.org-common/
  • eclipse/testUpdates*
  • eclipse/updates/3.2milestones
  • /eclipse/updates/3.6-I-builds/
  • *eclipse/updates/*-X*
  • *eclipse/updates/*-Y*
  • dev/TPTP*
  • /tools/cdt/builds
  • modeling/gmf/downloads/drops/B*
  • *drops/*/N*
  • *drops/*/I*
  • *javadoc/
  • *javadocs/
  • linuxtools/N*
  • *nightly*
  • *Nightly*
  • *staging*
  • /webtools/downloads/drops/*/M*
  • performance/
  • /releases/staging
  • /releases/europa
  • testresults/
  • /rt/eclipselink/nightly*
  • /technology/babel/update-site*
  • /technology/cosmos
  • /technology/ohf
  • /technology/tigerstripe
  • testcompilelogs/
  • testResults/
  • /tools/downloads
  • /tools/orbit/committers
  • */N202*
  • */I202*
  • */I.I202*
  • */I-*
  • */N-*
  • *integration*/
  • xref/
  • */M20*
  • /rt/eclipselink/maven.repo*

Use the Find a Mirror script?

See the section above.

Enable mirrors / use mirrorsURL for my p2 repo?

Your artifacts.xml (jar) should have a p2.mirrorsURL property. Here is a an example from https://download.eclipse.org/eclipse/updates/3.6/R-3.6.2-201102101200/artifacts.jar

<repository name='&quot;Eclipse Project Test Site&quot;' type='org.eclipse.equinox.p2.artifact.repository.simpleRepository' version='1'>
  <properties size='4'>  
    <property name='p2.compressed' value='true'/>
    <property name='p2.timestamp' value='1297373227427'/>
    <property name='publishPackFilesAsSiblings' value='true'/>
    <property name='p2.mirrorsURL' value='https://www.eclipse.org/downloads/download.php?file=/eclipse/updates/3.6/R-3.6.2-201102101200&amp;format=xml'/>  
  </properties>

A more detailed description can be found at Equinox/p2/p2.mirrorsURL.

Ideally, everyone, for all p2 repositories, should use this property, since even if not mirrored currently, it does not hurt anything in that case, and you never know when your repository might become mirrored. In fact, failure to use this property can result in too many requests for jar files coming directly to 'download.eclipse.org' and greatly slow down the network and use too much bandwidth. If this happens for your project (or repository) measures may be taken to automatically redirect all such requests somewhere else, which often does not work well; for examples, see .

Include a p2.index file at p2 repository site?

A little documented aide to p2 is to include a special file named "p2.index" at your p2 repository URL site. Every well-behaved, well-optimized p2 repository should have one. This is especially important for composite repository sites as it can save several unsuccessful round trips to download server looking for files that do not exist. For "how to" instructions, see the p2 wiki. For history and deeper technical discussion, see .

See download statistics?

The Find a Mirror script tracks download requests once the user has picked a mirror site (or the main Eclipse download site). You can also view download stats for files downloaded via p2 if you enable your p2 repository for download statistics. To view these statistics, use the Live Download Statistics tool (Portal > Project Committer > Tools for all Committers). Download statistics are not available for direct downloads.

For more information, please see the Project Download Stats page.

Sign my Jar/plugins/Windows exe/macOS App files?

The Eclipse Foundation allows committers to sign JAR and some executable files on its behalf. Signing is done from any of the Jenkins servers. There are three ways to sign:

CBI Maven signing plugin

Using the CBI Maven Plugins the signing process can be directly performed at the end of a Maven Tycho build.

JAR signing

Ensure that all created JAR files are correctly signed by using the eclipse-jarsigner-plugin

<plugin>
  <groupId>org.eclipse.cbi.maven.plugins</groupId>
  <artifactId>eclipse-jarsigner-plugin</artifactId>
  <version>${cbi-version}</version>
  <executions>
    <execution>
      <id>sign</id>
      <phase>verify</phase>
        <goals>
          <goal>sign</goal>
        </goals>
    </execution>
  </executions>
</plugin>

Windows signing

To sign the Windows executables use the eclipse-winsigner-plugin

<plugin>
  <groupId>org.eclipse.cbi.maven.plugins</groupId>
  <artifactId>eclipse-winsigner-plugin</artifactId>
  <version>${cbi-version}</version>
  <executions>
    <execution>
      <id>sign</id>
      <goals>
        <goal>sign</goal>
      </goals>
      <phase>package</phase>
      <configuration>
        <signFiles>
          <signFile>${project.build.directory}/products/${product-folder}/win32/win32/x86_64/eclipse.exe</signFile>
          <signFile>${project.build.directory}/products/${product-folder}/win32/win32/x86_64/eclipsec.exe</signFile>
        </signFiles>
      </configuration>
    </execution>
  </executions>
</plugin>

macOS signing

To sign the macOS executables use the eclipse-macsigner-plugin

<plugin>
  <groupId>org.eclipse.cbi.maven.plugins</groupId>
  <artifactId>eclipse-macsigner-plugin</artifactId>
  <version>${cbi-version}</version>
  <executions>
    <execution>
      <id>sign</id>
      <goals>
        <goal>sign</goal>
      </goals>
      <phase>package</phase>
      <configuration>
        <signFiles>
          <signFile>${project.build.directory}/products/${product-folder}/macosx/cocoa/x86_64/Eclipse.app</signFile>
        </signFiles>
        <timeoutMillis>300000</timeoutMillis> <!-- 5 min -->
        <continueOnFail>${macSigner.forceContinue}</continueOnFail>
        <entitlements>${project.basedir}/application.entitlement</entitlements>
      </configuration>`  
    </execution>
  </executions>
</plugin>

macOS DMG file creation

macOS applications are typically published as .dmg files, which are containers that serve as installers with additional security information to avoid that the application gets tampered. To create a DMG file the eclipse-dmg-packager can be used.

<plugin>
  <groupId>org.eclipse.cbi.maven.plugins</groupId>
  <artifactId>eclipse-dmg-packager</artifactId>
  <version>${cbi-version}</version>
  <executions>
    <execution>
      <goals>
        <goal>package-dmg</goal>
      </goals>
      <phase>integration-test</phase>
      <configuration>
        <source>${project.build.directory}/products/${product-id}-macosx.cocoa.x86_64.tar.gz</source>  
        <continueOnFail>true</continueOnFail>
        <timeoutMillis>600000</timeoutMillis> <!-- 10 min -->
        <continueOnFail>${macSigner.forceContinue}</continueOnFail>
        <sign>true</sign>
      </configuration>
    </execution>
  </executions>
</plugin>

macOS Notarization

Since macOS Catalina macOS software that is published outside the AppStore needs to be notarized, so the Gatekeeper gets information about trusting the software or not.

As of now, the notarization is not available as a Tycho plugin. Therefore the macos-notarization-service webservice needs to be used in the Jenkins job similar to the following snippet:

PRODUCT_ID=...
BUILD_DIR="${WORKSPACE}/${PRODUCT_ID}/target/products/"
DMG=${PRODUCT_ID}-macosx.cocoa.x86_64.dmg

pushd $BUILD_DIR

PRIMARY_BUNDLE_ID="app-bundle"

RESPONSE=$(curl -s -X POST -F file=@${DMG} -F 'options={"primaryBundleId": "'${PRIMARY_BUNDLE_ID}'", "staple": true};type=application/json' https://cbi.eclipse.org/macos/xcrun/notarize)  

UUID=$(echo $RESPONSE | grep -Po '"uuid"\s*:\s*"\K[^"]+')
STATUS=$(echo $RESPONSE | grep -Po '"status"\s*:\s*"\K[^"]+')

while [${STATUS} == 'IN_PROGRESS'](${STATUS}_==_'IN_PROGRESS' "wikilink"); do
  sleep 1m
  RESPONSE=$(curl -s https://cbi.eclipse.org/macos/xcrun/${UUID}/status)  
  STATUS=$(echo $RESPONSE | grep -Po '"status"\s*:\s*"\K[^"]+')
done

if [${STATUS} != 'COMPLETE'](${STATUS}_!=_'COMPLETE' "wikilink"); then  
  echo "Notarization failed: ${RESPONSE}"
  exit 1
fi

rm "${DMG}"

curl -JO "https://cbi.eclipse.org/macos/xcrun/${UUID}/download"
popd

A more detailed script is the Oomph script.

For further information on the CBI Maven Plugins have a look at: https://www.eclipse.org/cbi/sitedocs/

Note that these plugins use the web services in the background.

Web service

Using a web POST method, individual JAR files can be signed from any of the internal Jenkins servers with this service:

https://cbi.eclipse.org/jarsigner/sign

The output of that service will be the signed file. Please note that the web service does not pack or process jar files. You must condition/pack them yourself prior to signing if you wish to do so.

JAR FILES: Submit unsigned-jar.jar and save signed output to signedfile.jar

curl -o signedfile.jar -F file=@unsigned-jar.jar "https://cbi.eclipse.org/jarsigner/sign"

WINDOWS EXE: Submit Windows unsigned.exe and save signed output to signed.exe

curl -o signed.exe -F file=@unsigned.exe "https://cbi.eclipse.org/authenticode/sign"

WINDOWS MSI: Submit Windows unsigned.msi and save signed output to signed.msi

curl -o signed.msi -F file=@unsigned.msi "https://cbi.eclipse.org/authenticode/sign"

MAC: Submit unsigned and save signed output to signed.zip *Note: You must zip your entire .app directory, for example: zip -r unsigned.zip Eclipse.app

curl -o signed.zip -F file=@unsigned.zip https://cbi.eclipse.org/macos/codesign/sign

If you need to set entitlements on your app / binary (see https://developer.apple.com/documentation/security/hardened_runtime?preferredLanguage=occ for details), add an entitlements part to the request like below

curl -o signed.zip -F file=@unsigned.zip -F entitlements=@file.entitlements https://cbi.eclipse.org/macos/codesign/sign

Using the webservice is equally easy from Ant. Note that ${filename} cannot be a path. Input and output file name can be the same.

<exec dir="${dirname}" executable="curl">
  <arg value="--output"/>
  <arg value="${filename}"/>
  <arg value="--form"/>
  <arg value="file=@${filename}"/>
  <arg value="--silent"/>
  <arg value="--show-error"/>
  <arg value="--fail"/>
  <arg value="https://cbi.eclipse.org/jarsigner/sign"/>
</exec>

Using the web service to sign Mac and Windows applications is also easy from Tycho, see

What about GPG signing?

JAR signing of the bundles and GPG-signing of the Maven artifacts are two different steps. Once a jar has been "jar-signed", you may or may not GPG sign the corresponding Maven artifact (.jar + .pom file) so as it can be deployed on Central. As you hinted, JAR signing has to be done before the GPG signing, since doing it the other way around would break the GPG signature.

So you first have to sign your JAR file with the Eclipse Fdn certificate, either using the Maven plugin from CBI, the command line utility, or the signing web service – see above. Once you have your signed JAR, you can GPG sign it and stage it on Central like this:

mvn gpg:sign-and-deploy-file   \
    -DpomFile=target/myapp-1.0.pom  \
    -Dfile=target/myapp-1.0.jar  \
    -Durl=http://oss.sonatype.org/service/local/staging/deploy/maven2/  \  
    -DrepositoryId=sonatype_oss

Publish to Maven Central

To deploy to Maven Central from your JIPP, you'll need the infrastructure team's assistance to

  • Create a project-specific account at Sonatype OSSRH
  • Generate a GPG key pair for your JIPP user
  • Configure your JIPP to GPG sign and upload artifacts

It takes a bit of time but afterwards, you will only be required to use a dedicated Maven settings on your JIPP.

To get started, please file an issue against https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues asking for your JIPP to be configured to let you publish to Maven central (don't forget the name of your Eclipse project).

If you want to publish jars from already released p2 repositories, consider using the strategy adopted by the Eclipse Platform. More info: Platform-releng/Publish To Maven Central

Builds

Access/request Jenkins services

Please refer to the Jenkins document.

Code Quality Analysis

Mailing Lists

Set up a new mailing list?

Because Mailing Lists are subject to SPAM and can adversely affect eclipse.org performance (imaging sending 200 e-mails to a list that contains 3000 members), proper care is taken in configuring each list. New mailing lists are set up by the WebMaster for this reason. Also, the webmaster creates an HTML view (called mailing list archives) of mailing list postings for archive and search purposes.

View list members?

Because mailing lists contain private information, such as a member's e-mail address, name and surname, we cannot publicly display this information. However, the PMC or Project Lead can become the list administrator, which would allow you to view the membership information for your lists. The PMC/Project lead can inquire about list administration to the WebMaster, stating which lists they would like to manage.

Eclipse Wiki

The Eclipsepedia wiki has been scheduled for shutdown, see the shutdown plan

Eclipse Servers

Eclipse Foundation IT Service Level Objectives

Copyright © Eclipse Foundation, Inc. All Rights Reserved.     Privacy Policy | Terms of Use | Copyright Agent