Cleanup OSEE Jenkins instance

Summary

While preparing the migration of the OSEE Jenkins instance to our new backend storage device, I noticed the following:

• several jobs/branches that are 1-5 years old
• considerable disk space usage (109 GB)

Some examples:

• https://ci.eclipse.org/osee/job/osee_nr/
  • Last successful build was more than 5 years ago
• https://ci.eclipse.org/osee/job/osee_one/
  • Several builds are marked to be kept forever.

Tasks

• Please keep in mind that Jenkins is not a file server to archive build artifacts. Any build artifacts that should be kept around for some time, should be copied to the project's download directory at download.eclipse.org.

• Please consider all build jobs during clean up and decide if they are still needed.

Side note: If you are interested, we can also assist with transitioning from Freestyle jobs to Jenkins pipeline jobs, where the build configuration is stored in the Git repo. It also allows using the same or similar build configurations for different branches without having to re-create Jenkins jobs for each branch.

/cc @dmiller @mmasterso @ddunne @rbrooks

changed due date to January 20, 2025

added IT-prioritymedium IT-severitynormal service:jenkins labels

assigned to @dmiller

assigned to @mmasterso, @ddunne, and @rbrooks

changed due date to January 27, 2025

@dmiller @mmasterso @ddunne @rbrooks gentle ping

I agree we are likely keeping builds around we no longer need. Is there a threshold we can shoot for for this effort?

We definitely have some seemingly old jobs that are still in use by builds on Jenkins and some more being used by proprietary builds in house. I'd like to focus on just deleting files we don't need anymore for now and create a plan to archive those artifacts we do need somewhere else in the near future.

I agree we are likely keeping builds around we no longer need. Is there a threshold we can shoot for for this effort?

By default, Jenkins instances get 50 GB of disk space. So anything below that with some headroom is ok. Less is more.

I'd like to focus on just deleting files we don't need anymore for now and create a plan to archive those artifacts we do need somewhere else in the near future.

It's important to not only remove files, but to ensure that future builds will not store more artifacts than required. E.g. deleting 4 GB of artifacts now, but the next 4 builds will generate and keep 1 GB each, does not help here.

In most cases, keeping only the last build with artifacts is enough. This obviously depends on the size of the artifacts.

Your support with this is appreciated.

@fgurr What's the easiest way to see our current disk usage? I see individual file sizes in artifacts but is there a button for rolled up stats for entire folders or dashboards?

We are working on making the disk usage accessible for committers and project leads.

The OSEE Jenkins is using 78 GB of disk space at the moment.

See also:

@dmiller @mmasterso @ddunne @rbrooks we'd appreciate your support to clean up the instance.

If we don't hear back from the project before February 10, 2025, we will start cleaning up as we see fit.

changed due date to February 10, 2025

I've deleted 2 of the osee_one builds. We would like to keep 1 or 2 builds for some internal things. I will cleanup our OSEE jobs that are no longer needed.

I've gone through and cleaned up most of the builds taking up alot of space. What is our current disk usage?

64 GB

• https://ci.eclipse.org/osee/job/org.eclipse.ote.build.dependencies/
  • https://ci.eclipse.org/osee/job/org.eclipse.ote.build.dependencies/95/ and https://ci.eclipse.org/osee/job/org.eclipse.ote.build.dependencies/92/ are marked to keep them forever.
    => If you still want to keep around snapshot builds that are 4/5 years old, please move them to the project's respective directory on download.eclipse.org.
• What is the value of keeping around jobs and builds that have not been built in a long time? E.g.
  • https://ci.eclipse.org/osee/job/org.eclipse.ote.build.dependencies/ (no builds in 4 years and disabled)
  • https://ci.eclipse.org/osee/job/org.eclipse.ote_dev_review/ (no builds in 4 years and disabled)
  • https://ci.eclipse.org/osee/job/org.eclipse.ote.build.dependencies__ALPHA/ (no builds in 3 years and disabled)
  • https://ci.eclipse.org/osee/job/osee_dev_ote/ (no builds in 2.5 years and disabled)
  • ...

We generally use these as dependency p2 sites for various projects internally. I will work on this today and move these artifacts to a secure location.

I was able to go through all of those noted in this comment and every build artifact under the "OTE" views (OTE_0.26.0, OTE_0.26.13, etc) and archive to a secure area. I think we are free to remove those artifacts.

Should we go and delete each of those builds manually as well?

• First step: modify all builds to no longer be marked as "keep forever".
• Second step: remove builds that were marked as "keep forever" and are not the latest build or the last successful build
• Third step: consider removing build jobs completely that are no longer used

BTW: current disk usage is 57 GB.