Please Migrate SimRel to GitHub

Summary

The following repositories of the "SimRel" project should be migrated to a new eclipse-simrel GitHub organization:

• https://git.eclipse.org/c/simrel/org.eclipse.simrel.build.git/ - shorten the name of simrel.build
• https://git.eclipse.org/c/simrel/org.eclipse.simrel.tools.git/ - shorten the name to simrel.tools

The following repositories should also be created:

• .github
• .eclipsefdn

Use the following as the avatar, i.e., use the same one as used by https://github.com/eclipse-platform/:

• https://avatars.githubusercontent.com/u/95248176?s=200&v=4

Include a links to https://ci.eclipse.org/simrel and to https://accounts.eclipse.org/mailing-list/cross-project-issues-dev

The title should be Eclipse Simultaneous Release and the blurb can be this:

SimRel provides infrastructure for coordinating the release of projects used to build Eclipse IDE/RCP applications.

I.e., make the organization page look similar to this:

To avoid disruption to the 2023-09 release, please schedule the migration shortly after the September 13th release. I.e., September 14th or of 15 would be ideal.

Priority

• Urgent
• High
• Medium
• Low

Severity

• Blocker
• Major
• Normal
• Low

Impact

To reiterate, please do this right after the September 13th release.

added Migration label

added IT-prioritymedium IT-severitynormal labels

added statetodo label

changed due date to September 14, 2023

assigned to @chills2023

Please don't overlook the requested schedule!

I.e., DO NOT DO IT NOW.

Hi @emerks , is it OK to migrate repos today?

Yes. Things are quiet right now, so it's a good time.

@emerks I can't find the project in projects.eclipse.org, is there an alternate name for the project?

SimRel isn't actually "real" project in that sense.

@emerks can you please elaborate what you mean? We can't make the migrated repository work together with our tooling, unless we correctly identify it in Project Management Interface.

Unfortunately I can only answer with questions...

To which project does this ci instance belong?

https://ci.eclipse.org/simrel/

Which project owns these repositories:

Which project is associated with group genie.simrel which has these permissions in the file system?

https://ci.eclipse.org/simrel/view/Releng%20jobs/job/simrel.releng.ssh/4/console

Unfortunately I think SimRel is not a project in the PMI.

@jograham FYI, this "non-projectness" of SimRel could hold things up.

We may need to create a placeholder "project" in PMI. Simrel is not a project per se; but a project of projects, and operates just like a project on every level.

@wbeaton if my reality matches the real reality (?), do you agree with our aligning PMI & the projects structure to reflect this?

IMHO making SimRel a real PMI project is a good idea - it is an unusual project in the same way that e.g. Eclipse Orbit or Eclipse Packaging is, for example Orbit and Packaging both special rules on committers and SimRel does too. Choosing a PMC to place it under could be contentious, but either Tools (same as Orbit) or Technology (same as Packaging) would work.

Note that https://projects.eclipse.org/releases "namespace" on PMI is essentially owned by "non-project" SimRel project too and I suspect @wbeaton would be delighted if the project could manage this resource too.

Just to add some extra data(or confusion): simrel (as a project) is tracked via an internal only placeholder, but that's just in our backend..

I agree with @jograham and @droy that simrel needs to be a little more 'real' as a project(we should also use this opportunity to update the seriously old-school 'callisto-dev' permissions group)

I've always thought that simrel belongs to https://projects.eclipse.org/projects/technology.packaging and seen them as single project. So why not make simrel part of packaging project ?

As current PL of EPP bringing SimRel into EPP is ok - not sure how to manage distinct sets of committers on SimRel vs (current) EPP. Maybe we shouldn't worry about that and combine it together?

thats the way the adoptium project is setup. They have 1 GitHub organization with the name of the top level project, which also hosts all the subprojects, and each subproject has a distinct committers team that is configured via the PMI entry for each subproject.

The downside of that approach is that tooling is a bit more complicated wrt repositories as the assumption that all repos of a GitHub organization belong to a single project is not true anymore (we have a field in the PMI for that purpose, but that can not be used in such a scenario). But if the number of repos is low and not supposed to change a lot, this is not really a concern imho.

+1 to that

So what is the final decision? How do I need to setup the new repository on Github ?

Personally I'd just as soon (rather) that it be its own top-level organization in GitHub; nested projects was something we had in Modeling and we still haven't completely been able to get rid of that. I certainly don't want see these pushed together or become conflated:

• https://ci.eclipse.org/simrel/
• https://ci.eclipse.org/packaging/

In the end, EPP and SimRel are currently separate so it just seems cleaner to keep it that way such that each GitHub organization can focus on its own domain. So if it's my decision to make, then that would be my decision.

I'm perfectly fine and good with a SimRel PMI entry being created. I have no strong opinion about whether this be parented by tools versus technology; probably the later makes the most sense though.

+1 to what Ed said (with technology as TLP), which I think looks like:

• https://github.com/eclipse-simrel
• https://projects.eclipse.org/projects/technology.simrel
• https://ci.eclipse.org/simrel/
• etc...

If it walks like a duck, and quacks like a duck... for all intents and purposes, simrel is a project and has been behaving like one for many, many years.

It would be great to hear from the projects team on this. In the interim, let's assume we're doing the Right Thing(tm) and go with the above proposal, even if the new project provisioning process is not done yet.

+1 to a technology.simrel project. I believe we need a new project provisioning process to happen - can someone at the webmaster desk kick this off?

+1 to https://github.com/eclipse-simrel - @chills2023 you can do the migration there.

+1 to https://ci.eclipse.org/simrel/ - already exists.

/cc @wbeaton @mdelgado624

@emerks when trying to move the repo to Github, I'm having this error :

remote: error: object 1d52ed1f2c9846cb66bf446984bc1c816893f29e: badEmail: invalid author/committer line - bad email

git cat-file -p 1d52ed1f2c9846cb66bf446984bc1c816893f29e 1 ↵ tree 353d944b322d845e964ac6d63076684e37e42a0b parent ebf6f83ee64478a0b63bbd5dd307c41d017d3b3f author Markus Knauer < ``mknauer@eclipsesource.com`` 1425032351 +0100 committer Markus Knauer < ``mknauer@eclipsesource.com 1425032351 +0100

Change RAP p2 repository URLs to release URLs (2.3.2 final / Luna SR2)

Change-Id: Ibb1f9c63778c4ebe784f3e9a36fce3322c17d839 Signed-off-by: Markus Knauer ``mknauer@eclipsesource.com

Is it possible to fix this on your side? I don't want to mess with the commit history, unless you are not able to fix the issue on your side

I’m not able to fix such a thing.

And before you are asking… I am not able to fix that either. ;-)

No worries, I'm trying to fix it on my side :)

You could just ignore the issue altogether

git config fsck.badEmail ignore

from https://stackoverflow.com/a/70970736/3784643

Already tried that, doesn't work when I want to push to Github, they actively refuse commits with fsck errors

Maybe a skiplist helps as explained here: https://github.com/yi-editor/yi/issues/959#issuecomment-268020032

https://git-scm.com/docs/git-fsck#Documentation/git-fsck.txt-fsckskipList

Same behavior, this is to ignore/skip local check, but I'm pushing to github and the commit is still present when i'm pushing. I'm trying with git replace --edit 1d52ed1f2c9846cb66bf446984bc1c816893f29e fixed the error in the email address, now I need to push to the old repo, and then redo a clone locally and then push to Github after.

Seems like I don't have permission to push even if i'm in the Administrators group

remote: Unauthorized fatal: Authentication failed for '``https://git.eclipse.org/r/simrel/org.eclipse.simrel.build/``'

https://github.com/eclipse-simrel has been created, but there is still no simrel project in PMI. So the sync script won't be able to automatically assign permissions to project members.

@wbeaton can we fast-track the creation of this project? Does someone (e.g. @emerks) need to create a EMO ticket for that?

This repo is still empty though:

https://github.com/eclipse-simrel/simrel.build

I'm working on it.

@fgurr let me know when it's done how you fixed the bad commit issue (badEmail)

The Simrel has never been an Eclipse open source project. AFAIK the Simrel is "owned" Eclipse IDE Working Group and all resources required for its operation are managed as such.

If we now feel that the Simrel should be a proper Eclipse project, then we need to follow the usual process. I am happy to help expedite that process, but we need a project team and project lead to drive the process.

Note that "create a project" was only one proposed solution to the "permissions problems". I don't suppose there are any other solution that you could suggest? I don't care how we make it work as long as we get it working sooner rather than later because very soon folks will need to start contributing for 2023-12 M1 and I still need to get all the infrastructure working again with the new locations...

BTW, will "expediting" help with:

A proposal will be open for community review for a minimum of two weeks.

Ed is the natural project lead and about the project team - IMHO in this case this is already available information by the list of people with commit rights to https://git.eclipse.org/c/simrel/org.eclipse.simrel.build.git/

@wbeaton I don't want a temporary solution to become a permanent solution, but would it make sense to give simrel committers access to the GitHub repos manually until the simrel Eclipse project is up and running? This would lower the level of urgency at least.

Note that "create a project" was only one proposed solution to the "permissions problems". I don't suppose there are any other solution that you could suggest?

All of the simultaneous release infrastructure is current managed as working group resources (I believe that this includes the builds, for example). Managing this as an Eclipse open source project makes a lot of sense to me; it should make a lot of the management tasks a lot easier.

To the best of my knowledge we have two options: leverage our existing project infrastructure by making it a project, or continue to treat it as resource that we manage as a special case on behalf of the working group.

Eliminating the special case will make ongoing support easier for everybody involved.

BTW, will "expediting" help with:

I'll see what I can do. I believe, however, that if we can just leave it as a special case a little longer as Fred suggests, it doesn't really matter.

would it make sense to give simrel committers access to the GitHub repos manually until the simrel Eclipse project is up and running?

It's already a special case. So, sure.

@wbeaton

That all sounds very good! Making it less special is going to be a good thing in the long term.

I'm also very happy with Fred's suggestion so that I can ensure that infrastructure is working in time for people starting to use it. This makes expediting the other things kind of a non-issue.

I will write a project proposal as soon as possible...

Thanks everyone for the constructive suggestions and for the help moving this forward!!

assigned to @wbeaton

unassigned @chills2023

It's (almost) done: https://github.com/eclipse-simrel/simrel.build

Here is what I did (locally):

git clone https://git.eclipse.org/r/simrel/org.eclipse.simrel.build.git
cd org.eclipse.simrel.build/
git remote add origin-gh git@github.com:eclipse-simrel/simrel.build.git
git replace --edit 1d52ed1f2c9846cb66bf446984bc1c816893f29e
#fix issue
git branch -M main
git push -u origin-gh main
for tag in $(git tag); do git push origin-gh $tag; done

Push only failed for tag LunaSR2:

Enumerating objects: 423, done.
Counting objects: 100% (423/423), done.
Delta compression using up to 16 threads
Compressing objects: 100% (180/180), done.
Writing objects: 100% (381/381), 44.21 KiB | 44.21 MiB/s, done.
Total 381 (delta 259), reused 320 (delta 201), pack-reused 0
remote: error: object 1d52ed1f2c9846cb66bf446984bc1c816893f29e: badEmail: invalid author/committer line - bad email
remote: fatal: fsck error in packed object
error: remote unpack failed: index-pack failed
To github.com:eclipse-simrel/simrel.build.git
 ! [remote rejected]   LunaSR2 -> LunaSR2 (failed)
error: failed to push some refs to 'github.com:eclipse-simrel/simrel.build.git'

I will try to fix it.

Also, a heads-up: I've changed the default branch for the following repos from master to main:

• https://github.com/eclipse-simrel/simrel.build
• https://github.com/eclipse-simrel/simrel.tools

btw. is there a way to create a "private" Eclipse project simply for our tooling to be able to handle that gracefully, but in fact its not a real Eclipse project with all the required paperwork or technicalities.

Our tooling to a large degree relies on some conventions which is a good thing, but that also requires that things are always the same, which would be easier to achieve with something like that. With private I mean that projects like that would not be visible e.g. via projects.eclipse.org and the like.

I will try to fix the missing tag in the simrel.build repo and set up permissions for committers before Friday EOB.

I bet no one would notice a missing tag. It unlikely the p2 sites in the model’s historical state generally exist anyway.

My approach from #3624 (comment 1211204) was not working out. I had to do it again.

Here are the commands I used:

git clone https://git.eclipse.org/r/simrel/org.eclipse.simrel.build.git
cd org.eclipse.simrel.build/
git remote add origin-gh git@github.com:eclipse-simrel/simrel.build.git
git replace --edit 1d52ed1f2c9846cb66bf446984bc1c816893f29e
#fix issue
git filter-repo --force
git branch -M main
git push -u origin-gh main
git push origin-gh --tags
git push origin-gh --all

I used git-filter-repo in the process.

With these steps, I was able to push all branches and tags cleanly to the new GitHub repo.

CAVEAT: I had to rewrite history, so any local commits that have not been pushed yet, will need to be rebased.

• 111 members were found in the callisto-dev LDAP group.
• 94 users of them were added to the GitHub simrel-committers team.
• 17 users don't have a GitHub handle specified in their Eclipse account, so they could not be added.
• 3 of those 17 users are bot accounts.
• @emerks has been added to the GitHub simrel-project-leads team.

The list of committers probably contains people that are no longer involved in Simrel releases. So it should be cleaned up at some point.

A GitHub bot (ID: eclipse-simrel-bot) has been created. Credentials have been added to the simrel JIPP.

@netomi can you set up otterdog for https://github.com/eclipse-simrel?

@emerks self-service has now been enabled for the simrel org, the starting point is available at https://eclipse-simrel.github.io/.eclipsefdn/ .

But you already know your way around so I will leave it up to your for now

The PMI config is still missing, will be completed when the project is available. Right now I did setup permissions manually to some degree.

Things are really progressing well thanks to you guys' great help! I've already gotten the aggregation and the validation working via this two-in-one job:

https://ci.eclipse.org/simrel/job/simrel.build/

Please don't forget about the .github repo...

ty for the reminder, will create a PR

just for the sake of it, now that the self-service is enabled, you could create (or at least request) that as well :-)

mentioned in issue eclipsefdn/security/otterdog#112 (closed)

marked this issue as related to eclipsefdn/security/otterdog#112 (closed)

A project proposal has been opened:

• eclipsefdn/emo-team/emo#599 (closed)
• https://projects.eclipse.org/proposals/eclipse-simrel

Eclipse SimRel is a project now, see: https://projects.eclipse.org/projects/technology.simrel.

I guess we can close this ticket. Please re-open if something is still missing.

Yes, I have no outstanding needs. Thanks everyone who helped!

closed

removed statetodo label