[Bug 577152] Shutdown the Foundations Gerrit instance.
| Bugzilla Link | 577152 |
| Status | NEW |
| Importance | P3 normal |
| Reported | Nov 09, 2021 11:23 EDT |
| Modified | Dec 01, 2021 10:24 EDT |
| Blocks | 577150 |
Description
Right now the Foundation has 2 Git services: Gerrit and Gitlab. As more projects move towards wanting better integration between their code, issues and community interaction, the Webmaster team feels it's time to start planning the shutdown of our Gerrit service.
Ideally projects would move organically, but that is likely to take more time than desirable. Running multiple services that kinda-sorta-maybe duplicate certain functionality is overhead we can do without.
Thoughts, comments, questions?
-M.
Activity
(In reply to Thomas Wolf from comment #2)
Or they can leave altogether and move to some provider offering a managed
Gerrit. Alternatives exist.Do I get it right that EGit/JGit would be happy moving to GerritHub?
If so
@webmasters: would adding also enabling GerritHub as an exceptional external service for established projects that prefer it be possible? Would it fit in your goals?By Thomas Wolf
(In reply to Mickael Istria from comment #3)
(In reply to Thomas Wolf from comment #2)\
Or they can leave altogether and move to some provider offering a managed
Gerrit. Alternatives exist.
Do I get it right that EGit/JGit would be happy moving to GerritHub?For now this is just a possibility that is being discussed. Don't rush things. The technicalities would have to be figured out first. AFAIK, GerritHub is a mirroring setup between Github and a Gerrit. I don't know what might need to be set up differently from normal Eclipse Github projects, if anything.
A move to Gerrithub might have the advantage that we wouldn't have to lose old reviews and review comments. (Which I find extremely helpful when trying to figure out the history of piece of code.)
(Perhaps the ability for JIRO Jenkins jobs listening on an event stream from a GerritHub Gerrit? In any case we'd like to keep using JIRO for now -- until the foundation perhaps decides in the future that maintaining that has become too much of a hassle and not worth the trouble because everybody has moved to Github and will be using Github actions...)
And no, we wouldn't be "happy". It'd be the lesser of evils. Any move means we have to spend our volunteer time on things we're not really interested in. I at least am not paid to maintain EGit or JGit.
I would imagine Trace Compass is in the same boat.
Matthew's opinion, I would need to consult with the Trace Compass team to have a more full view:
Gerrit's review capabilities are far superior to the PR design. I hope we can stay in Gerrit for a good long while as it prevents many bugs and speeds up reviews.
@webmasters: would adding also enabling GerritHub as an exceptional external
service for established projects that prefer it be possible? Would it fit in
your goals?It would not fit into our goals, as we would have to maintain the ECA validators and backup mechanisms for another third party service for possibly only two projects.
(In reply to Thomas Wolf from comment #4)
And no, we wouldn't be "happy".
Totally understand. We're not happy either. In a perfect world, folks use the same tool perpetually and we never have to waste time migrating to anything.
Migrations are a ton of work for us as well. But we must follow the tools and trends are developers adopt en masse.
By Thomas Wolf
(In reply to Denis Roy from comment #6)
@webmasters: would adding also enabling GerritHub as an exceptional external
service for established projects that prefer it be possible? Would it fit in
your goals?
It would not fit into our goals, as we would have to maintain the ECA
validators and backup mechanisms for another third party service for
possibly only two projects.Not sure. After all, Gerrithub is all about using the Gerrit workflow with Github repos. Might turn out that on the EF side, not much was needed. If it becomes a real option for us, I suppose someone more technically knowledgeable on these issues than me will get in touch.
As many discussions have happened even before this issue was created, I have the impression that it's hard for projects to easily consider the move to GitHub because of some missing functionality regarding issues.
So in order to better encourage projects moving away from Gerrit, it seems like it's important that they get a temporary decent experience mixing GitHub and Bugzilla. The #1 missing thing is the auto-linking between GitHub PRs and Bugzilla: for Gerrit, when opening a Gerrit patch set creates the link to Bugzilla if some conventions are used; for GitHub, there is nothing like that and it's terribly missing.
Could the EF implement such a bot that monitors project PRs and look for typical Bugzilla pattern and add the link from BZ to GitHub.
I know this is extra work for a deprecated service, but I think having this would allow to move easily initiate the transition away from Gerrit to GitHub, and then facilitate the overall migration experience (once code is on GitHub, using other GitHub services like issues will become more natural and less contentious).(In reply to Mickael Istria from comment #8)
Could the EF implement such a bot that monitors project PRs and look for
typical Bugzilla pattern and add the link from BZ to GitHub.Concretely, this is identified as a requirement from Eclipse Project for a successful adoption of GitHub PRs instead of Gerrit.
Could the EF implement such a bot that monitors project PRs and look for typical Bugzilla pattern and add the link from BZ to GitHub.
The EF used to have such a bot running - it was turned off because it was over eager IIRC. Here is an example of it https://github.com/eclipse/lsp4j/pull/129#issuecomment-343982743
(In reply to Jonah Graham from comment #10)
The EF used to have such a bot running - it was turned off because it was
over eager IIRC. Here is an example of it
https://github.com/eclipse/lsp4j/pull/129#issuecomment-343982743OK thanks. And would it be possible to enable it for a given repo/project. Assuming a project start using a different pattern for GitHub issue (ie "Issue #123" instead of "Bug 123"), I think it would cover the concerns of Eclipse Project at least.
Hi Mickael,
Thanks for doing the recent migrations on Tycho and LSP4E to GH. I am following closely your work in this area as I will be taking on the same for other projects.For Gerrit have you had any success in preserving gerrit information after migration? For example, AFAICT Tycho's gerrit entries no longer exist now - see for example https://bugs.eclipse.org/bugs/show_bug.cgi?id=548726 and its linked gerrit which is no longer found. Does that info exist elsewhere now?
(In reply to Jonah Graham from comment #12)
For Gerrit have you had any success in preserving gerrit information after
migration? For example, AFAICT Tycho's gerrit entries no longer exist now -
see for example https://bugs.eclipse.org/bugs/show_bug.cgi?id=548726 and its
linked gerrit which is no longer found. Does that info exist elsewhere now?I don't know if it was kept for Tycho, and no-one wondered before you as far as I know ;)
For m2e, as the Gerrit backlog was more important, we asked for the Git repo to be made read-only for some time; so the data was still here, in Gerrit, as usual; but Gerrit couldn't be used for anything else other that browsing history. That can be a good 1st iteration.
When the Foundation decides to shutdown Gerrit, if history of Gerrit patches is important, a strategy to backup data will be necessary. Similar discussions are taking place for Bugzilla on the other bug.I have the impression that it's hard for projects to easily consider the move
to GitHub because of some missing functionality regarding issues.
So in order to better encourage projects moving away from Gerrit...Having to start using GitHub/PRs for code reviews of some other projects is the worse thing that happened to me work-wise in recent years. It is a daily cause of frustrations. I would never consider intentionally moving from Gerrit to GitHub. Maybe I'm just not hip to what the masses are using now. I don't even have an Instagram ;) I am really, really not looking forward to this shutdown.
The JGit and EGit projects would like to continue to use Gerrit. There are many reasons, see the discussion in [1] and [2].
Reasons:\
- we prefer the commit-centric review style Gerrit implements over branch-centric PRs in github or MRs in gitlab\
- we think reviewing contributions in Gerrit is more efficient that in github or gitlab\
- we don't want to lose all the invaluable review comments we collected during the last 12 years developing in Gerrit.\
- Gerrit runs on JGit and we should eat our own dog food\
- for a thorough comparison of code review workflows see [3] written by a mercurial maintainer
Both GerritForge (Luca Milanesio) and Google (Han-Wen Nienhuys) offered to host a managed gerrit instance for Eclipse projects that want to continue using Gerrit.
The solution GerritHub [4] offered by GerritForge uses a combination of Gerrit and Github in a way that the repository is available on both but code reviews happen in Gerrit. This way there should be no extra effort to maintain another backup solution. It uses Github authentication.
Both these solutions are deployed geo-distributed in multiple datacenters.
This means availability should be great and disaster recovery is included.
Push performance is typically a bit slower due to the overhead of coordinating across datacenters.There is some migration effort to move repositories to another gerrit instance. GerritForge has tooling to help with that.
If the maintenance effort for the ECA plugin is the problem I am volunteering to help with its maintenance.
[1] https://www.eclipse.org/lists/jgit-dev/msg04095.html\ [2] https://www.eclipse.org/lists/egit-dev/msg04805.html\ [3] https://gregoryszorc.com/blog/2020/01/07/problems-with-pull-requests-and-how-to-fix-them/\ [4] http://gerrithub.io/
marked this issue as related to #678
We are requesting your input on this proposed deprecation/migration plan for Gerrit and Bugzilla:
Hey, I would like to know... is there any chance to have a gerrit wrapper for github. Honestly, the productivity boost we get from real reviews vs prs is hard to measure.
https://hub.packtpub.com/using-gerrit-github/ something like that?
I am the author of the chapter https://hub.packtpub.com/using-gerrit-github/ (@makhouzam hope you liked the reading) and it refers to GerritHub.io.
The way it works is described by the short video https://www.youtube.com/watch?v=jeWTvDad6VM.
In a nutshell, GitHub becomes a "mirror" of the code review activity that happens on GerritHub.io:
- All changes uploaded to GerritHub are mirrored to GitHub
- All PRs created on GitHub can be imported to GerritHub and reviewed as Gerrit changes
- All CI activity can continue on GitHub
Many popular open-source projects adopted this workflow. At the last Gerrit User Summit in 2021, the Cue project did a short lightning talk about how they do it and is published in the video https://www.youtube.com/watch?v=2B2PZTZlPJg.
One last but important advantage, we could keep all the history of reviews of the changes in Gerrit even after the Eclipse's owned Gerrit shutdown. That is possible because all changes would be mirrored to GitHub anymore and GerritHub.io could understand and index them, rendering them in the Gerrit UI.
In case of questions or clarifications on the option above, please let me know :-) I am also happy to have a short session on how it works and helping migrating to it.
Luca Milanesio
Generally speaking as long as:
- The 'canonical' repo is a either Gitlab/Github
- The 'wrapper' follows the EDP requirements for commit restrictions(valid ECA or Committer permissions etc.)
- The project has(will have?) a way to 'manage' this extra service in a meritocratic way
Then I don't think it really matters which of these you use(at least to Webmaster).
We'll certainly take you up on your offer to provide support for the Gerrit ECA plugin, since at some point we will stop maintaining it.
@wbeaton, am I out of line here?
I believe that we can regard this as an "External Resource" as described in the handbook.
We'll certainly take you up on your offer to provide support for the Gerrit ECA plugin, since at some point we will stop maintaining it.
@mward... This implies a commitment to keeping Gerrit itself running. I don't think that's what you intend.
I'm not sure that it implies that we'll keep Gerrit running, only that someone else(thanks @msohn, for graciously volunteering) will need to keep the ECA checker Gerrit plugin maintained(since presumably these external Gerrit instances support plugins) once we stop maintaining it, which will happen in step with the shutdown of our Gerrit instance.
Now if the projects using these services want to enforce the EDP in some other way, I leave that to you(@wbeaton, EDP wizard) and them(E/Jgit for certain).
Sounds like this may become the path forward for us die-hard Gerrit fans.
AFAIK GerritHub runs Gerrit plugins so I assume it should be possible to deploy the ECA plugin. I can take over maintaining the ECA plugin, already contributed patches in the past.
I will talk to Luca from GerritForge to come up with a plan.
The preparation plan to support Eclipse projects to use Gerrit on GerritForge on the GerritForge side is here https://kanbanflow.com/board/1d5iBMR
mentioned in issue #2600 (closed)
mentioned in issue #674 (closed)
mentioned in issue #3137 (closed)
I filed #3137 (closed) to discuss details of migrating JGit and EGit repos to GerritHub/GitHub.
Just to provide an update, we've moved ~69% percent of projects to either this Gitlab instance or Github, so we are on schedule to finish the transitions by/in Q4 2024.
Once these transitions are finished we'll schedule and execute some service brownouts on our way to the final service shutdown in May 2025.
If your project hasn't already moved, or been contacted to schedule a transition, there's no need to wait for us, feel free to file an issue requesting the transition and we'll do our best to work with your schedule.
Edited by Matt Ward@mward thanks for the update. One question: what would happen to the review history after the shutdown? Would the review data be available in read-only somewhere for people to use as a reference?
What we can do is to mirror them to GerritHub.io, so that when the Eclipse foundation https://git.eclipse.org/r will be down, you could configure a simple rewrite rule to send that traffic down to GerritHub.io and all existing links to existing code-review data could still be online and the associated historical discussions on the code in a readable format.
Edited by Luca Milanesio
