Consider supporting arbitrary nesting of groups

For consideration...

As I understand it, GitLab supports arbitrarily nested groups.

Our standard practice is allocate resources "flat", independent of project structure. My observation is that with GitLub, "flat" means that we're creating a group for each project under the "eclipse" group, e.g., "/eclipse/dash", and all repositories associated with the corresponding project are created in that group.

We adopted the "flat" notion quite some time ago to mitigate the amount of work and negative effects of having to move resources following project restructuring (e.g., when projects move from one TLP to another).

That GitLab provides support for arbitrary nesting of groups, sets up automatic redirects for moved repositories, and otherwise provides what I assume are reasonably good management tools, presents an opportunity to reconsider the policy.

I'm thinking that a default position is to have a distinct group for each project to mitigate the risk of confusion regarding which project owns which repository (and by extension which developer has commit rights on which repository), but allow that project groups can themselves be arbitrarily organized into groups.

From the @webmaster workload perspective, I believe that we'll have to manage expectations that the webmaster will not be able to continuously tweak and move groups around.

Would allowing arbitrary nesting introduce significant work in the provisioning process? Are there scenarios that are particularly onerous?

For one, at very least, keeping track of arbitrary nesting and what goes where will require some additional thought/management/tracking.

@cguindon are permissions assigned at the group level or the repository level? Does the script that manages committer and project leads care about the group nesting?

@cguindon are permissions assigned at the group level or the repository level? Does the script that manages committer and project leads care about the group nesting?

At the group level. For example, Eclipse Titan: https://gitlab.eclipse.org/eclipse/titan

Committers were added to the https://gitlab.eclipse.org/eclipse/titan project which grants them access to all the sub-projects within that space!

The sync script is responsible for creating this nested structure.

So that confirms the need to have a specific group for each project. I like that requirement to help keep the relationship between projects and potential multiple repositories clear.

Do the scripts care about nesting? e.g., if we were to create a repository, /eclipse/stuff/dash/stuff.git, would the script create permission groups for the "/eclipse/stuff/dash/" group?

Do permissions trickle up/down to the "/eclipse/stuff" group?

@wbeaton can you give an example of how "project groups" could be re-arranged? IIUC, you are referring to a structure similar to the top-level projects and their (sub-)projects, e.g.

• group "tools"
  • subgroup "acute"
  • subgroup "ajdt"
  • subgroup "cdt"
    • repository "cdt"
    • repository "cdt-vscode"
    • [...]
• group "technology"
  • subgroup "cbi"
  • [...]

I anticipate that there may be some top level projects that might want this.

The immediate example I have are two Technology subprojects, AsciiDoc Language and Eclipse Austin that work on different aspects of the same project.

I should note that I started this discussion with an expectation that nesting groups has some value greater than simple organization, e.g., combined analytics, issues, boards, etc.

The main issue I have is that the current situation is now inconsistent. For asciidoc-lang, the group is asciidoc. That gave me the impression we were putting projects under a group for the TLP. Therefore, I expected it to be /asciidoc/asciidoc-lang and /asciidoc/austen (pending approval of the AsciiDoc TLP). If, on the other hand, we are making one group per project (which seems excessive, but if it's a limitation of the tech, okay), then it should have been /asciidoc-lang/asciidoc-lang.

I really don't see any reason why projects can't be created under a group for the TLP*. I think too much concern is being given to fine-grained permissions. To a certain degree, there needs to be some trust (within the TLP) or else you end up over-engineering the solution.

* Obviously, the "technology" TLP would need to be some sort of exception.

Do the scripts care about nesting? e.g., if we were to create a repository, /eclipse/stuff/dash/stuff.git, would the script create permission groups for the "/eclipse/stuff/dash/" group?

By design, The PMI won't allow an admin/webmaster to add a repo outside of https://gitlab.eclipse.org/eclipse/:shortProjectId/[:repo-name]

The sync script takes actions based on data from the API on projects.eclipse.org.

We will need to make some changes to the sync script and PMI if we want to change this convention.

mentioned in issue #43

The immediate example I have are two Technology subprojects, AsciiDoc Language and Eclipse Austin that work on different aspects of the same project.

While I can see certain (management) benefits from the project's perspective, I think arbitrary grouping/nesting can lead to a lot of confusion. Especially if it's different from the structure in PMI. E.g. if a user tries to find the Eclipse Austin source code in GitLab, but can't find it (without using a search box) since it's hidden in a subgroup, I think the price is too high.

Do permissions trickle up/down to the "/eclipse/stuff" group?

"A user’s permissions in a subgroup cannot be lower than in any of its ancestor groups. Therefore, you cannot reduce a user’s permissions in a subgroup with respect to its ancestor groups."

From https://docs.gitlab.com/ee/user/group/subgroups/index.html

So permissions should only be given on the leaf-node projects and the sync script needs to be aware of parent sub-groups.

E.g. if a user tries to find the Eclipse Austin source code in GitLab, but can't find it (without using a search box) since it's hidden in a subgroup, I think the price is too high.

At least in this case, the intention is to provide a grouping to make it easier to build a brand and shared presence around the technology and for the community to find related projects.

At least in this case, the intention is to provide a grouping to make it easier to build a brand and shared presence around the technology and for the community to find related projects.

To Grouping! The Cause Of, And Solution To, All Of Life's Problems https://www.youtube.com/watch?v=SXyrYMxa-VI

Although all projects appear nested under the Eclipse top-level group, we're still maintaining the 1-group-per-Eclipse project. I'm not sure if we want nested groups unless it's something we can automate/self-serve from the get-go.

mentioned in issue #47 (closed)

If we change the nesting of our project on GitLab, we need to change it for all projects or we need to start tracking these nesting relations in the PMI.

The sync script will need to know which TLP should be nesting projects.

Right now the nesting is /eclipse/project/repo.git. Permissions are granted on the group level, so our script configures /eclipse/project with the permissions for the committers.

Is it possible to keep putting permissions on the leaf directory, and just skip over the extra levels? That is, if we add an extra layer of nesting, say /eclipse/tlp/project/repo.git, can we just set the committer permissions on /eclipse/tlp/project? And, in the absence of additional information, just leave with either an empty group or some default? (would that actually work?).

If we then create a repository for the TLP a /eclipse/tlp/repo.git, could we then create a group with the committers for at /eclipse/tlp?

Does GitLab have a notion of a hierarchy of groups? e.g., is there some "risk" that committers would have permissions on ?

Right now the nesting is /eclipse/project/repo.git. Permissions are granted on the group level, so our script configures /eclipse/project with the permissions for the committers.

Correct.

Is it possible to keep putting permissions on the leaf directory, and just skip over the extra levels? That is, if we add an extra layer of nesting, say /eclipse/tlp/project/repo.git, can we just set the committer permissions on /eclipse/tlp/project? And, in the absence of additional information, just leave with either an empty group or some default? (would that actually work?).

We need a group per project to keep things efficient.

We don't want to manage users at the project level since that will require us to add/remove the same committers on every repo/project. The sync script currently takes 1 hour to run and we only have a few projects on Gitlab.

If we then create a repository for the TLP a /eclipse/tlp/repo.git, could we then create a group with the committers for at /eclipse/tlp?

We can't create a group within a group. In your example, the group is TPL, and users added to that group would inherit the same role for any sub-group and sub-project created under the TPL group.

Something to note: It's possible to add existing groups to another group or project.

Does GitLab have a notion of a hierarchy of groups? e.g., is there some "risk" that committers would have permissions on ?

Any users added to the TLP project would retain the same role for any sub-project or sub-group.

• /eclipse/asciidoc - Webmaster (Owner)
• /eclipse/asciidoc/asciidoc-lang - Webmaster inherit the role of owner because of eclipse/asciidoc. It's not possible to remove it without removing webmaster from /eclipse/asciidoc

To conclude, I have a good idea how we could implement TLP nesting. However, I am not convinced yet that this is worth fixing. If we do, it will probably be an exception built-in in our sync script that will always require staff to get involved to make this happen.

I don't want our sync script to start moving repos around. This could break the builds job of a Project since the clone URL will be changed by moving the project to another location.

I think each TLP will have different requirements. For example, if we decide to enable this to the Technology TLP, we will need to pause the sync script, enable TLP nesting in our sync script for Technology TLP, migrate any existing project to the new nesting structure. Once everything is in place, we would need to restart the sync script service.

The process should be simpler if we decide to enable this to a TLP that doesn't have any projects on Eclipse GitLab yet.

mentioned in issue eclipsefdn/emo-team/emo#65 (closed)

mentioned in issue eclipsefdn/emo-team/emo#95 (closed)

@cguindon @wbeaton We'd be very happy @ Eclipse Oniro Working Group to be the guinea pigs for trying out nesting of groups. The flat recommended structure is honestly very limiting to us. We have a fairly nice structure over at https://booting.oniroproject.org/distro with all components nicely separated in the Components sub-group.

Trying to flatten this isn't helping developers in understanding the code structure either. And a project having multiple git repos with different maintainers is more natural in an OS project.

Can we try this out?

cc: @agherzan

@idlethread,

The Asciidoc WG is also requesting this feature. I would like to take some time to better understand your expectations before I commit to making any changes to our structure.

I am looking at https://booting.oniroproject.org/distro, is it safe for me to assume that each group in that list will map to a new Eclipse Project?

I am basically trying to visualize what you are trying to achieve with Eclipse projects in mind.

From what I can see in the PMI, we currently have 2 projects for Oniro:

• TLP Eclipse Oniro: https://projects.eclipse.org/projects/oniro
• Eclipse Oniro Core Platform: https://projects.eclipse.org/projects/oniro.oniro-core

To clarify my question, will we have an Eclipse Blueprint project or are you looking to create a similar structure from https://booting.oniroproject.org/distro under the Eclipse Oniro Core Platform project?

@cguindon

I am looking at https://booting.oniroproject.org/distro, is it safe for me to assume that each group in that list will map to a new Eclipse Project?

Not all the groups. All the meta- prefixed ones will end up under oe below. We're thinking something like this:

oniro-core
 |
 |
 |---- oniro
 |---- docs
 |---- infrastructure/*
 |      |
 |      |
 |      .
 |      .
 |---- oe/*
 |      |
 |      |---- 3rdparty/*
 |      .
 |      .
 |---- components/*
 |      |
 |      |---- 3rdparty/*
 |      .
 |      .

To clarify my question, will we have an Eclipse Blueprint project or are you looking to create a similar structure from https://booting.oniroproject.org/distro under the Eclipse Oniro Core Platform project?

Yes, there will be a separate Eclipse Oniro Blueprint project because the sources will be maintained independently of Oniro Core.

But oniro-core allowing sub-groups and the above structure will make lives much easier for the following reasons:

1. Easier understanding of the project layout when browsing through the web interface
2. We have tens of 3rd party components that we need to integrate into the core OS. They don't belong in a separate project. Having them all at one root level will make things hard.
3. Prefixes don't work for us because certain prefixes have special meanings in Yocto tooling e.g. meta-. Builds will break if we change that.
4. Adding subgroups will simplify this for CQ and ongoing IP reviews since forks can be stored together e.g. in 3rdparty/* above.
5. For now, we can start with a single root-level permissions (committers) for the entire project.
6. But eventually, we'd like the ability to have sub-group and repo-level permissions for committers. We don't want to initiate new committers with committer access to the entire high-level group.

mentioned in issue eclipsefdn/emo-team/emo#168 (closed)

But eventually, we'd like the ability to have sub-group and repo-level permissions for committers. We don't want to initiate new committers with committer access to the entire high-level group.

This is not supported by the Eclipse Foundation Development Process (EDP). According to the EDP, all committers have uniform access to all project resources. If something different is required, then an update to the EDP is required.