Gitlab repositories still reflected on projects page, cannot be updated accurately

added Bug label

I am thinking we should hide the deprecated Gitlab Repositories field and replace it with a link from the new Gitlab namespace field.

/cc @wbeaton

assigned to @epoirier

added priorityhigh severitynormal labels

@mward @jmazanek4ep As a reminder, we created a new Gitlab Namespace field. When provisioning a project on Gitlab, please use this new field instead of the old one. We plan on deleting that old field in the future.

You will notice that we added a deprecated note here:

Note that the GitLab repositories field is now deprecated. Please refer to "Gitlab project groups" and "Gitlab Excluded sub-groups" to add GitLab repositories to this project.

If the field is redundant or irrelevant, then we should remove it (I'll leave it to you to decide if remove actually means "hide"). The likelihood that the contents of this old field will become out-of-date is, AFAICT, high, given that project leads can create repositories in GitLab and will likely have no motivation to update the PMI to reflect the information there.

But first, this information is in the wrong place. The contributors should be on the "Who's Involved" Page (#93 (closed)). The GitLab-related fields should be on the "Developer Resources" page.

Ideally, the Developer Resources page should display all of the repositories (i.e., get the list from GitLab and display it here). At very least, there needs to be some clickable link that navigates to the project's group.

Can the "GitLab Project Group" be edited by a committer, or is this exclusive to the IT Team/EMO?

What is the "Gitlab Excluded Sub-Groups"? Does this field need to be exposed to everybody? Can it be edited by committers?

If the field is redundant or irrelevant, then we should remove it (I'll leave it to you to decide if remove actually means "hide"). The likelihood that the contents of this old field will become out-of-date is, AFAICT, high, given that project leads can create repositories in GitLab and will likely have no motivation to update the PMI to reflect the information there.

Indeed. That was my motivation for proposing that we switch from tracking repos to namespaces: eclipsefdn&12 (closed)

Known issues

Error prone: PL can create new projects under their project group without notifying our webmasters. In this situation, the ECA will not enforce the ECA validation since it’s unable to determine that these new repos are owned by an Eclipse project.

This started to be an issue when we decided to set Maintainer permission to PL by default: eclipsefdn/helpdesk#17 (comment 256)

This was not reported in the risk assessment because of the asterisk under Create project in group (Default project creation role can be changed at the instance level and The group level.).

When we first rolled out the sync script for Gitlab, my team recommended that we set the default role for PL to be triage. However, it was decided that it would be best to give PL more power to reduce the admin overhead.

By default, Gitlab allows developers to create projects in groups. However, we changed the default creation role to set at the maintainer role. The risk assessment assumed that developers already had that ability when in reality, that was not true.

I want to delete this data once I am sure it's no longer needed. The information is redundant now. I am keeping it there for now in case we find an issue that would require us to revert our changes but my plan is to remove it.

What is the "Gitlab Excluded Sub-Groups"? Does this field need to be exposed to everybody? Can it be edited by committers?

The excluded sub-group was created for the supporting 3rd party that the ECA validator should be ignored: eclipsefdn&12 (closed)

This is also being discussed via: eclipsefdn/helpdesk#1525 (comment 877261)

Can the "GitLab Project Group" be edited by a committer, or is this exclusive to the IT Team/EMO?

Currently exclusive to IT/EMO but we can change that if you think the project should be managing this.

Ideally, the Developer Resources page should display all of the repositories (i.e., get the list from GitLab and display it here). At very least, there needs to be some clickable link that navigates to the project's group.

With #103 (comment 877082), I was proposing that we include a link to the Gitlab group. That would be the simplest solution. However, we could use Gitlab API to display a link for each repo.

@epoirier I think a good start here would be to include a link to the GitLab namespace of a project (If they are using it).

In order to gather commit statistics that we use for KPIs, displaying company logos, etc., I need the projects API to provide me a list of Git repositories.

Right now, I'm looking at the source_repo and gitlab_repos fields to get this information.

e.g.,

$ wget https://projects.eclipse.org/json/project/technology.dash -O -
{
   "projects" : {
      "technology.dash" : {
         ...
         "github_repos" : [
            {
               "attributes" : [],
               "original_title" : null,
               "original_url" : "https://github.com/eclipse/dash-licenses",
               "title" : null,
               "url" : "https://github.com/eclipse/dash-licenses"
            },
            {
               "attributes" : [],
               "original_title" : null,
               "original_url" : "https://github.com/eclipse/dash-website",
               "title" : null,
               "url" : "https://github.com/eclipse/dash-website"
            }
         ],
         "gitlab_repos" : [
            {
               "attributes" : [],
               "original_title" : null,
               "original_url" : "https://gitlab.eclipse.org/eclipse/dash/org.eclipse.dash.handbook",
               "title" : null,
               "url" : "https://gitlab.eclipse.org/eclipse/dash/org.eclipse.dash.handbook"
            }
         ],
         ...
         "source_repo" : [
            {
               "name" : "dash-licenses",
               "path" : "https://github.com/eclipse/dash-licenses",
               "type" : "github",
               "url" : "https://github.com/eclipse/dash-licenses"
            },
            {
               "name" : "dash-website",
               "path" : "https://github.com/eclipse/dash-website",
               "type" : "github",
               "url" : "https://github.com/eclipse/dash-website"
            }
         ],
...

I've included the github_repos field to acknowledge that it exists. We don't actually use it to identify repositories (it's data is duplicated in the source_repo field).

We will need a means of the getting the URLs for each of the repositories. It doesn't necessarily need to be in the API results, I just need to be able to find them one way or another. Is there a GitLab API for this?

There is! Here is the example for dash: https://gitlab.eclipse.org/api/v4/groups/eclipse%2Ftechnology%2Fdash/projects

So the format is https://gitlab.eclipse.org/api/v4/groups/{escaped_gitlab_namespace}/projects where esacaped_gitlab_namespace is the gitlab.project_group value from the projects API, that has been URL encoded. For most of the projects, that's just going to be replacing / with %2F. It isn't purely just the URLs, but the URL for the projects should be in the results for sure

Might be easier for @wbeaton to fetch that data using our API instead of https://projects.eclipse.org/json/project/technology.dash:

https://projects.eclipse.org/api/projects/technology.dash.json

We share gitlab (namespace now), github and gerrit urls.

Might be easier for @wbeaton to fetch that data using our API

If this is the preferred API, then I'll update the scripts.

AFAICT, though, it doesn't give me an actual list of GitLab repositories. All it provides is the gitlab/project_group.

But... if I grab the project_group of eclipse/technology/dash and convert that to eclipse%2Ftechnology%2Fdash and splice that into the GitLab API call suggested by @malowe, I should get what I need.

Is access to these APIs throttled? At what point to requests get rejected?

That would be a question for @droy , @fgurr , or potentially @mward . There is an API limit baked into Gitlab, but I don't know what it is, or if there is a way to allow list certain IP ranges to make it easier for you. Unfortunately, the folks that I know would have the answer are on vacation, and I'm not sure if Matt has looked at this part of GL ever.

@wbeaton THE rate limit is 3600 unauthenticated web requests per hour per IP and 7200 authenticated web requests per hour per user.

Is the Gitlab namespace field up-to-date and accurate for all projects that use GitLab?

I went through and updated those by hand to point at the proper repo locations just before we enabled the upgraded script. I used Gitlab as my source of truth to look them up, so they should all be up to date. Our current sync code doesn't even use the gitlab_repos field currently, just the excluded groups and project group field under gitlab.

closed

reopened

@cguindon @wbeaton, I created a MR to update the path of the Gitlab links on the Developer page:

https://gitlab.eclipse.org/eclipsefdn/it/webdev/drupal/pmi/-/merge_requests/20

Here is an example of the changed link on staging:

https://projects-staging.eclipse.org/projects/technology.dash/developer

Let me know if there's anything else.

Thanks.

I am thinking we need to do something closer to how we are displaying links for GH.

(Gitlab font-awesome icon*) Project repositories are hosted on Eclipse Gitlab

Browse Repositories

*https://fontawesome.com/v4/icon/gitlab

I created a quick example of what I am thinking:

@wbeaton what do you think?

Also, I am not sure if we are allowed to refer to our GitLab instance as Eclipse Gitlab. It might be best to say gitlab.eclipse.org.

@wbeaton I was thinking we should also include a note about excluded repositories but waiting to see how eclipsefdn/helpdesk#1525 (closed) will get resolved.

It's likely a trademark violation to refer to it as "Eclipse GitLab". Trademark management is all about avoiding confusion so something like "The Eclipse Foundation's GitLab instance" is probably okay.

@wbeaton what do you think?

LGTM.

To the best of my knowledge, the vendors write the name in camel case: "GitLab". We should try to be consistent. Correct me if I'm wrong...

Ok this is what i'm going to use then. Thanks!

I updated staging with the suggestions.

https://projects-staging.eclipse.org/projects/technology.dash/developer

Let me know if there's anything else.

Thanks.

Like in my example, I would pluralize the text. It's very likely that there is more than 1 repo. Singular makes sense for GH since we include each individual repos:

eclipse/dash - Project repositories are hosted on The Eclipse Foundation's GitLab instance.
 Browse Repositories

I updated the text:

https://projects-staging.eclipse.org/projects/technology.dash/developer

This looks good to me! Note that we should be removing the GitHub repos from Dash on production. I will do so once your changes are live.

Sounds good!

I created the following MR:

https://gitlab.eclipse.org/eclipsefdn/it/webdev/drupal/pmi/-/merge_requests/20

added statewip label

added statedone label and removed statewip label

The changes are now on production.

https://projects.eclipse.org/projects/technology.dash/developer

Let me know if there's anything else.

Thanks.

closed

mentioned in issue #112 (closed)

Gitlab repositories still reflected on projects page, cannot be updated accurately

Designs

Child items ...

Activity

Known issues