Commit 7ffefeae authored by Martin Lowe's avatar Martin Lowe 🇨🇦
Browse files

Merge branch 'malowe/master/gl-restruc' into 'master'

Rebuild GL sync in TS, add arb. group nesting

See merge request !210
parents a6f33d72 43cee809
Pipeline #5246 failed with stage
in 0 seconds
......@@ -17,6 +17,21 @@
"parserOptions": {
"requireConfigFile": false
},
"overrides": [
{
"files": ["src/**/*.ts"],
"parser": "@typescript-eslint/parser",
"plugins": ["@typescript-eslint"],
"extends": [
"eslint:recommended",
"plugin:@typescript-eslint/eslint-recommended",
"plugin:@typescript-eslint/recommended"
],
"rules": {
"@typescript-eslint/no-non-null-assertion": ["off"]
}
}
],
"rules": {
"max-len": [2, 140, 2],
"eqeqeq": 2,
......
......@@ -12,9 +12,14 @@ The Eclipse Foundation toolkit for maintaining permissions across multiple versi
- [Gitlab Permissions mapping](#gitlab-permissions-mapping)
- [Usage](#usage)
- [Manual run parameters](#manual-run-parameters)
- [Github](#github)
- [Gitlab](#gitlab)
- [Github params](#github-params)
- [Gitlab params](#gitlab-params)
- [Running the toolset for development](#running-the-toolset-for-development)
- [Running for Github](#running-for-github)
- [Running for Gitlab](#running-for-gitlab)
- [Gitlab Secrets](#gitlab-secrets)
- [Gitlab script run command](#gitlab-script-run-command)
- [Creating a tagged release](#creating-a-tagged-release)
- [Maintainers](#maintainers)
- [Trademarks](#trademarks)
- [Copyright and license](#copyright-and-license)
......@@ -22,7 +27,7 @@ The Eclipse Foundation toolkit for maintaining permissions across multiple versi
## Gaining Access to a project
To be granted access to a repository through the Eclipse Sync scripts (Gitlab or Github), a user must first either be nominated as a committer or a project lead within the [PMI(projects management infrastructure)](https://wiki.eclipse.org/Project_Management_Infrastructure), or be added as a contributor by an active committer or project lead. Depending on the role granted within the PMI, different access rights will be granted through the sync scripts. Should a user be promoted or retired within a project, the new permission sets should be active within a few hours of finalization.
To be granted access to a repository through the Eclipse Sync scripts (Gitlab or Github), a user must first either be nominated as a committer or a project lead within the [PMI(projects management infrastructure)](https://wiki.eclipse.org/Project_Management_Infrastructure) or be added as a contributor by an active committer or project lead. Depending on the role granted within the PMI, different access rights will be granted through the sync scripts. Should a user be promoted or retired within a project, the new permission sets should be active within a few hours of finalization.
Bot access within repositories is possible, but managed by a manual process and tracked by a [publicly available API](https://api.eclipse.org/bots) rather than through the sync script. How these bot permissions are typically interpreted varies by platform, and more info for each is available in the Github and Gitlab Sync sections within this readme.
......@@ -32,9 +37,9 @@ _[^ Back to top](#eclipsefdn-github-sync)_
## Github Sync
Within Github, there is a mixed strategy for management of projects within the space. Projects that are started while under the Eclipse umbrella or from a project that was incepted within the Eclipse ecosystem are by default created under the central Eclipse organization. On request, projects can be migrated to a separate organization that is still managed by the EclipseWebmaster account. Repositories or projects born from organizations or groups that have joined Eclipse Foundation post inception are usually managed under organizations managed by the EclipseWebmaster. While there are cases where projects can cross organizational bounds, it is uncommon (and covered by the sync script).
Within Github, there is a mixed strategy for the management of projects within the space. Projects that are started while under the Eclipse umbrella or from a project that was incepted within the Eclipse ecosystem are by default created under the central Eclipse organization. On request, projects can be migrated to a separate organization that is still managed by the EclipseWebmaster account. Repositories or projects born from organizations or groups that have joined Eclipse Foundation post inception are usually managed under organizations managed by the EclipseWebmaster. While there are cases where projects can cross organizational bounds, it is uncommon (and covered by the sync script).
Permissions to projects are managed through hidden teams that are then granted access to each repository for the given project within the current organization. For each organization that a project has repositories in, a set of contributor, committer, and project-lead teams will be created to give access to those repositories. Each of these teams will have the same set of users as defined within the project management interface on projects.eclipse.org. For users to be properly added by this mechanism, they must set their Github Handle within their Eclipse Account on accounts.eclipse.org.
Permissions to projects are managed through hidden teams that are then granted access to each repository for the given project within the current organization. For each organization that a project has repositories in, a set of the contributor, committer, and project-lead teams will be created to give access to those repositories. Each of these teams will have the same set of users as defined within the project management interface on projects.eclipse.org. For users to be properly added by this mechanism, they must set their Github Handle within their Eclipse Account on accounts.eclipse.org.
In regards to bot access, this is typically granted at the repository level, but can also be added at the team level if more broad access is needed. These permissions, while not removed by the script are currently managed manually by the Eclipse Foundation. If there are issues regarding bot access, new or existing, an issue should be created within our [bug-tracking system](https://bugs.eclipse.org) rather than within this project.
......@@ -67,24 +72,33 @@ Information on Github permissions is available in the [documentation for organiz
_[^ Back to top](#eclipsefdn-github-sync)_
## Gitlab Sync
In Gitlab, a nested group strategy was chosen to manage access to both groups and projects. This gives greater control over inherited permissions without having to manage teams across multiple base groups. For each Open Source group with repositories managed by the Eclipse Foundation (such as Eclipse Foundation and the OpenHWGroup), a base group will exist to encapsulate all projects for that group. Within each of these groups, each active project will have a subgroup (such as Eclipse Dash and Eclipse Marketplace Client) that will manage permissions for all repositories active within the Gitlab instance.
In Gitlab, a nested group strategy was chosen to manage access to both groups and projects. This gives greater control over inherited permissions without having to manage teams across multiple base groups. For each Open Source group with repositories managed by the Eclipse Foundation (such as Eclipse Foundation and the OpenHWGroup), a base group will exist to encapsulate all projects for that group. Within each of these groups, projects will have the option of choosing between 2 options when migrating over to their targeted projects' namespace:
1. Using the short project ID as the path of their group (i.e asciidoc-lang, dash). While the path of the project is determined programmatically, a display name may be chosen by the Project Leads of the project and a request made through the Eclipse Foundation Gitlab Helpdesk or requested at the time of the creation of the group.
2. Nesting the group mentioned in the first option within a group for the TLP (top-level project) of the given project. (i.e. asciidoc/asciidoc-lang, technology/dash). The name of the TLP group is determined by the Project Leads of the TLP, and similarly to the first option, a request may be made via the Eclipse Foundation Gitlab Helpdesk.
Within the namespace of your project, Project Leads may create further groups to better encapsulate their projects. This provides much better management of the subsequent projects through tools such as shared issue queues, kanban boards, milestones, and custom labels. Adding additional members outside of bot users to these subgroups is disallowed and will be cleaned regularly to ensure proper permissions are maintained.
In regards to bot access, this can be granted at either the subgroup or project (repository) level depending on the needs of the project. These permissions, while not removed by the script are currently managed manually by the Eclipse Foundation. If there are issues regarding bot access, new or existing, an issue should be created within our [bug-tracking system](https://bugs.eclipse.org) rather than within this project.
Below is an example of a few projects within the Eclipse Gitlab instance and their structure:
Below is an example of a few projects within the Eclipse Gitlab instance and their possible structure:
```
Eclipse/ (group)
├─ Eclipse Dash/ (group)
│ ├─ dash-gitlab-testing (project)
│ ├─ org.eclipse.dash.handbook (project)
├─ Eclipse Marketplace Client/ (group)
│ ├─ MPC Client (project)
│ ├─ org.eclipse.epp.mpc (project)
├─ Technology/ (group)
│ ├─ Eclipse Marketplace Client/ (group)
│ │ ├─ MPC Client (project)
│ │ ├─ org.eclipse.epp.mpc (project)
Eclipse Foundation/ (group)
├─ webdev/ (group)
│ ├─ eclipsefdn-api-common (project)
```
```
One thing to note with the nested groups model. If project groups are nested under the TLP for said projects, the committers and Project Leads of the TLP will be functionally the same role for all projects nested beneath them. This is a side effect of inherited permissions, and while [not optimal, there is no IP exposure, and isn't a clear breaking of the EDP](https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/42#note_828611).
### Gitlab Permissions mapping
......@@ -103,7 +117,7 @@ _[^ Back to top](#eclipsefdn-github-sync)_
The following parameters can be used when running the sync scripts manually.
#### Github
#### Github params
| Parameter name | Required | Accepts | Default | Description |
|----------------|:--------:|---------|---------|-------------|
......@@ -116,7 +130,7 @@ The following parameters can be used when running the sync scripts manually.
|-h, --help | x | N/A (flag) | N/A | Prints the help text for the script parameters. |
|-V, --verbose | x | boolean flag | `false` | Sets the script to run in verbose mode (ranges from 1-4 for more verbose logging). |
#### Gitlab
#### Gitlab params
| Parameter name | Required | Accepts | Default | Description |
|----------------|:--------:|---------|---------|-------------|
......@@ -128,31 +142,58 @@ The following parameters can be used when running the sync scripts manually.
|-h, --help | x | N/A (flag) | N/A | Prints the help text for the script parameters. |
|-p, --provider | x | string | `oauth2_generic` | The OAuth provider name set in GitLab for the Eclipse Accounts binding. |
|-P, --project | x |string | N/A | The project ID (e.g. technology.dash) of the project that should be updated (at the exclusion of all other projects) |
|-r, --rootGroup | x |string | `eclipse` | The path of the root project to use when syncing PMI permissions. This is used to help in scoping retrievals and caching. |
|-V, --verbose | x | boolean flag | `false` | Sets the script to run in verbose mode (ranges from 1-4 for more verbose logging). |
### Running the toolset for development
#### Running for Github
By default, the script is run in docker containers to emulate the production environment (Openshift). This sync tool can be run in standard and verbose mode. The difference between the modes is that in verbose all log messages are printed to the STDOUT of the container.
Before running, an `api-token` file should be created that contains the GitHub API key to be used for connectivity. This should be created in a `secret` folder in the root of the project (this has been excluded from Git commits so there is less danger of pushing live keys to a public repository).
Before running, an `api-token` file should be created that contains the GitHub API key to be used for connectivity. This should be created in a `secret` folder at the root of the project (this has been excluded from Git commits so there is less danger of pushing live keys to a public repository).
```
docker build -f Dockerfile -t ef/gh-test .
docker run -i --rm -v <fullpath to current project folder>/secrets:/run/secrets --env DRYRUN=true ef/gh-test
```
_[^ Back to top](#eclipsefdn-github-sync)_
#### Running for Gitlab
The default state for the script is to run against production Gitlab, which isn't feasible for development/testing. To better enable testing, a local instance of Gitlab should be started. This can be started using docker-compose: `docker-compose up -d gitlab`. This starts and binds a Gitlab instance to ports 22, 80, and 443 to best emulate a real instance of Gitlab. This can be changed in the docker-compose if needed.
Once a local instance is started, retrieve the password set for the root admin user using the following command: `sudo docker exec -it gitlab grep 'Password:' /etc/gitlab/initial_root_password`. This will print the root user password as plain text to the console for use. Once retrieved, you can log in and update the password to something more memorable or easy to remember or set the password into a password manager.
When you are logged in, you will need to create a base group to test the sync on. Using `eclipse` requires no additional config and is recommended. Once this group is created, any groups needed for projects should be added as they are currently not created if missing. For devMode, this group will be the group `spider.pig` under the `eclipse` group.
##### Gitlab Secrets
This step assumes the previous Gitlab setup to be completed, or you have access to the admin account running on the target instance. Open the [user preferences](http://localhost/-/profile/preferences) page and navigate to the `Access tokens` section. Once here, create an access token with the `api` permission set. This is needed to update or remove user permissions from groups and projects within the instance. Save this under your secrets directory within the file `access-token`.
Additionally, an eclipse-oauth-config should be created when running the GitLab sync script. This file will define how connections to the Eclipse OAuth server should be handled. If this is missing, the GitLab sync script will fail. The file should be in JSON format, with the following being an example of format:
Additionally, an `eclipse-oauth-config` should be created when running the GitLab sync script. This file will define how connections to the Eclipse OAuth server should be handled. If this is missing, the GitLab sync script will fail. The file should be in JSON format, with the following being an example of the format:
```
{"oauth":{"timeout":3600, "client_id":"<client id>","client_secret":"<client secret>","endpoint":"https://accounts.eclipse.org","scope":"eclipsefdn_view_all_profiles","redirect":"http://localhost"}}
```
Should you not have access to these values and you are on the EF Webdev team, reach out on slack and they will be provided. Otherwise, these credentials are internal and not distributed to the general public.
##### Gitlab script run command
Included below is an example command that can be used within Unix environments given the default setup instructions are followed:
```
docker build -f Dockerfile -t ef/gh-test .
docker run -i --rm -v <fullpath to current project folder>/secrets:/run/secrets --env DRYRUN=true ef/gh-test
npm run lab-sync -- --devMode --verbose --secretLocation=$PWD/secrets -H "http://localhost"
```
_[^ Back to top](#eclipsefdn-github-sync)_
_[^ Back to top](#eclipsefdn-github-sync)_
## Creating a tagged release
Once a release candidate has been identified for production, tests should be run locally using the master branch to ensure that the integration of patches are stable as the pull requests that had been validated. Additionally, `npm ci && npm run test` should be run to ensure that the current package lock file is stable, passes tests, and contains no live vulnerabilities from upstream packages. If there is issues with this, patches should be created to address these issues before the release goes live.
Once a release candidate has been identified for production, tests should be run locally using the master branch to ensure that the integration of patches are stable as the pull requests that had been validated. Additionally, `npm ci && npm run test` should be run to ensure that the current package lock file is stable, passes tests, and contains no live vulnerabilities from upstream packages. If there are issues with this, patches should be created to address these issues before the release goes live.
When determining the version bump [semantic versioning spec](https://semver.org/) should be used, which also forms the base of how [NPM versioning works](https://docs.npmjs.com/about-semantic-versioning). The new version should be applied through a PR on the master branch. Once this is done a new tag can be created on the master branch with the format `v#.#.#`, where the numbers should match the version stated in the package.json file. While there is no anticipated need to create an NPM package release, it is good practice and can help keep code clean.
......@@ -171,7 +212,7 @@ Once the master branch tag is created it can then be merged into the `production
## Copyright and license
Copyright 2019 the [Eclipse Foundation, Inc.](https://www.eclipse.org) and the [eclipsefdn-github-sync authors](https://github.com/eclipsefdn/eclipsefdn-github-sync/graphs/contributors). Code released under the [Eclipse Public License Version 2.0 (EPL-2.0)](https://github.com/eclipsefdn/eclipsefdn-github-sync/blob/master/LICENSE).
Copyright 2019, 2022 the [Eclipse Foundation, Inc.](https://www.eclipse.org) and the [eclipsefdn-github-sync authors](https://github.com/eclipsefdn/eclipsefdn-github-sync/graphs/contributors). Code released under the [Eclipse Public License Version 2.0 (EPL-2.0)](https://github.com/eclipsefdn/eclipsefdn-github-sync/blob/master/LICENSE).
_[^ Back to top](#eclipsefdn-github-sync)_
......@@ -10,6 +10,25 @@ services:
- DRYRUN=true
secrets:
- api-token
gitlab:
container_name: gitlab
image: 'gitlab/gitlab-ee:14.7.7-ee.0'
restart: always
environment:
VIRTUAL_HOST: "gitlab.dev.docker"
VIRTUAL_PORT: 443
VIRTUAL_PROTO: https
CERT_NAME: dev.docker
GITLAB_OMNIBUS_CONFIG: "external_url 'http://localhost/';"
shm_size: '256m'
ports:
- 443:443
- 80:80
- 22:22
volumes:
- '/localdocker/gitlab/config:/etc/gitlab'
- '/localdocker/gitlab/logs:/var/log/gitlab'
- '/localdocker/gitlab/data:/var/opt/gitlab'
secrets:
api-token:
......
This source diff could not be displayed because it is too large. You can view the blob instead.
......@@ -10,8 +10,8 @@
"scripts": {
"start": "node src/Sync.js -c",
"pretest": "eslint --ignore-path .gitignore .",
"test": "mocha --timeout 60000",
"lab-sync": "node src/GitlabSync.js -c",
"test": "mocha --timeout 60000 && mocha --config=./test/ts/.mocharc.json --timeout 60000",
"lab-sync": "ts-node src/scripts/gl/GitlabSync.ts -c",
"import-backup": "node src/auto_backup/Import.js"
},
"license": "EPL-2.0",
......@@ -20,27 +20,38 @@
},
"homepage": "https://github.com/eclipsefdn/eclipsefdn-github-sync#readme",
"dependencies": {
"@gitbeaker/node": "35.6.0",
"@gitbeaker/core": "^35.6.0",
"@gitbeaker/node": "^35.6.0",
"@octokit/plugin-retry": "^3.0.3",
"@octokit/plugin-throttling": "^3.3.0",
"@octokit/rest": "^18.0.3",
"@types/node": "^17.0.32",
"@types/yargs": "^17.0.10",
"axios": "^0.21.4",
"flat-cache": "^2.0.1",
"nodemailer": "^6.5.0",
"openid-client": "^3.15.6",
"parse-link-header": "^2.0.0",
"simple-oauth2": "^4.1.0",
"ts-node": "^10.7.0",
"uuid": "^8.3.2",
"winston": "3.2.1",
"yargs": "^13.3.0"
},
"devDependencies": {
"@babel/eslint-parser": "^7.17.0",
"@types/mocha": "^9.1.1",
"@types/parse-link-header": "^2.0.0",
"@types/simple-oauth2": "^4.1.1",
"@types/uuid": "^8.3.4",
"@typescript-eslint/eslint-plugin": "^5.27.0",
"@typescript-eslint/parser": "^5.27.0",
"chai": "^4.2.0",
"eslint": "^7.5.0",
"eslint-config-strongloop": "^2.1.0",
"faker": "^5.5.3",
"mocha": "^7.0.1",
"sinon": "^10.0.0"
"sinon": "^10.0.0",
"typescript": "^4.6.4"
}
}
......@@ -183,7 +183,7 @@ module.exports = class EclipseAPI {
},
})
.then(result => result.data)
.catch(err => this.#logger.error(err));
.catch(err => this.#logger.error(`${err}`));
}
async eclipseBots() {
......@@ -192,7 +192,7 @@ module.exports = class EclipseAPI {
}
var botsRaw = await axios.get('https://api.eclipse.org/bots')
.then(result => result.data)
.catch(err => this.#logger.error(err));
.catch(err => this.#logger.error(`${err}`));
if (botsRaw === undefined || botsRaw.length <= 0) {
this.#logger.error('Could not retrieve bots from API');
process.exit(EXIT_ERROR_STATE);
......
......@@ -993,5 +993,5 @@ function logError(err, root) {
log.error(`${err.errors[i].message}`);
}
}
log.error(err);
log.error(`${err}`);
}
// set up yargs command line parsing
var argv = require('yargs')
.usage('Usage: $0 [options]')
.example('$0', '')
.option('d', {
alias: 'dryrun',
description: 'Runs script as dry run, not writing any changes to API',
boolean: true,
})
.option('D', {
alias: 'devMode',
description: 'Runs script in dev mode, which returns API data that does not impact production organizations/teams.',
boolean: true,
})
.option('V', {
alias: 'verbose',
description: 'Sets the script to run in verbose mode',
boolean: true,
})
.option('H', {
alias: 'host',
description: 'GitLab host base URL',
default: 'http://gitlab.eclipse.org/',
})
.option('p', {
alias: 'provider',
description: 'The OAuth provider name set in GitLab',
default: 'oauth2_generic',
})
.option('P', {
alias: 'project',
description: 'The short project ID of the target for the current sync run',
})
.option('s', {
alias: 'secretLocation',
description: 'The location of the access-token file containing an API access token',
})
.help('h')
.alias('h', 'help')
.version('0.1')
.alias('v', 'version')
.epilog('Copyright 2019 Eclipse Foundation inc.')
.argv;
const ADMIN_PERMISSIONS_LEVEL = 50;
const uuid = require('uuid');
const { SecretReader, getBaseConfig } = require('./SecretReader.js');
const { Gitlab } = require('@gitbeaker/node');
const EclipseAPI = require('./EclipseAPI.js');
const { getLogger } = require('./logger.js');
let logger = getLogger(argv.V ? 'debug' : 'info', 'main');
var api;
var eApi;
var bots;
var namedGroups = {};
var namedProjects = {};
var namedUsers = {};
var gMems = {};
_prepareSecret();
/**
* Retrieves secret API token from system, and then starts the script via _init
*
* @returns
*/
function _prepareSecret() {
// retrieve the secret API token
var accessToken, eclipseToken;
// retrieve the secret API file root if set
var settings = getBaseConfig();
if (argv.s !== undefined) {
settings.root = argv.s;
}
var reader = new SecretReader(settings);
var data = reader.readSecret('access-token');
if (data !== null) {
accessToken = data.trim();
// retrieve the Eclipse API token (needed for emails)
data = reader.readSecret('eclipse-oauth-config');
if (data !== null) {
eclipseToken = data.trim();
run(accessToken, eclipseToken);
} else {
logger.error('Could not find the Eclipse OAuth config, exiting');
}
} else {
logger.error('Could not find the GitLab access token, exiting');
}
}
async function run(secret, eclipseToken) {
api = new Gitlab({
host: argv.H,
token: secret,
});
eApi = new EclipseAPI(JSON.parse(eclipseToken));
eApi.testMode = argv.D;
// get raw project data and post process to add additional context
var data = await eApi.eclipseAPI();
data = eApi.postprocessEclipseData(data, 'gitlab_repos');
// get the bots for the projects
var rawBots = await eApi.eclipseBots();
bots = eApi.processBots(rawBots, 'gitlab.eclipse.org');
// get all current groups for the instance
var groups = await api.Groups.all();
var projects = await api.Projects.all();
var users = await api.Users.all();
// map the groups/projects/users to their name
for (var groupIdx in groups) {
namedGroups[sanitizeGroupName(groups[groupIdx].path)] = groups[groupIdx];
}
for (var projectIdx in projects) {
namedProjects[getCompositeProjectKey(projects[projectIdx].name, projects[projectIdx].namespace.id)] = projects[projectIdx];
}
for (var userIdx in users) {
namedUsers[users[userIdx].username] = users[userIdx];
}
// fetch org group from results, create if missing
logger.info('Starting sync');
var g = await getGroup('Eclipse', 'eclipse', undefined);
if (g === undefined) {
if (argv.d) {
logger.error('Unable to start sync of GitLab content. Base Eclipse group could not be found and dryrun is set');
} else {
logger.error('Unable to start sync of GitLab content. Base Eclipse group could not be created');
}
return;
}
for (projectIdx in data) {
var project = data[projectIdx];
if (argv.P !== undefined && project.short_project_id !== argv.P) {
logger.info(`Project target set ('${argv.P}'). Skipping non-matching project ID ${project.short_project_id}`);
continue;
}
logger.info(`Processing '${project.short_project_id}'`);
// fetch project group from results, create if missing
var projGroup = await getGroup(project.name, project.short_project_id, g);
if (projGroup === undefined) {
if (argv.d) {
logger.warn(`Unable to continue processing project with ID '${project.short_project_id}'.`
+ ' Group does not exist and dryrun has been set.');
} else {
logger.error(`Unable to continue processing project with ID '${project.short_project_id}'.`
+ ' Group does not exist and could not be created.');
}
continue;
}
// get the list of users to be added for current project
var userList = getUserList(project);
// for each user, get their gitlab user and add to the project group
var usernames = Object.keys(userList);
for (var usernameIdx in usernames) {
var uname = usernames[usernameIdx];
var user = await getUser(uname, userList[uname].url);
if (user === undefined) {
logger.verbose(`Could not retrieve user for UID '${uname}', skipping`);
continue;
}
await addUserToGroup(user, projGroup, userList[uname].access_level);
}
// remove users that don't match the expected users
await removeAdditionalUsers(userList, projGroup, project.short_project_id);
// for each of the repos in the Eclipse project, ensure there is a GL
// project
for (var repoIdx in project.gitlab_repos) {
var extRepo = project.gitlab_repos[repoIdx];
if (extRepo === undefined || extRepo.repo === undefined || extRepo.org === undefined) {
continue;
}
if (argv.V) {
logger.debug(`Processing repo '${extRepo.url}'`);
}
// retrieving current project
var p = await getProject(extRepo.repo, projGroup);
if (p !== undefined) {
await cleanUpProjectUsers(p, project.short_project_id);
}
}
}
}
async function removeAdditionalUsers(expectedUsers, group, projectID) {
if (argv.V) {
logger.debug(`GitlabSync:removeAdditionalUsers(expectedUsers = ${expectedUsers}, group = ${group}, projectID = ${projectID})`);
}
// get the current list of users for the group
var members = await getGroupMembers(group);
if (members === undefined) {
logger.warn(`Could not find any group members for ID ${group.id}'. Skipping user removal check`);
return;
}
// check that each of the users in the group match whats expected
var expectedUsernames = Object.keys(expectedUsers);
for (var memberIdx in members) {
var member = members[memberIdx];
// check access and ensure user isn't an owner
logger.verbose(`Checking user '${member.username}' access to group '${group.name}'`);
if (member.access_level !== ADMIN_PERMISSIONS_LEVEL && expectedUsernames.indexOf(member.username) === -1
&& !isBot(member.username, projectID)) {
if (argv.d) {
logger.info(`Dryrun flag active, would have removed user '${member.username}' from group '${group.name}'`);
continue;
}
logger.info(`Removing user '${member.username}' from group '${group.name}'`);
try {
await api.GroupMembers.remove(group.id, member.id);
} catch (err) {
if (argv.V) {
logger.error(err);
}
logger.warn(`Error while removing user '${member.username}' from group '${group.name}'`);
}
}
}
}
async function cleanUpProjectUsers(project, projectID) {
if (argv.V) {
logger.debug(`GitlabSync:cleanUpProjectUsers(project = ${project.id})`);
}
var projectMembers = await api.ProjectMembers.all(project.id, { includeInherited: false });
for (var idx in projectMembers) {
let member = projectMembers[idx];
// skip bot user or admin users
if (isBot(member.username, projectID) || member.access_level === ADMIN_PERMISSIONS_LEVEL) {
continue;
}
if (argv.d) {
logger.debug(`Dryrun flag active, would have removed user '${member.username}' from project '${project.name}'(${project.id})`);
continue;
}
logger.info(`Removing user '${member.username}' from project '${project.name}'(${project.id})`);
try {
await api.ProjectMembers.remove(project.id, member.id);
} catch (err) {
if (argv.V) {
logger.error(err);
}
logger.error(`Error while removing user '${member.username}' from project '${project.name}'(${project.id})`);
}
}
}
function isBot(uname, projectID) {
var botList = bots[projectID];
// check if the current user is in the current key-values list
return botList !== undefined && botList.indexOf(uname) !== -1;
}
/** API FUNCTIONS */
async function addUserToGroup(user, group, perms) {
if (argv.V) {
logger.debug(`GitlabSync:addUserToGroup(user = ${user}, group = ${group}, perms = ${perms})`);
}
// get the members for the current group
var members = await getGroupMembers(group);
if (members === undefined) {
logger.warn(`Could not find any references to group with ID ${group.id}`);
return;
}
// check if user is already present
for (var memberIdx in members) {
if (members[memberIdx].username === user.username) {
logger.verbose(`User '${user.username}' is already a member of ${group.name}`);
if (members[memberIdx].access_level !== perms) {
// skip if dryrun
if (argv.d) {
logger.info(`Dryrun flag active, would have updated user '${members[memberIdx].username}' in group '${group.name}'`);
return;
}
// modify user, catching errors
logger.info(`Fixing permission level for user '${user.username}' in group '${group.name}'`);
try {
var updatedMember = await api.GroupMembers.edit(group.id, user.id, perms);
// update inner array
members[memberIdx] = updatedMember;
gMems[group.id] = members;
} catch (err) {
if (argv.V) {
logger.error(err);
}
logger.warn(`Error while fixing permission level for user '${user.username}' in group '${group.name}'`);
return;
}
}