Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in
  • E eclipsefdn-vc-tools
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 20
    • Issues 20
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
    • Requirements
  • Merge requests 1
    • Merge requests 1
  • Deployments
    • Deployments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • Code review
    • Insights
    • Issue
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Commits
  • Issue Boards
Collapse sidebar
  • Eclipse FoundationEclipse Foundation
  • IT
  • Webdev
  • eclipsefdn-vc-tools
  • Merge requests
  • !208

Update Gitlab sync + import to be modern

  • Review changes

  • Download
  • Email patches
  • Plain diff
Merged Martin Lowe requested to merge malowe/eclipsefdn-github-sync:malowe/master/update-gl into master Apr 12, 2022
  • Overview 3
  • Commits 3
  • Changes 10

Updates to npm packages including base GL binding. Some API changes, but overall mostly the same. Updated scripts where necessary, tested the import and sync with the following results.

Sync:

❯ node src/GitlabSync.js --devMode=true --host=http://localhost --secretLocation=$PWD/secrets -V --trace-warnings --tls-min-v1.0 --unhandled-rejections=strict
2022-04-12T13:30:11 [main] DEBUG GitlabSync:sanitizeGroupName(pid = eclipse)
2022-04-12T13:30:11 [main] DEBUG GitlabSync:sanitizeGroupName(pid = gitlab-instance-a8f39b1c)
2022-04-12T13:30:11 [main] DEBUG GitlabSync:sanitizeGroupName(pid = backup-20220411)
2022-04-12T13:30:11 [main] DEBUG GitlabSync:sanitizeGroupName(pid = github-backup)
2022-04-12T13:30:11 [main] DEBUG GitlabSync:sanitizeGroupName(pid = openhw-group)
2022-04-12T13:30:11 [main] INFO Starting sync
2022-04-12T13:30:11 [main] DEBUG GitlabSync:getGroup(name = Eclipse, path = eclipse, parent = undefined, visibility = public)
2022-04-12T13:30:11 [main] DEBUG GitlabSync:sanitizeGroupName(pid = eclipse)
2022-04-12T13:30:11 [main] INFO Processing 'spider.pig'
2022-04-12T13:30:11 [main] DEBUG GitlabSync:getGroup(name = Spider pig does what a spider pig does, path = spider.pig, parent = [object Object], visibility = public)
2022-04-12T13:30:11 [main] DEBUG GitlabSync:sanitizeGroupName(pid = spider.pig)
2022-04-12T13:30:11 [main] VERBOSE Creating new group with name 'Spider pig does what a spider pig does'
2022-04-12T13:30:11 [main] DEBUG Creating group with options: {"project_creation_level":"maintainer","visibility":"public","request_access_enabled":false,"parent_id":53}
2022-04-12T13:30:11 [main] DEBUG GitlabSync:sanitizeGroupName(pid = spider.pig)
2022-04-12T13:30:12 [main] DEBUG Created group: {"id":54,"web_url":"http://gitlab.dev.docker/groups/eclipse/spider.pig","name":"Spider pig does what a spider pig does","path":"spider.pig","description":"","visibility":"public","share_with_group_lock":false,"require_two_factor_authentication":false,"two_factor_grace_period":48,"project_creation_level":"maintainer","auto_devops_enabled":null,"subgroup_creation_level":"maintainer","emails_disabled":null,"mentions_disabled":null,"lfs_enabled":true,"default_branch_protection":2,"avatar_url":null,"request_access_enabled":false,"full_name":"Eclipse / Spider pig does what a spider pig does","full_path":"eclipse/spider.pig","created_at":"2022-04-12T17:30:11.846Z","parent_id":53,"shared_with_groups":[],"projects":[],"shared_projects":[]}
2022-04-12T13:30:12 [main] DEBUG GitlabSync:sanitizeGroupName(pid = spider.pig)
2022-04-12T13:30:12 [main] DEBUG GitlabSync:getUserList(project = {"project_id":"spider.pig","short_project_id":"spider.pig","name":"Spider pig does what a spider pig does","summary":"Can he fly? No, hes a pig. Look out, here comes the spider pig","logo":"","tags":["simpsons","doh","spider pig"],"github_repos":[{"url":"https://github.com/eclipsefdn-webdev/spider-pig"}],"gitlab_repos":[{"url":"https://gitlab.eclipse.org/eclipsefdn/webdev/gitlab-testing","org":"webdev","repo":"gitlab-testing"}],"contributors":[],"committers":[{"username":"malowe","url":"https://api.eclipse.org/account/profile/malowe"},{"username":"epoirier","url":"https://api.eclipse.org/account/profile/epoirier"}],"project_leads":[{"username":"malowe","url":"https://api.eclipse.org/account/profile/malowe"},{"username":"cguindon","url":"https://api.eclipse.org/account/profile/cguindon"}],"working_groups":[{"name":"Cloud Development Tools","id":"cloud-development-tools"}],"spec_project_working_group":[],"state":"Regular","pp_repos":["gitlab-testing"],"pp_orgs":["webdev"]})
2022-04-12T13:30:12 [main] DEBUG GitlabSync:getUser(uname = malowe, url = https://api.eclipse.org/account/profile/malowe)
2022-04-12T13:30:13 [main] VERBOSE Creating new user with name 'malowe'
2022-04-12T13:30:13 [main] DEBUG Creating user with options: {"username":"malowe","password":"redacted","force_random_password":true,"name":"Martin Lowe","email":"martin.lowe@eclipse-foundation.org","extern_uid":"473605","provider":"oauth2_generic","skip_confirmation":true}
2022-04-12T13:30:13 [main] ERROR Response code 409 (Conflict)
2022-04-12T13:30:13 [main] VERBOSE Could not retrieve user for UID 'malowe', skipping
2022-04-12T13:30:13 [main] DEBUG GitlabSync:getUser(uname = epoirier, url = https://api.eclipse.org/account/profile/epoirier)
2022-04-12T13:30:14 [main] VERBOSE Creating new user with name 'epoirier'
2022-04-12T13:30:14 [main] DEBUG Creating user with options: {"username":"epoirier","password":"redacted","force_random_password":true,"name":"Eric Poirier","email":"eric.poirier@eclipse-foundation.org","extern_uid":"2573","provider":"oauth2_generic","skip_confirmation":true}
2022-04-12T13:30:15 [main] DEBUG GitlabSync:addUserToGroup(user = [object Object], group = [object Object], perms = 30)
2022-04-12T13:30:15 [main] DEBUG GitlabSync:getGroupMembers(group = [object Object])
2022-04-12T13:30:15 [main] INFO Adding 'epoirier' to 'Spider pig does what a spider pig does' group
2022-04-12T13:30:15 [main] DEBUG GitlabSync:getUser(uname = cguindon, url = https://api.eclipse.org/account/profile/cguindon)
2022-04-12T13:30:16 [main] VERBOSE Creating new user with name 'cguindon'
2022-04-12T13:30:16 [main] DEBUG Creating user with options: {"username":"cguindon","password":"redacted","force_random_password":true,"name":"Christopher Guindon","email":"chris.guindon@eclipse-foundation.org","extern_uid":"9","provider":"oauth2_generic","skip_confirmation":true}
2022-04-12T13:30:17 [main] DEBUG GitlabSync:addUserToGroup(user = [object Object], group = [object Object], perms = 40)
2022-04-12T13:30:17 [main] DEBUG GitlabSync:getGroupMembers(group = [object Object])
2022-04-12T13:30:17 [main] INFO Adding 'cguindon' to 'Spider pig does what a spider pig does' group
2022-04-12T13:30:17 [main] DEBUG GitlabSync:removeAdditionalUsers(expectedUsers = [object Object], group = [object Object], projectID = spider.pig)
2022-04-12T13:30:17 [main] DEBUG GitlabSync:getGroupMembers(group = [object Object])
2022-04-12T13:30:17 [main] VERBOSE Checking user 'root' access to group 'Spider pig does what a spider pig does'
2022-04-12T13:30:17 [main] VERBOSE Checking user 'epoirier' access to group 'Spider pig does what a spider pig does'
2022-04-12T13:30:17 [main] VERBOSE Checking user 'cguindon' access to group 'Spider pig does what a spider pig does'
2022-04-12T13:30:17 [main] DEBUG Processing repo 'https://gitlab.eclipse.org/eclipsefdn/webdev/gitlab-testing'
2022-04-12T13:30:17 [main] DEBUG GitlabSync:getProject(name = gitlab-testing, parent = [object Object])
2022-04-12T13:30:17 [main] VERBOSE Creating new project with name 'gitlab-testing'
2022-04-12T13:30:17 [main] DEBUG Creating project with options: {"path":"gitlab-testing","visibility":"public","namespace_id":54}
2022-04-12T13:30:18 [main] DEBUG Created project: {"id":45,"description":null,"name":"gitlab-testing","name_with_namespace":"Eclipse / Spider pig does what a spider pig does / gitlab-testing","path":"gitlab-testing","path_with_namespace":"eclipse/spider.pig/gitlab-testing","created_at":"2022-04-12T17:30:17.724Z","default_branch":"main","tag_list":[],"topics":[],"ssh_url_to_repo":"git@gitlab.dev.docker:eclipse/spider.pig/gitlab-testing.git","http_url_to_repo":"http://gitlab.dev.docker/eclipse/spider.pig/gitlab-testing.git","web_url":"http://gitlab.dev.docker/eclipse/spider.pig/gitlab-testing","readme_url":null,"avatar_url":null,"forks_count":0,"star_count":0,"last_activity_at":"2022-04-12T17:30:17.724Z","namespace":{"id":54,"name":"Spider pig does what a spider pig does","path":"spider.pig","kind":"group","full_path":"eclipse/spider.pig","parent_id":53,"avatar_url":null,"web_url":"http://gitlab.dev.docker/groups/eclipse/spider.pig"},"_links":{"self":"http://gitlab.dev.docker/api/v4/projects/45","issues":"http://gitlab.dev.docker/api/v4/projects/45/issues","merge_requests":"http://gitlab.dev.docker/api/v4/projects/45/merge_requests","repo_branches":"http://gitlab.dev.docker/api/v4/projects/45/repository/branches","labels":"http://gitlab.dev.docker/api/v4/projects/45/labels","events":"http://gitlab.dev.docker/api/v4/projects/45/events","members":"http://gitlab.dev.docker/api/v4/projects/45/members"},"packages_enabled":true,"empty_repo":true,"archived":false,"visibility":"public","resolve_outdated_diff_discussions":false,"container_expiration_policy":{"cadence":"1d","enabled":false,"keep_n":10,"older_than":"90d","name_regex":".*","name_regex_keep":null,"next_run_at":"2022-04-13T17:30:17.802Z"},"issues_enabled":true,"merge_requests_enabled":true,"wiki_enabled":true,"jobs_enabled":true,"snippets_enabled":true,"container_registry_enabled":true,"service_desk_enabled":false,"service_desk_address":null,"can_create_merge_request_in":true,"issues_access_level":"enabled","repository_access_level":"enabled","merge_requests_access_level":"enabled","forking_access_level":"enabled","wiki_access_level":"enabled","builds_access_level":"enabled","snippets_access_level":"enabled","pages_access_level":"enabled","operations_access_level":"enabled","analytics_access_level":"enabled","container_registry_access_level":"enabled","security_and_compliance_access_level":"private","emails_disabled":null,"shared_runners_enabled":true,"lfs_enabled":true,"creator_id":1,"import_status":"none","import_error":null,"open_issues_count":0,"runners_token":"GR1348941sKFsyiGDjENDZiE5gfzd","ci_default_git_depth":20,"ci_forward_deployment_enabled":true,"ci_job_token_scope_enabled":false,"public_jobs":true,"build_git_strategy":"fetch","build_timeout":3600,"auto_cancel_pending_pipelines":"enabled","build_coverage_regex":null,"ci_config_path":null,"shared_with_groups":[],"only_allow_merge_if_pipeline_succeeds":false,"allow_merge_on_skipped_pipeline":null,"restrict_user_defined_variables":false,"request_access_enabled":true,"only_allow_merge_if_all_discussions_are_resolved":false,"remove_source_branch_after_merge":true,"printing_merge_request_link_enabled":true,"merge_method":"merge","squash_option":"default_off","suggestion_commit_message":null,"merge_commit_template":null,"squash_commit_template":null,"auto_devops_enabled":true,"auto_devops_deploy_strategy":"continuous","autoclose_referenced_issues":true,"repository_storage":"default","keep_latest_artifact":true,"runner_token_expiration_interval":null}
2022-04-12T13:30:18 [main] DEBUG GitlabSync:cleanUpProjectUsers(project = 45)

Import:

❯ npm run import-backup -- --verbose=true --host="http://localhost" --target=9 --organization=openhwgroup --secretLocation=/localdev/eclipsefdn-webdev-toolkit/projects-js/eclipsefdn-github-sync/secrets/import

> eclipsefdn-github-sync@1.0.0-BETA import-backup
> node src/auto_backup/Import.js "--verbose=true" "--host=http://localhost" "--target=9" "--organization=openhwgroup" "--secretLocation=/localdev/eclipsefdn-webdev-toolkit/projects-js/eclipsefdn-github-sync/secrets/import"

Importing with config given: {"target":9,"source":"openhwgroup","count":1,"host":"http://localhost","flags":{}}
createBackupGroup(backup-20220412, 9)
2022-04-12T13:47:52 [GitWrapper] DEBUG prefetchRepos(org = openhwgroup)
2022-04-12T13:47:52 [GitWrapper] VERBOSE Starting prefetch for repos in org=openhwgroup
2022-04-12T13:47:54 [GitWrapper] VERBOSE Finished prefetch for org=openhwgroup, got 42 repos
2022-04-12T13:47:54 [GitWrapper] DEBUG getDiscoveredRepos(org = openhwgroup)
importRepository(52028445,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(118652371,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(153259725,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(188901413,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(197242563,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(206841504,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(224885491,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(224904153,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(238013358,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(257965812,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(261766165,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(262127046,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(271641901,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(274802727,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(287073308,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(287224035,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(293831036,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(310614811,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(312659750,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(332727847,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(337370762,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(337477204,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(339651955,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(339652021,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(341497701,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(341502497,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(344473829,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(346717591,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(352510400,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(364263635,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(371703637,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(375289726,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(379653247,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(413464491,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(443370833,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(450620243,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(452067146,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(453158731,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(457069453,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(466032244,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(468498579,openhw-group/github-backup/backup-20220412,http://localhost)
importRepository(472438946,openhw-group/github-backup/backup-20220412,http://localhost)
Found 2 groups (with a keep count of 1)
Processing group backup-20220412 for potential pruning
Skipping group backup-20220412 for pruning
Processing group backup-20220411 for potential pruning
Successes:
[
  "Successfully imported repository 'openhwgroup.org' to openhw group backup / github backup / backup-20220412 / openhwgroup.org(54)",
  "Successfully imported repository 'core-v-freertos' to openhw group backup / github backup / backup-20220412 / core-v-freertos(48)",
  "Successfully imported repository 'pulp_soc' to openhw group backup / github backup / backup-20220412 / pulp_soc(46)",
  "Successfully imported repository 'aws-codebuild-run-build' to openhw group backup / github backup / backup-20220412 / aws-codebuild-run-build(55)",
  "Successfully imported repository 'core-v-docs' to openhw group backup / github backup / backup-20220412 / core-v-docs(52)",
  "Successfully imported repository 'aws-infra' to openhw group backup / github backup / backup-20220412 / aws-infra(47)",
  "Successfully imported repository 'cva5' to openhw group backup / github backup / backup-20220412 / cva5(58)",
  "Successfully imported repository 'cva6' to openhw group backup / github backup / backup-20220412 / cva6(56)",
  "Successfully imported repository 'cv32e40s' to openhw group backup / github backup / backup-20220412 / cv32e40s(65)",
  "Successfully imported repository 'corev-elf-psabi-doc' to openhw group backup / github backup / backup-20220412 / corev-elf-psabi-doc(53)",
  "Successfully imported repository 'embdebug-target-core-v' to openhw group backup / github backup / backup-20220412 / embdebug-target-core-v(64)",
  "Successfully imported repository 'riscv-ovpsim-corev' to openhw group backup / github backup / backup-20220412 / riscv-ovpsim-corev(61)",
  "Successfully imported repository 'core-v-verif' to openhw group backup / github backup / backup-20220412 / core-v-verif(57)",
  "Successfully imported repository 'corev-gcc' to openhw group backup / github backup / backup-20220412 / corev-gcc(63)",
  "Successfully imported repository 'OpenHW-Admin-Tools' to openhw group backup / github backup / backup-20220412 / OpenHW-Admin-Tools(74)",
  "Successfully imported repository 'core-v-ide-cdt' to openhw group backup / github backup / backup-20220412 / core-v-ide-cdt(60)",
  "Successfully imported repository 'corev-binutils-gdb' to openhw group backup / github backup / backup-20220412 / corev-binutils-gdb(59)",
  "Successfully imported repository 'apb_interrupt_cntrl' to openhw group backup / github backup / backup-20220412 / apb_interrupt_cntrl(69)",
  "Successfully imported repository 'linux' to openhw group backup / github backup / backup-20220412 / linux(50)",
  "Successfully imported repository 'timer_unit' to openhw group backup / github backup / backup-20220412 / timer_unit(71)",
  "Successfully imported repository 'core-v-sw' to openhw group backup / github backup / backup-20220412 / core-v-sw(49)",
  "Successfully imported repository 'riscv_vm' to openhw group backup / github backup / backup-20220412 / riscv_vm(62)",
  "Successfully imported repository 'cve2' to openhw group backup / github backup / backup-20220412 / cve2(77)",
  "Successfully imported repository 'cva6-sdk' to openhw group backup / github backup / backup-20220412 / cva6-sdk(70)",
  "Successfully imported repository 'core-v-mcu-devkit' to openhw group backup / github backup / backup-20220412 / core-v-mcu-devkit(66)",
  "Successfully imported repository 'corev-llvm-project' to openhw group backup / github backup / backup-20220412 / corev-llvm-project(68)",
  "Successfully imported repository 'osdforum.org' to openhw group backup / github backup / backup-20220412 / osdforum.org(67)",
  "Successfully imported repository 'cv32e41p' to openhw group backup / github backup / backup-20220412 / cv32e41p(51)",
  "Successfully imported repository 'core-v-mcu' to openhw group backup / github backup / backup-20220412 / core-v-mcu(73)",
  "Successfully imported repository 'u-boot' to openhw group backup / github backup / backup-20220412 / u-boot(72)",
  "Successfully imported repository 'cv32e40p' to openhw group backup / github backup / backup-20220412 / cv32e40p(76)",
  "Successfully imported repository 'core-v-cores' to openhw group backup / github backup / backup-20220412 / core-v-cores(75)",
  "Successfully imported repository 'experimental-repo' to openhw group backup / github backup / backup-20220412 / experimental-repo(81)",
  "Successfully imported repository 'infra' to openhw group backup / github backup / backup-20220412 / infra(79)",
  "Successfully imported repository 'marketing' to openhw group backup / github backup / backup-20220412 / marketing(80)",
  "Successfully imported repository 'core-v-freertos-kernel' to openhw group backup / github backup / backup-20220412 / core-v-freertos-kernel(78)",
  "Successfully imported repository 'core-v-trace' to openhw group backup / github backup / backup-20220412 / core-v-trace(84)",
  "Successfully imported repository 'core-v-mcu-cli-test' to openhw group backup / github backup / backup-20220412 / core-v-mcu-cli-test(82)",
  "Successfully imported repository 'force-riscv' to openhw group backup / github backup / backup-20220412 / force-riscv(83)",
  "Successfully imported repository 'cv32e40x' to openhw group backup / github backup / backup-20220412 / cv32e40x(86)",
  "Successfully imported repository 'core-v-xif' to openhw group backup / github backup / backup-20220412 / core-v-xif(85)",
  "Successfully imported repository 'downloads.openhwgroup.org' to openhw group backup / github backup / backup-20220412 / downloads.openhwgroup.org(87)"
]
Errors:
[]
Run complete!
Edited Apr 12, 2022 by Martin Lowe
Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: malowe/master/update-gl

Copyright © Eclipse Foundation, Inc. All Rights Reserved.     Privacy Policy | Terms of Use | Copyright Agent