Build instructions
System requirements:
- python3.10 (mandatory): install using
apt install python3
- jsonnet-bundler (mandatory): install using
go install -a github.com/jsonnet-bundler/jsonnet-bundler/cmd/jb@v0.5.1
- bitwarden cli tool (optional): install using
snap install bw
- pass cli tool (optional): install using
apt install pass
Building Steps
- Create a virtual python environment and install necessary python dependencies using poetry:
$ make init
- Testing build
$ ./otterdog.sh -h
Setup
The general configuration for supported organizations and their corresponding credentials in order to access their GitHub settings has to be placed in a json file (default: otterdog.json, can be changed with the -c flag):
{
...
"organizations": [
{
"name": "<org name>",
"github_id": "<github org id>",
"credentials": {
"provider": "<bitwarden | pass>",
... // provider specific data
}
}
]
...
}
Credentials
Otterdog needs certain credentials to access information from an organization and its repositories on GitHub:
- username / password / 2FA seed
- API token
The login / username / 2FA seed are required to access the web interface of GitHub in order to retrieve certain settings that are not accessible via its rest / graphql API.
The GitHub api token needs to have the following scopes enabled:
- repo
- workflow
- admin:org
- admin:org_hook
The credentials can be stored in different providers (bitwarden, pass).
Bitwarden
When using bitwarden to store the credentials, you need to enter a valid item id as additional credential data:
{
...
"organizations": [
{
"name": "<org name>",
"github_id": "<github org id>",
"credentials": {
"provider": "bitwarden",
"item_id" : "<bitwarden item id>"
}
}
]
...
}
The item stored in bitwarden needs to contain the following information (a sample json output of such an item):
{
"object": "item",
"id": "<bitwarden item id>",
"name": "<item name>",
"fields": [
{
"name": "api_token_admin",
"value": "<github API token>"
}
],
"login": {
"username": "<github username>",
"password": "<github password>",
"totp": "<github TOTP text code>"
}
}
Mandatory items:
- Field with name "api_token_admin" and as value the GitHub token to access the organization
- login.username of a user that can access the organization with enabled 2FA
- login.password the password of that user
- login.totp the TOTP text code
Pass
When using pass to store the credentials, you need to enter fully qualified pass names to access the various required credential data:
{
...
"organizations": [
{
"name": "<org name>",
"github_id": "<github org id>",
"credentials": {
"provider": "pass",
"api_token": "<path/to/api_token>",
"username": "<path/to/username>",
"password": "<path/to/password>",
"2fa_seed": "<path/to/2fa_seed>"
}
}
]
...
}
In case your password storage dir is not located at the default location, you can
configurate that in the defaults
:
{
"defaults": {
"pass": {
"password_store_dir": "path/to/storage/dir"
}
}
}
Supported settings
Organization Settings
Field | Type | Description | Comment |
---|---|---|---|
name | string or null | Name of the organization | |
billing_email | string | Billing email | |
company | string or null | Company name | |
string or null | Main contact point of the organization | ||
twitter_username | string or null | Twitter username | |
location | string or null | Location | |
description | string or null | Description | |
blog | string or null | Url of the blog | |
plan | string | Billing plan of the organization | read-only |
has_organization_projects | boolean | If the organization can have projects | |
has_repository_projects | boolean | If the organization has repository projects | |
default_repository_permission | string | Default repository permissions | |
members_can_create_private_repositories | boolean | If members can create private repos | |
members_can_create_public_repositories | boolean | If members can create public repos | |
members_can_fork_private_repositories | boolean | If members can fork private repos | |
web_commit_signoff_required | boolean | If web commit signoff is required | |
members_can_create_pages | boolean | If members can create pages | |
members_can_create_public_pages | boolean | If members can create public pages | |
dependabot_alerts_enabled_for_new_repositories | boolean | If dependabot alerts are enabled for new repos | |
dependabot_security_updates_enabled_for_new_repositories | boolean | If dependabot security updates are enabled for new repos | |
dependency_graph_enabled_for_new_repositories | boolean | If dependency graph is enabled for new repos | |
members_can_change_repo_visibility | boolean | If members can change repo visibility | |
members_can_delete_repositories | boolean | If members can delete repos | |
members_can_delete_issues | boolean | If members can delete issues | |
readers_can_create_discussions | boolean or null | If readers can create discussions | |
members_can_create_teams | boolean | If members can create teams | |
two_factor_requirement | boolean | If two factor is required for all members | read-only |
team_discussions_allowed | boolean | If team discussions are allowed | |
default_branch_name | string | The default branch name for repos | |
packages_containers_public | boolean | If members can push public releases / containers | |
packages_containers_internal | boolean | If members can push private releases / containers | |
organization_projects_enabled | boolean | If members can create organization projects | |
members_can_change_project_visibility | boolean | If members can change visibility of organization projects | |
default_workflow_permissions | string | The default permissions granted to the GITHUB_TOKEN when running workflows in this organization | "read" or "write" |
security_managers | list[string] | List of teams that should act as security managers |
Webhooks
Field | Type | Description |
---|---|---|
active | boolean | If the webhook is active |
events | array of strings | List of events that trigger the webhook |
url | string | Url the webhook should access |
content_type | string | The content type the webhook shall use |
insecure_ssl | string | If the webhook uses insecure ssl connections, either "0" or "1" |
secret | string or null | The secret the webhook shall use if any |
The secret value can be resolved using a credential provider. The supported format is
<credential_provider>:<provider specific data>
:
- Bitwarden:
bitwarden:<bitwarden item id>@<custom_field_key>
- Pass:
pass:<path/to/secret>
Examples:
{
"secret": "bitwarden:118276ad-158c-4720-b68d-af8c00fe3481@webhook_secret"
}
{
"secret": "pass:myorg/mywebhook_secret"
}
Note: After executing an import
operation, the secret will be set to ******
as GitHub will only send redacted
secrets. You will need to update the definition file with the real secret values, either by entering the secret
value (not adivsed), or referencing it via a credential provider.
Repository Settings
Field | Type | Description | Comment |
---|---|---|---|
name | string | Name of the repository | |
description | string or null | Project description | |
homepage | string or null | Link to the homepage | |
private | boolean | If the project is private | |
has_issues | boolean | If the repo can have issues | |
has_projects | boolean | If the repo can have projects | |
has_wiki | boolean | If the repo has a wiki | |
is_template | boolean | If the repo is can be used as a template repository | |
template_repository | string or null | The template repository to use when creating the repo | read-only |
default_branch | string | Name of the default branch | |
allow_rebase_merge | boolean | If rebase merges are permitted | |
allow_merge_commit | boolean | If merge commits are permitted | |
allow_squash_merge | boolean | If squash merges are permitted | |
allow_auto_merge | boolean | If auto merges are permitted | |
delete_branch_on_merge | boolean | If branches shall automatically be deleted after a merge | |
allow_update_branch | boolean | If pull requests should suggest updates | |
squash_merge_commit_title | string | Can be PR_TITLE or COMMIT_OR_PR_TITLE for a default squash merge commit title | |
squash_merge_commit_message | string | Can be PR_BODY, COMMIT_MESSAGES, or BLANK for a default squash merge commit message | |
merge_commit_title | string | Can be PR_TITLE or MERGE_MESSAGE for a default merge commit title | |
merge_commit_message | string | Can be PR_BODY, PR_TITLE, or BLANK for a default merge commit message | |
archived | boolean | If the repo is archived | |
allow_forking | boolean | If the repo allows private forking | |
web_commit_signoff_required | boolean | If the repo requires web commit signoff | |
secret_scanning | string | If secret scanning is "enabled" or "disabled" | |
secret_scanning_push_protection | string | If secret scanning push protection is "enabled" or "disabled" | |
dependabot_alerts_enabled | boolean | If the repo has dependabot alerts enabled | |
branch_protection_rules | array | branch protection rules of the repo, see section below for details |
Branch Protection Rules
Field | Type | Description |
---|---|---|
pattern | string | Pattern to match branches |
allowsDeletions | boolean | If the branch can be deleted |
allowsForcePushes | boolean | If branch allows force pushes |
bypassForcePushAllowances | list[actor] | List of actors able to force push for this branch protection rule |
bypassPullRequestAllowances | list[actor] | List of actors able to bypass PRs for this branch protection rule |
dismissesStaleReviews | boolean | Dismiss approved reviews automatically when a new commit is pushed |
isAdminEnforced | boolean | Enforces status checks for admin |
lockAllowsFetchAndMerge | boolean | If the repo has dependabot alerts enabled |
lockBranch | boolean | If the branch is read-only |
pushRestrictions | list[actor] | List of actors that are permitted to push to the branch |
requireLastPushApproval | boolean | Whether the most recent push must be approved by someone other than the person who pushed it |
requiredApprovingReviewCount | integer or null | TBD |
requiresApprovingReviews | boolean | TBD |
requiresCodeOwnerReviews | boolean | If reviews from code owners are required to update matching branches |
requiresCommitSignatures | boolean | If commits are required to be signed |
requiresLinearHistory | boolean | If merge commits are prohibited from being pushed to this branch |
requiresStatusChecks | boolean | TBD |
requiresStrictStatusChecks | boolean | TBD |
restrictsReviewDismissals | boolean | If only allowed actors can dismiss reviews on pull requests |
reviewDismissalAllowances | list[actor] | List of actors that are permitted to dismiss reviews on pull requests |
requiredStatusChecks | list[statuscheck] | List of status checks that must pass before branches can be merged |
Note:
-
allowsForcePushes
: if this set toTrue
, any actor with push permission can force push to the branch -
bypassForcePushAllowances
: if the actor list is non-empty butallowsForcePushes
is set to True, a validation error will be issued -
pushRestrictions
: the contents of the actor list controls whether push restriction is enabled or disabled, i.e. an empty list disables it -
reviewDismissalAllowances
: if the actor list is non-empty butrestrictsReviewDismissals
is set to False, a validation error will be issued
Actor Format
- User:
/login
, e.g./netomi
- Team:
<organization>/<team-slug>
, e.g.OtterdogTest/committers
Status Check Format
- GitHub Action Status:
<status-name>
, e.g.Run CI
- Status from an app:
<app-slug>:<status-name>
, e.g.eclipsefdn:eca
- Status from any source:
any:<status-name
, e.g.any:Run CI
Usage
Run the import operation to retrieve the current live configuration for an organization:
$ otterdog.sh import <organization>
The created configuration file for the organization can be found at <data-directory>/orgs/<organization>.jsonnet
Run the plan operation to highlight differences between the live configuration and the written configuration:
$ otterdog.sh plan <organization>
Run apply operation to reflect the written configuration on github itself:
$ otterdog.sh apply <organization>
Container Runtime (Linux/MacOS)
Requirements
- An otterdog.json already in your current directory
- (Recommended) a directory orgs
Bulding Container Image
- Creating a container local image
make container_build
Building a Development Container Image
- Creating a container development local image
make container_build_dev
Running otterdog in a container
Using Bitwarden client
- Firstly you need to login and unlock your Bitwarden session by executing the command below
bw login
bw unlock
- As result, you will get a token session. Please follow example below to make available in your terminal (Linux/MacOS)
export BW_SESSION="ffdsajklloremipsumfxs000f000r0loremipsum"
Using pass client
- Pass needs to be already installed as well as configured with all data needed in otterdog.json by executing
pass
in your profile
Activating Otterdog Container Runtime
- Activate otterdog environment will create an alias
otterdog
source scripts/bin/active-otterdog
- Checking otterdog alias
otterdog -h
- Deactivating otterdog environment
deactivate-otterdog
otterdog container Arguments
- An table arguments
Argument | Description |
---|---|
-g | .gnupg directory PATH by default $HOME/.gnupg if it is not provided |
-o | Output ORGS directory path by default $PWD/orgs if it is not provided |
-c | otterdog json file path by default $PWD/otterdog.json if it is not provided |
-p | .password-store directory PATH by default $HOME/.password-store if it is not provided |
-h | Help about container arguments and otterdog |
- Please find below an example to show configuration
otterdog -c $PWD/otterdog.json -g $HOME/.gnupg -o $PWD/orgs apply -f
Usage Otterdog Container Runtime
- Please follow the section Usage
- Please bear in mind that all command need to drop .sh
Activating Development Otterdog Container Runtime
- Activating developemnt otterdog environment will run eclipse/otterdog:dev. Thus it can be used container shell. Please find below an example
export OTTERDOG_DEV=1; source scripts/bin/active-otterdog
otterdog /bin/bash
- To activate development otterdog environment with arbument
export OTTERDOG_DEV=1; source scripts/bin/active-otterdog
otterdog -c $PWD/otterdog.json -g $HOME/.gnupg -o $PWD/orgs /bin/bash
- Checking otterdog environment
/app/otterdog.sh -h
- Deactivating development otterdog runtime type
exit
thendeactivate-otterdog
Usage Development Otterdog Container Runtime
- Please follow the section Usage
Cleaning container environment
- Please use the macro below
make container_clean
Known issues
GraphQL
- branch protection rule property
blocksCreations
can not be updated via an update or create mutation, always seem to befalse
- repo setting
secret_scanning_push_protection
seems to be only available for GitHub Enterprise billing, omitting for now