Build instructions

System requirements:

• python3.10 (mandatory): install using apt install python3
• jsonnet-bundler (mandatory): install using go install -a github.com/jsonnet-bundler/jsonnet-bundler/cmd/jb@v0.5.1
• bitwarden cli tool (optional): install using snap install bw
• pass cli tool (optional): install using apt install pass

Building Steps

• Create a virtual python environment and install necessary python dependencies using poetry:

$ make init

• Testing build

$ ./otterdog.sh -h

Setup

The general configuration for supported organizations and their corresponding credentials in order to access their GitHub settings has to be placed in a json file (default: otterdog.json, can be changed with the -c flag):

{ 
  ...
  "organizations": [
    {
      "name": "<org name>",
      "github_id": "<github org id>",
      "credentials": {
        "provider": "<bitwarden | pass>",
        ... // provider specific data
      }
    }
  ]
  ...
}

Credentials

Otterdog needs certain credentials to access information from an organization and its repositories on GitHub:

• username / password / 2FA seed
• API token

The login / username / 2FA seed are required to access the web interface of GitHub in order to retrieve certain settings that are not accessible via its rest / graphql API.

The GitHub api token needs to have the following scopes enabled:

• repo
• workflow
• admin:org
• admin:org_hook

The credentials can be stored in different providers (bitwarden, pass).

Bitwarden

When using bitwarden to store the credentials, you need to enter a valid item id as additional credential data:

{ 
  ...
  "organizations": [
    {
      "name": "<org name>",
      "github_id": "<github org id>",
      "credentials": {
        "provider": "bitwarden",
        "item_id" : "<bitwarden item id>"
      }
    }
  ]
  ...
}

The item stored in bitwarden needs to contain the following information (a sample json output of such an item):

{
  "object": "item",
  "id": "<bitwarden item id>",
  "name": "<item name>",
  "fields": [
    {
      "name": "api_token_admin",
      "value": "<github API token>"
    }
  ],
  "login": {
    "username": "<github username>",
    "password": "<github password>",
    "totp": "<github TOTP text code>"
  }
}

Mandatory items:

• Field with name "api_token_admin" and as value the GitHub token to access the organization
• login.username of a user that can access the organization with enabled 2FA
• login.password the password of that user
• login.totp the TOTP text code

Pass

When using pass to store the credentials, you need to enter fully qualified pass names to access the various required credential data:

{ 
  ...
  "organizations": [
    {
      "name": "<org name>",
      "github_id": "<github org id>",
      "credentials": {
        "provider": "pass",
        "api_token": "<path/to/api_token>",
        "username": "<path/to/username>",
        "password": "<path/to/password>",
        "2fa_seed": "<path/to/2fa_seed>"
      }
    }
  ]
  ...
}

In case your password storage dir is not located at the default location, you can configurate that in the defaults:

{
  "defaults": {
    "pass": {
      "password_store_dir": "path/to/storage/dir"
    }
  }
}

Supported settings

Organization Settings

Webhooks

The secret value can be resolved using a credential provider. The supported format is <credential_provider>:<provider specific data>:

• Bitwarden: bitwarden:<bitwarden item id>@<custom_field_key>
• Pass: pass:<path/to/secret>

Examples:

{
  "secret": "bitwarden:118276ad-158c-4720-b68d-af8c00fe3481@webhook_secret"
}

{
  "secret": "pass:myorg/mywebhook_secret"
}

Note: After executing an import operation, the secret will be set to ****** as GitHub will only send redacted secrets. You will need to update the definition file with the real secret values, either by entering the secret value (not adivsed), or referencing it via a credential provider.

Repository Settings

Branch Protection Rules

Note:

• allowsForcePushes: if this set to True, any actor with push permission can force push to the branch
• bypassForcePushAllowances: if the actor list is non-empty but allowsForcePushes is set to True, a validation error will be issued
• pushRestrictions: the contents of the actor list controls whether push restriction is enabled or disabled, i.e. an empty list disables it
• reviewDismissalAllowances: if the actor list is non-empty but restrictsReviewDismissals is set to False, a validation error will be issued

Actor Format

• User: /login, e.g. /netomi
• Team: <organization>/<team-slug>, e.g. OtterdogTest/committers

Status Check Format

• GitHub Action Status: <status-name>, e.g. Run CI
• Status from an app: <app-slug>:<status-name>, e.g. eclipsefdn:eca
• Status from any source: any:<status-name, e.g. any:Run CI

Usage

Run the import operation to retrieve the current live configuration for an organization:

$ otterdog.sh import <organization>

The created configuration file for the organization can be found at <data-directory>/orgs/<organization>.jsonnet

Run the plan operation to highlight differences between the live configuration and the written configuration:

$ otterdog.sh plan <organization>

Run apply operation to reflect the written configuration on github itself:

$ otterdog.sh apply <organization>

Container Runtime (Linux/MacOS)

Requirements

• An otterdog.json already in your current directory
• (Recommended) a directory orgs

Bulding Container Image

• Creating a container local image

make container_build

Building a Development Container Image

• Creating a container development local image

make container_build_dev

Running otterdog in a container

Using Bitwarden client

• Firstly you need to login and unlock your Bitwarden session by executing the command below

bw login
bw unlock

• As result, you will get a token session. Please follow example below to make available in your terminal (Linux/MacOS)

export BW_SESSION="ffdsajklloremipsumfxs000f000r0loremipsum"

Using pass client

• Pass needs to be already installed as well as configured with all data needed in otterdog.json by executing pass in your profile

Activating Otterdog Container Runtime

• Activate otterdog environment will create an alias otterdog

source scripts/bin/active-otterdog

• Checking otterdog alias

otterdog -h

• Deactivating otterdog environment

deactivate-otterdog

otterdog container Arguments

• An table arguments

• Please find below an example to show configuration

otterdog -c $PWD/otterdog.json -g $HOME/.gnupg -o $PWD/orgs apply -f

Usage Otterdog Container Runtime

• Please follow the section Usage
• Please bear in mind that all command need to drop .sh

Activating Development Otterdog Container Runtime

• Activating developemnt otterdog environment will run eclipse/otterdog:dev. Thus it can be used container shell. Please find below an example

export OTTERDOG_DEV=1; source scripts/bin/active-otterdog
otterdog /bin/bash

• To activate development otterdog environment with arbument

export OTTERDOG_DEV=1; source scripts/bin/active-otterdog
otterdog -c $PWD/otterdog.json -g $HOME/.gnupg -o $PWD/orgs /bin/bash

• Checking otterdog environment

/app/otterdog.sh -h

• Deactivating development otterdog runtime type exit then deactivate-otterdog

Usage Development Otterdog Container Runtime

• Please follow the section Usage

Cleaning container environment

• Please use the macro below

make container_clean

Known issues

GraphQL

• branch protection rule property blocksCreations can not be updated via an update or create mutation, always seem to be false
• repo setting secret_scanning_push_protection seems to be only available for GitHub Enterprise billing, omitting for now