Gitlab Runner As Code - GRAC!

GRAC! is a infrastructure as code of gitlab runner in kubernetes infrastructure for projects hosted by the Eclipse Foundation at gitlab.eclipse.org.

What is GRAC?

The goal of GRAC! is to make the administration of hundreds of gitlab runner instances on a Kubernetes based infrastructure bearable. It uses automation, templates and a configuration-as-code approach. GRAC! consist of a set of custom shell scripts and a jsonnet generation. It relies also on the Docker CLI, OpenShift CLI.

Why GRAC?

With the successfull implementation of jenkins instance as code at Eclipse Foundation under the name of JIRO, and over +250 instances in production, Gitlab Continuous Integration has been request by projects to be supported. So as a natural extension, GRAC! is Born, under the woods: same approach and technologies.

As JIRO, it's expected a lower administration overhead for recurring task like setting up a new gitlab runner for projects/groups, configure, allowing ressources, handle specifics requests, etc.

Before starting

Instance notion

Each groups or projects in gitlab link to the instance reference in all grac documentation

Project: https://gitlab.eclipse.org/eclipsefdn/it/releng/gitlab-runner-as-code

• Instance: eclipsefdn.it.releng.gitlab-runner-as-code

Group: https://gitlab.eclipse.org/eclipsefdn/it/releng

• Instance: eclipsefnd.it.releng

Generation in directory: ./instances

This notion will affect generation in cluster

• Ex: ef-grac-eclipsefnd-it-releng namespace

Kubernetes Context

All commands in GRAC depends on kube current-context.

Before launching a command, it is recommend to verify the current target : kubectl config current-context or kubectx

All generation files will be attach to this context

CBI Configuration

Create file ~/.cbi/config

{
  "kubeconfig": {
    "path": "~/.kube/config",
    "contextMapping": {
      "okd-c1": "okd-c1",
      "okd-c2": "okd-c2",
    }
  },
  "password-store": {
    "cbi-dir": "~/.password-store-cbi",
    "it-dir": "~/.password-store"
  },
  "gitlab-token": "XXXXXXXXXXXXXX",
 }

kubeconfig: allow to configure path to kubernetes configuration file. Default: ~/.kube/config

password-store: configuration of pass tool.

gitlab-token: Personal Acces Token from gitlab. Need to be create from profil UI gitlab. PAT

NOTE: password-store is only used to retrieve gitlab runner registration token for shared runner store from administration UI.

Kube configuration

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: ...
    server: https://api.xxxx.org
  name: okd-c1
- cluster:
    certificate-authority-data: ...
    server: https://api.xxxx.org
  name: okd-c2
contexts:
- context:
    cluster: okd-c1
    namespace: ef-grac-...
    user: sebastien.heurtematte@eclipse-foundation.org/xxxx
  name: okd-c1
- context:
    cluster: okd-c2
    namespace: ef-grac-...
    user: sebastien.heurtematte@eclipse-foundation.org/xxxx
  name: okd-c2
current-context: okd-c2
kind: Config
preferences: {}
users:
- name: sebastien.heurtematte@eclipse-foundation.org/xxxxx
  user:
    token: sha256~XXXXXXX
- name: sebastien.heurtematte@eclipse-foundation.org/xxxx
  user:
    token: sha256~XXXXXXX

Build docker image

docker build . --tag grac 

Execute command

docker run \
    -v "${PWD}":/app \
    -v ~/.cbi/config:/home/grac/.cbi/config \
    -v ~/.kube/config:/home/grac/.kube/config \
    -v ~/.password-store-cbi:/home/grac/.password-store-cbi \
    --env KUBECONFIG=/home/grac/.kube/config \
    --env PASSWORD_STORE_DIR=/home/grac/.password-store-cbi \
    --network host \
    grac make deploy instance=eclipsefdn.it.releng.gitlab-runner-as-code

or directly with the script grac.sh

./grac.sh deploy eclipsefdn.it.releng.gitlab-runner-as-code

IMPORTANT: All command execute depends on Kubernetes context. see current context : kubectl config current-context or kubectx

Tasks

Tasks *-all

All tasks define have an equivalent with -all suffix. Ex: create-all, deploy-all, k8s-all, ...

They apply to all instances of the directory ./instances. And depends on the kubernetes context.