Add a scheduled job to reconcile data that is missed by system hooks
To better make sure that we don't miss any of the projects, and keep clean and accurate data, we will create a scheduled task in Quarkus (see Scheduler) to reconcile data that is either missing or out of date. This task should use the Gitlab Projects API with a visibility filter in place to fetch all private projects in the instance. For each page of results, we will make SQL queries to fetch the rows in the database in batches (using something like an
ids filter in the DtoFilter) and check that they are up to date.
The different operations we care about:
- Current path is up to date (create entry if none match exactly)
- If not found, mark all associated entries as deleted
- If it doesn't exist at all, create initial entry