Skip to content

openPASS: Kubernetes pods not starting up

Summary

Despite no other pods running, resource quota blocks launching a new build agent.

Steps to reproduce

Start a build job in https://ci.eclipse.org/openpass/job/openPASS_simulator_build/

What is the current bug behavior?

Build agent doesn't start up.

What is the expected correct behavior?

Build agents starts succesfully.

Relevant logs and/or screenshots

https://ci.eclipse.org/openpass/job/openPASS_simulator_build/view/change-requests/job/MR-219/12/console

08:29:59  All nodes of label ‘openPASS_simulator_build_MR-219_12-7whtq’ are offline
08:30:03  WARNING: Unable to create pod: kubernetes openpass/openpass-simulator-build-mr-219-12-7whtq-ktqqr-ptxh0 because kubernetes resource quota exceeded. 
08:30:03  Failure executing: POST at: https://10.30.0.1:443/api/v1/namespaces/openpass/pods. Message: pods "openpass-simulator-build-mr-219-12-7whtq-ktqqr-ptxh0" is forbidden: exceeded quota: jenkins-instance-quota, requested: limits.memory=16896Mi,requests.cpu=4100m,requests.memory=16640Mi, used: limits.memory=37376Mi,requests.cpu=9650m,requests.memory=36Gi, limited: limits.memory=49664Mi,requests.cpu=13450m,requests.memory=49664Mi. Received status: Status(apiVersion=v1, code=403, details=StatusDetails(causes=[], group=null, kind=pods, name=openpass-simulator-build-mr-219-12-7whtq-ktqqr-ptxh0, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=pods "openpass-simulator-build-mr-219-12-7whtq-ktqqr-ptxh0" is forbidden: exceeded quota: jenkins-instance-quota, requested: limits.memory=16896Mi,requests.cpu=4100m,requests.memory=16640Mi, used: limits.memory=37376Mi,requests.cpu=9650m,requests.memory=36Gi, limited: limits.memory=49664Mi,requests.cpu=13450m,requests.memory=49664Mi, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=Forbidden, status=Failure, additionalProperties={}).
08:30:03  Retrying...

Priority

  • Urgent
  • High
  • Medium
  • Low

Severity

  • Blocker
  • Major
  • Normal
  • Low

Impact

No CI -> no merges