PDLC ACM CR bug on Gathering

Description

The CODECO application CR (codecoapp) does not use the avgServiceFailure value that PDLC tries to gather.

Traceback (most recent call last):
  File "/app/server.py", line 57, in <module>
    cluster_sample = get_sample()
                     ^^^^^^^^^^^^
  File "/app/k8s_calling/sample.py", line 365, in get_sample
    cluster_sample = get_acm_information(cluster_sample)          # Get the ACM information
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/k8s_calling/sample.py", line 110, in get_acm_information
    pod_obj["pod_failure"] = pod["avgServiceFailure"]
                             ~~~^^^^^^^^^^^^^^^^^^^^^
KeyError: 'avgServiceFailure'

Example of CODECO Application CR as JSON:

{'apiVersion': 'codeco.he-codeco.eu/v1alpha1', 'items': [{'apiVersion': 'codeco.he-codeco.eu/v1alpha1', 'kind': 'CodecoApp', 'metadata': {'annotations': {'kubectl.kubernetes.io/last-applied-configuration': '{"apiVersion":"codeco.he-codeco.eu/v1alpha1","kind":"CodecoApp","metadata":{"annotations":{},"generation":1,"name":"codecoappinstance3","namespace":"he-codeco-acm","resourceVersion":"1456","uid":"5c948d7e-43d6-425b-b0b2-76402b606e07"},"spec":{"appEnergyLimit":"20","appFailureTolerance":"","appName":"acm-swm-app","codecoapp-msspec":[{"nwbandwidth":"1.2","nwlatency":"3","podspec":{"containers":[{"image":"quay.io/skupper/hello-world-backend:latest","name":"skupper-backend","ports":[{"containerPort":8080,"name":"skupper-backend","protocol":"TCP"}],"resources":{"limits":{"cpu":"2","memory":"4Gi"}}}]},"serviceChannels":[{"advancedChannelSettings":{"frameSize":"100","maxDelay":"1","minBandwidth":"5","sendInterval":"10"},"channelName":"frontend","otherService":{"appName":"acm-swm-app","port":9090,"serviceName":"front-end"}}],"serviceName":"backend"},{"nwbandwidth":"1.2","nwlatency":"3","podspec":{"containers":[{"image":"quay.io/dekelly/frontend-app:v0.0.2","name":"front-end","ports":[{"containerPort":8080,"protocol":"TCP"}]}]},"serviceChannels":[{"advancedChannelSettings":{"frameSize":"100","maxDelay":"1","minBandwidth":"5","sendInterval":"10"},"channelName":"backend","otherService":{"appName":"acm-swm-app","port":8080,"serviceName":"backend"}}],"serviceName":"front-end"}],"complianceClass":"High","qosClass":"Gold","securityClass":"Good"}}\n'}, 'creationTimestamp': '2024-10-16T07:51:05Z', 'generation': 1, 'managedFields': [{'apiVersion': 'codeco.he-codeco.eu/v1alpha1', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:annotations': {'.': {}, 'f:kubectl.kubernetes.io/last-applied-configuration': {}}}, 'f:spec': {'.': {}, 'f:appEnergyLimit': {}, 'f:appFailureTolerance': {}, 'f:appName': {}, 'f:codecoapp-msspec': {}, 'f:complianceClass': {}, 'f:qosClass': {}, 'f:securityClass': {}}}, 'manager': 'kubectl-client-side-apply', 'operation': 'Update', 'time': '2024-10-16T07:51:05Z'}, {'apiVersion': 'codeco.he-codeco.eu/v1alpha1', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:status': {'.': {}, 'f:appMetrics': {'.': {}, 'f:numPods': {}, 'f:serviceMetrics': {}}, 'f:nodeMetrics': {}}}, 'manager': 'manager', 'operation': 'Update', 'subresource': 'status', 'time': '2024-10-16T08:49:08Z'}], 'name': 'codecoappinstance3', 'namespace': 'he-codeco-acm', 'resourceVersion': '19821', 'uid': 'db063007-e505-473b-a068-32ab5db586a4'}, 'spec': {'appEnergyLimit': '20', 'appFailureTolerance': '', 'appName': 'acm-swm-app', 'codecoapp-msspec': [{'nwbandwidth': '1.2', 'nwlatency': '3', 'podspec': {'containers': [{'image': 'quay.io/skupper/hello-world-backend:latest', 'name': 'skupper-backend', 'ports': [{'containerPort': 8080, 'name': 'skupper-backend', 'protocol': 'TCP'}], 'resources': {'limits': {'cpu': '2', 'memory': '4Gi'}}}]}, 'serviceChannels': [{'advancedChannelSettings': {'frameSize': '100', 'maxDelay': '1', 'minBandwidth': '5', 'sendInterval': '10'}, 'channelName': 'frontend', 'otherService': {'appName': 'acm-swm-app', 'port': 9090, 'serviceName': 'front-end'}}], 'serviceName': 'backend'}, {'nwbandwidth': '1.2', 'nwlatency': '3', 'podspec': {'containers': [{'image': 'quay.io/dekelly/frontend-app:v0.0.2', 'name': 'front-end', 'ports': [{'containerPort': 8080, 'protocol': 'TCP'}]}]}, 'serviceChannels': [{'advancedChannelSettings': {'frameSize': '100', 'maxDelay': '1', 'minBandwidth': '5', 'sendInterval': '10'}, 'channelName': 'backend', 'otherService': {'appName': 'acm-swm-app', 'port': 8080, 'serviceName': 'backend'}}], 'serviceName': 'front-end'}], 'complianceClass': 'High', 'qosClass': 'Gold', 'securityClass': 'Good'}, 'status': {'appMetrics': {'numPods': 2, 'serviceMetrics': [{'avgServiceCpu': '0.002035', 'avgServiceMemory': '20209664.000000', 'clusterName': 'codeco-cluster-1', 'nodeName': 'kind-control-plane', 'podName': 'acm-swm-app-front-end', 'serviceName': 'backend'}, {'avgServiceCpu': '0.001203', 'avgServiceMemory': '17732949.333333', 'clusterName': 'codeco-cluster-1', 'nodeName': 'kind-control-plane', 'podName': 'acm-swm-app-backend', 'serviceName': 'front-end'}]}, 'nodeMetrics': [{'avgNodeCpu': '0.148639', 'avgNodeMemory': '130049856512.000000', 'nodeName': 'kind-control-plane'}, {'avgNodeCpu': '0.146958', 'avgNodeMemory': '130047940608.000000', 'nodeName': 'c2'}, {'avgNodeCpu': '0.150347', 'avgNodeMemory': '129995383808.000000', 'nodeName': 'c1'}]}}], 'kind': 'CodecoAppList', 'metadata': {'continue': '', 'resourceVersion': '19872'}}

Example of CODECO Application CR as YAML:

apiVersion: codeco.he-codeco.eu/v1alpha1
kind: CodecoApp
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"codeco.he-codeco.eu/v1alpha1","kind":"CodecoApp","metadata":{"annotations":{},"generation":1,"name":"codecoappinstance3","namespace":"he-codeco-acm","resourceVersion":"1456","uid":"5c948d7e-43d6-425b-b0b2-76402b606e07"},"spec":{"appEnergyLimit":"20","appFailureTolerance":"","appName":"acm-swm-app","codecoapp-msspec":[{"nwbandwidth":"1.2","nwlatency":"3","podspec":{"containers":[{"image":"quay.io/skupper/hello-world-backend:latest","name":"skupper-backend","ports":[{"containerPort":8080,"name":"skupper-backend","protocol":"TCP"}],"resources":{"limits":{"cpu":"2","memory":"4Gi"}}}]},"serviceChannels":[{"advancedChannelSettings":{"frameSize":"100","maxDelay":"1","minBandwidth":"5","sendInterval":"10"},"channelName":"frontend","otherService":{"appName":"acm-swm-app","port":9090,"serviceName":"front-end"}}],"serviceName":"backend"},{"nwbandwidth":"1.2","nwlatency":"3","podspec":{"containers":[{"image":"quay.io/dekelly/frontend-app:v0.0.2","name":"front-end","ports":[{"containerPort":8080,"protocol":"TCP"}]}]},"serviceChannels":[{"advancedChannelSettings":{"frameSize":"100","maxDelay":"1","minBandwidth":"5","sendInterval":"10"},"channelName":"backend","otherService":{"appName":"acm-swm-app","port":8080,"serviceName":"backend"}}],"serviceName":"front-end"}],"complianceClass":"High","qosClass":"Gold","securityClass":"Good"}}
  creationTimestamp: "2024-10-16T07:51:05Z"
  generation: 1
  managedFields:
  - apiVersion: codeco.he-codeco.eu/v1alpha1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .: {}
          f:kubectl.kubernetes.io/last-applied-configuration: {}
      f:spec:
        .: {}
        f:appEnergyLimit: {}
        f:appFailureTolerance: {}
        f:appName: {}
        f:codecoapp-msspec: {}
        f:complianceClass: {}
        f:qosClass: {}
        f:securityClass: {}
    manager: kubectl-client-side-apply
    operation: Update
    time: "2024-10-16T07:51:05Z"
  - apiVersion: codeco.he-codeco.eu/v1alpha1
    fieldsType: FieldsV1
    fieldsV1:
      f:status:
        .: {}
        f:appMetrics:
          .: {}
          f:numPods: {}
          f:serviceMetrics: {}
        f:nodeMetrics: {}
    manager: manager
    operation: Update
    subresource: status
    time: "2024-10-16T09:08:25Z"
  name: codecoappinstance3
  namespace: he-codeco-acm
  resourceVersion: "23770"
  uid: db063007-e505-473b-a068-32ab5db586a4
spec:
  appEnergyLimit: "20"
  appFailureTolerance: ""
  appName: acm-swm-app
  codecoapp-msspec:
  - nwbandwidth: "1.2"
    nwlatency: "3"
    podspec:
      containers:
      - image: quay.io/skupper/hello-world-backend:latest
        name: skupper-backend
        ports:
        - containerPort: 8080
          name: skupper-backend
          protocol: TCP
        resources:
          limits:
            cpu: "2"
            memory: 4Gi
    serviceChannels:
    - advancedChannelSettings:
        frameSize: "100"
        maxDelay: "1"
        minBandwidth: "5"
        sendInterval: "10"
      channelName: frontend
      otherService:
        appName: acm-swm-app
        port: 9090
        serviceName: front-end
    serviceName: backend
  - nwbandwidth: "1.2"
    nwlatency: "3"
    podspec:
      containers:
      - image: quay.io/dekelly/frontend-app:v0.0.2
        name: front-end
        ports:
        - containerPort: 8080
          protocol: TCP
    serviceChannels:
    - advancedChannelSettings:
        frameSize: "100"
        maxDelay: "1"
        minBandwidth: "5"
        sendInterval: "10"
      channelName: backend
      otherService:
        appName: acm-swm-app
        port: 8080
        serviceName: backend
    serviceName: front-end
  complianceClass: High
  qosClass: Gold
  securityClass: Good
status:
  appMetrics:
    numPods: 2
    serviceMetrics:
    - avgServiceCpu: "0.001424"
      avgServiceMemory: "20209664.000000"
      clusterName: codeco-cluster-1
      nodeName: kind-control-plane
      podName: acm-swm-app-front-end
      serviceName: backend
    - avgServiceCpu: "0.001437"
      avgServiceMemory: "17732949.333333"
      clusterName: codeco-cluster-1
      nodeName: kind-control-plane
      podName: acm-swm-app-backend
      serviceName: front-end
  nodeMetrics:
  - avgNodeCpu: "0.149440"
    avgNodeMemory: "130114941952.000000"
    nodeName: kind-control-plane
  - avgNodeCpu: "0.149463"
    avgNodeMemory: "130070980608.000000"
    nodeName: c2
  - avgNodeCpu: "0.150088"
    avgNodeMemory: "130128721920.000000"
    nodeName: c1

Possible Solution

Check if the metric is used by PDLC; if not, remove it from the gathering or give it a default value (PDLC could also try to gather it itself—not recommended)

Edited Oct 16, 2024 by Panagiotis Karamolegkos