CODEF LPM-deployment not running and Empty netma-CR
Hi @tjaarda and @alejandromuniztid
- In my case of deploying CODECO via CODEF I found that the "prometheus-lpm-network" is not running while
kubectl rollout restart deployment/prometheus-lpm-networkdid nothing. These are some of the logs that might help, let me know if you need more. I have also made a cluster in the shared cloud space if you want to check it.
kc get po -A
kc get po -Acert-manager cert-manager-7fbbc65b49-rsp8r 1/1 Running 0 106m
cert-manager cert-manager-cainjector-6664fc84f6-69jmq 1/1 Running 0 106m
cert-manager cert-manager-webhook-59598898fd-ss72d 1/1 Running 0 106m
default athm4-lpm-7c7b87f7d6-drzpw 1/1 Running 0 98m
default athw7-lpm-6fd48d7dfb-5v8jp 1/1 Running 0 98m
default athw8-lpm-6d78bd4878-x6r55 1/1 Running 0 98m
default netperf-host-hqrl5 1/1 Running 0 97m
default netperf-host-lpmn7 1/1 Running 0 97m
default netperf-host-td88q 1/1 Running 0 97m
default netperf-pod-568cbcbbf7-k6qsd 1/1 Running 0 97m
default netperf-pod-568cbcbbf7-vfpcx 1/1 Running 0 97m
default netperf-pod-6lmzz 1/1 Running 0 97m
default netperf-pod-ntqbt 1/1 Running 0 97m
default netperf-pod-vps96 1/1 Running 0 97m
he-codeco-acm acm-operator-controller-manager-6874997f74-gcbhs 2/2 Running 0 104m
he-codeco-mdm freshness-connector-check-connection-6c845dbfd8-xfs99 1/1 Running 0 93m
he-codeco-mdm k8s-connector-check-connection-684558df6d-km8gq 1/1 Running 0 93m
he-codeco-mdm kubescape-connector-check-connection-84c84fdf9d-pc58d 1/1 Running 0 93m
he-codeco-mdm mdm-api-0 1/1 Running 0 93m
he-codeco-mdm mdm-controller-0 1/1 Running 2 (91m ago) 93m
he-codeco-mdm mdm-kafka-0 1/1 Running 0 96m
he-codeco-mdm mdm-neo4j-0 1/1 Running 0 93m
he-codeco-mdm mdm-zookeeper-0 1/1 Running 0 96m
he-codeco-netma l2sm-controller-6bcd667c4f-n74g2 1/1 Running 0 102m
he-codeco-netma l2sm-controller-manager-8bb8cbfd7-9zb7x 2/2 Running 1 (100m ago) 102m
he-codeco-netma l2sm-operator-654ffb97c4-nnr84 1/1 Running 0 102m
he-codeco-netma l2sm-switch-cmcb2 1/1 Running 0 102m
he-codeco-netma l2sm-switch-wtstc 1/1 Running 0 102m
he-codeco-netma l2sm-switch-xzcr8 1/1 Running 0 102m
he-codeco-netma mysql-pod 1/1 Running 0 102m
he-codeco-netma nemesys-5bc8b77bd8-gkz9m 1/1 Running 0 44m
he-codeco-pdlc gnn-controller-679597ccdf-gpwz4 1/1 Running 0 67m
he-codeco-pdlc pdlc-ca-c5f6fcbd4-cdq8j 1/1 Running 0 68m
he-codeco-pdlc pdlc-dp-7c5bcf6749-7dfcz 1/1 Running 0 69m
he-codeco-pdlc pdlc-gnn-inference-58459df9d4-jv679 1/1 Running 0 67m
he-codeco-pdlc pdlc-rl-8678d87c8d-zrwpn 1/1 Running 0 63m
he-codeco-swm controller-manager-74fdfb86b-88gwg 2/2 Running 0 59m
he-codeco-swm solver-76957d7db6-fzz7v 1/1 Running 0 59m
kepler kepler-exporter-46zhw 1/1 Running 0 58m
kepler kepler-exporter-4q9df 1/1 Running 0 58m
kepler kepler-exporter-rhsvq 1/1 Running 0 58m
kube-flannel kube-flannel-ds-plp29 1/1 Running 0 109m
kube-flannel kube-flannel-ds-pm65n 1/1 Running 0 109m
kube-flannel kube-flannel-ds-twnrc 1/1 Running 0 109m
kube-system coredns-7c65d6cfc9-k74dl 1/1 Running 0 109m
kube-system coredns-7c65d6cfc9-mbppn 1/1 Running 0 109m
kube-system etcd-athm4 1/1 Running 0 109m
kube-system kube-apiserver-athm4 1/1 Running 0 109m
kube-system kube-cni-install-ds-g79kl 1/1 Running 0 59m
kube-system kube-cni-install-ds-lhh5n 1/1 Running 0 59m
kube-system kube-cni-install-ds-n6hwb 1/1 Running 0 59m
kube-system kube-controller-manager-athm4 1/1 Running 0 109m
kube-system kube-multus-ds-5z2kt 1/1 Running 0 109m
kube-system kube-multus-ds-7tz78 1/1 Running 0 109m
kube-system kube-multus-ds-kk8qk 1/1 Running 0 109m
kube-system kube-proxy-5x7dg 1/1 Running 0 109m
kube-system kube-proxy-6sqs8 1/1 Running 0 109m
kube-system kube-proxy-pl6pp 1/1 Running 0 109m
kube-system kube-scheduler-athm4 1/1 Running 0 109m
kube-system nodedaemon-88d7t 1/1 Running 0 59m
kube-system nodedaemon-jxdth 1/1 Running 0 59m
kube-system nodedaemon-rh5b4 1/1 Running 0 59m
kube-system qos-multus-cni-5cvkk 1/1 Running 0 59m
kube-system qos-multus-cni-6lqtv 1/1 Running 0 59m
kube-system qos-multus-cni-nf5mt 1/1 Running 0 59m
kube-system qos-scheduler-6f9c6848dd-9k4jz 1/1 Running 0 59m
local-path-storage local-path-provisioner-84967477f-szbvt 1/1 Running 0 96m
monitoring alertmanager-main-0 2/2 Running 0 103m
monitoring alertmanager-main-1 2/2 Running 0 103m
monitoring alertmanager-main-2 2/2 Running 0 103m
monitoring blackbox-exporter-d4d45d65-w6v58 3/3 Running 0 104m
monitoring grafana-68958fc7b9-98klm 1/1 Running 0 104m
monitoring kube-state-metrics-6d8b8c455-7mjvp 3/3 Running 0 104m
monitoring node-exporter-jbrlf 2/2 Running 0 104m
monitoring node-exporter-jkrzr 2/2 Running 0 104m
monitoring node-exporter-p2rcm 2/2 Running 0 104m
monitoring prometheus-adapter-784f566c54-9zpgd 1/1 Running 0 104m
monitoring prometheus-adapter-784f566c54-j8dzt 1/1 Running 0 104m
monitoring prometheus-k8s-0 2/2 Running 0 103m
monitoring prometheus-k8s-1 2/2 Running 0 103m
monitoring prometheus-operator-7cd6d5b6b-vpzf5 2/2 Running 0 104m
network-k8s-namespace network-base-54d75dc8cf-xrzfq 1/1 Running 0 59m
kc get deployments
kc get deploymentsNAME READY UP-TO-DATE AVAILABLE AGE
athm4-lpm 1/1 1 1 63m
athw7-lpm 1/1 1 1 63m
athw8-lpm 1/1 1 1 63m
netperf-pod 2/2 2 2 62m
prometheus-lpm-network 0/1 0 0 62m
kc describe deployment prometheus-lpm-network
kc describe deployment prometheus-lpm-networkName: prometheus-lpm-network
Namespace: default
CreationTimestamp: Tue, 20 May 2025 11:17:30 +0000
Labels: <none>
Annotations: deployment.kubernetes.io/revision: 3
Selector: app=prometheus-lpm-network
Replicas: 1 desired | 0 updated | 0 total | 0 available | 2 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 25% max unavailable, 25% max surge
Pod Template:
Labels: app=prometheus-lpm-network
Annotations: kubectl.kubernetes.io/restartedAt: 2025-05-20T12:20:06Z
Service Account: lpm-network-sa
Containers:
prometheus:
Image: prom/prometheus:v2.30.3
Port: 9090/TCP
Host Port: 0/TCP
Args:
--config.file=/etc/prometheus/prometheus.yml
--storage.tsdb.path=/prometheus
Environment: <none>
Mounts:
/etc/prometheus from config-volume (rw)
/prometheus from data-volume (rw)
exporter:
Image: alexdecb/lpm-exporter:1.2.1
Port: <none>
Host Port: <none>
Environment:
TOPOLOGY_NAMESPACE: network-l2sm-namespace
Mounts: <none>
Volumes:
config-volume:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: prometheus-config-lpm-network
Optional: false
data-volume:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
Node-Selectors: <none>
Tolerations: <none>
Conditions:
Type Status Reason
---- ------ ------
Available False MinimumReplicasUnavailable
ReplicaFailure True FailedCreate
Progressing False ProgressDeadlineExceeded
OldReplicaSets: prometheus-lpm-network-58b458db8c (0/0 replicas created), prometheus-lpm-network-78d889f8bd (0/1 replicas created)
NewReplicaSet: prometheus-lpm-network-56b694d9bf (0/1 replicas created)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ScalingReplicaSet 52m deployment-controller Scaled up replica set prometheus-lpm-network-78d889f8bd to 1
Normal ScalingReplicaSet 32m deployment-controller Scaled down replica set prometheus-lpm-network-58b458db8c to 0 from 1
Normal ScalingReplicaSet 32m deployment-controller Scaled up replica set prometheus-lpm-network-56b694d9bf to 1 from 0
kc describe replicaset prometheus-lpm-network-56b694d9bf
kc describe replicaset prometheus-lpm-network-56b694d9bfName: prometheus-lpm-network-56b694d9bf
Namespace: default
Selector: app=prometheus-lpm-network,pod-template-hash=56b694d9bf
Labels: app=prometheus-lpm-network
pod-template-hash=56b694d9bf
Annotations: deployment.kubernetes.io/desired-replicas: 1
deployment.kubernetes.io/max-replicas: 2
deployment.kubernetes.io/revision: 3
Controlled By: Deployment/prometheus-lpm-network
Replicas: 0 current / 1 desired
Pods Status: 0 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
Labels: app=prometheus-lpm-network
pod-template-hash=56b694d9bf
Annotations: kubectl.kubernetes.io/restartedAt: 2025-05-20T12:20:06Z
Service Account: lpm-network-sa
Containers:
prometheus:
Image: prom/prometheus:v2.30.3
Port: 9090/TCP
Host Port: 0/TCP
Args:
--config.file=/etc/prometheus/prometheus.yml
--storage.tsdb.path=/prometheus
Environment: <none>
Mounts:
/etc/prometheus from config-volume (rw)
/prometheus from data-volume (rw)
exporter:
Image: alexdecb/lpm-exporter:1.2.1
Port: <none>
Host Port: <none>
Environment:
TOPOLOGY_NAMESPACE: network-l2sm-namespace
Mounts: <none>
Volumes:
config-volume:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: prometheus-config-lpm-network
Optional: false
data-volume:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
Node-Selectors: <none>
Tolerations: <none>
Conditions:
Type Status Reason
---- ------ ------
ReplicaFailure True FailedCreate
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedCreate 14m (x19 over 36m) replicaset-controller Error creating: pods "prometheus-lpm-network-56b694d9bf-" is forbidden: error looking up service account default/lpm-network-sa: serviceaccount "lpm-network-sa" not found
kc get serviceaccount -n he-codeco-netma
kc get serviceaccount -n he-codeco-netmaNAME SECRETS AGE
default 0 106m
l2sm-controller-manager 0 106m
l2sm-operator 0 106m
lpm-network-sa 0 102m
nemesys-sa 0 101m- Not sure if the next issue is related with the 1st one but nonetheless I'm asking ... I know it was found in other Issues as well, however the
kubectl rollout restart deployment nemesys -n he-codeco-netmadid not do anything and I cannot delete the LPM
kubectl get netma-topology netma-sample -o yaml -n he-codeco-netma
kubectl get netma-topology netma-sample -o yaml -n he-codeco-netmaapiVersion: codeco.com/v1
kind: NetmaTopology
metadata:
creationTimestamp: "2025-05-20T11:18:26Z"
generation: 1
name: netma-sample
namespace: he-codeco-netma
resourceVersion: "3720"
uid: abc3b373-de13-4c6a-9e40-ff627e089c09
overlay-topology:
links: []
networkImplementation: l2sm-network
nodes: []
physicalBase: logical-network
underlay-topology:
links: []
networkImplementation: underlay-network
nodes: []
paths: []
physicalBase: physical-networkEdited by Georgios Koukis