Metrics not being shown in execution-run.json after training.

A metric calculation step is being started directly after the training. Training might take longer but in this example a small dataset is being used so the training is as fast as a inference. Although model has logger.info('MericsAvailable') and logging metrics as shown in the screenshot below:

Here is what is being logged:

Logged metric

2023-12-12 08:57:26,409 __main__ INFO {'metrics': {'type': 'coco-metrics/v1', 'date_time': '2023-12-12 08:57:22', 'status_text': 'success', 'more_is_better': {'AP': 0.0, 'AP_50': 0.0}, 'less_is_better': {}}}

And here is how the execution-run.json after the training:

execution-run.json

{ "probeIndicator": [ { "value": "false" } ], "nodes": [ { "proto_uri": "org/acumos/d4e836ce-d145-4cab-a2e7-8c0b6bd10ebb/detrmodel/1.0.0/detrmodel-1.0.0.proto", "image": "cicd.ai4eu-dev.eu:7444/tutorials/detr_objectdetection/detr_model:latest", "node_type": "MLModel", "container_name": "detrmodel1", "operation_signature_list": [ { "connected_to": [], "operation_signature": { "operation_name": "startTraining", "output_message_name": "TrainStatus", "input_message_name": "TrainingConfig", "output_message_stream": false, "input_message_stream": false } } ], "checksum": "" }, { "proto_uri": "org/acumos/7ea21f2e-0631-474f-a9d1-3d4e6ab6839f/detrs3databroker/1.0.0/detrs3databroker-1.0.0.proto", "image": "cicd.ai4eu-dev.eu:7444/tutorials/detr_objectdetection/detr_s3_databroker:latest", "node_type": "MLModel", "container_name": "detrs3databroker1", "operation_signature_list": [ { "connected_to": [ { "container_name": "detrmodel1", "operation_signature": { "operation_name": "startTraining" } } ], "operation_signature": { "operation_name": "send_training_signal", "output_message_name": "TrainingConfig", "input_message_name": "Empty", "output_message_stream": false, "input_message_stream": false } } ], "checksum": "docker-pullable://cicd.ai4eu-dev.eu:7444/tutorials/detr_objectdetection/detr_s3_databroker@sha256:78d65e6b5ecb4850845b5c34943ea15577931d618dc2b94144465a640e4bedf6" }, { "proto_uri": "org/acumos/37dd144b-2d43-49cd-b1a4-bed93541ce6f/detrtensorboard1/1.0.0/detrtensorboard1-1.0.0.proto", "image": "cicd.ai4eu-dev.eu:7444/tutorials/detr_objectdetection/detr_tensorboard:latest", "node_type": "MLModel", "container_name": "detrtensorboard11", "operation_signature_list": [], "checksum": "docker-pullable://cicd.ai4eu-dev.eu:7444/tutorials/detr_objectdetection/detr_tensorboard@sha256:d5140ff4a3cc35c24ecc63d716b9c1b9f13770b5cc1c7f2553c8f6284d09aaed" } ], "name": "trainingdetr", "version": "1", "input_ports": [], "system_info": { "system_name": "node03", "fqdn": "202.61.251.28", "cpu": "10", "gpu": "", "memory": "65850664Ki" } }