[Quality Evaluator] FileNotFoundError in waitForDriverRules: Missing JSON File for Rules Processing

When trigger the Quality profiling, there's an error in the log file regarding the User Rules for a E-Redes dataset (attached to the issue).

Additionally, not all KPIs are computed for all columns. There are some columns without quality values, examples: Power Connection-In/Awaiting Confirmation [MVA] (Last Quarter), Hosting capacity HV [MVA] (Last Quarter).

User Rules error:

172.20.0.107 - - [12/Feb/2025:07:12:36 +0000] "POST /evaluate-data HTTP/1.1" 500 265 "-" "python-requests/2.31.0"
datamite-demo_quality-evaluator.1.rm1nbxe4ltkm@datamite-quality    | [2025-02-12 07:12:47,492] ERROR in app: Exception on /get-user-rules/string_RNT Connection-Bus bar 60kV (Forecast) [GET]
datamite-demo_quality-evaluator.1.rm1nbxe4ltkm@datamite-quality    | Traceback (most recent call last):
datamite-demo_quality-evaluator.1.rm1nbxe4ltkm@datamite-quality    |   File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1511, in wsgi_app
datamite-demo_quality-evaluator.1.rm1nbxe4ltkm@datamite-quality    |     response = self.full_dispatch_request()
datamite-demo_quality-evaluator.1.rm1nbxe4ltkm@datamite-quality    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
datamite-demo_quality-evaluator.1.rm1nbxe4ltkm@datamite-quality    |   File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 919, in full_dispatch_request
datamite-demo_quality-evaluator.1.rm1nbxe4ltkm@datamite-quality    |     rv = self.handle_user_exception(e)
datamite-demo_quality-evaluator.1.rm1nbxe4ltkm@datamite-quality    |          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
datamite-demo_quality-evaluator.1.rm1nbxe4ltkm@datamite-quality    |   File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 917, in full_dispatch_request
datamite-demo_quality-evaluator.1.rm1nbxe4ltkm@datamite-quality    |     rv = self.dispatch_request()
datamite-demo_quality-evaluator.1.rm1nbxe4ltkm@datamite-quality    |          ^^^^^^^^^^^^^^^^^^^^^^^
datamite-demo_quality-evaluator.1.rm1nbxe4ltkm@datamite-quality    |   File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 902, in dispatch_request
datamite-demo_quality-evaluator.1.rm1nbxe4ltkm@datamite-quality    |     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
datamite-demo_quality-evaluator.1.rm1nbxe4ltkm@datamite-quality    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
datamite-demo_quality-evaluator.1.rm1nbxe4ltkm@datamite-quality    |   File "/app/evaluator_middleware.py", line 294, in endpoint_with_string_input
datamite-demo_quality-evaluator.1.rm1nbxe4ltkm@datamite-quality    |     profileResult = waitForDriverRules(rules_dataset_id, spark_master_ip, subID, rules_dataset_id)
datamite-demo_quality-evaluator.1.rm1nbxe4ltkm@datamite-quality    |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
datamite-demo_quality-evaluator.1.rm1nbxe4ltkm@datamite-quality    |   File "/app/evaluator_middleware.py", line 253, in waitForDriverRules
datamite-demo_quality-evaluator.1.rm1nbxe4ltkm@datamite-quality    |     with open(f"/app/spark_jobs/data/{ds_id}_received_rules.json", 'r') as file:
datamite-demo_quality-evaluator.1.rm1nbxe4ltkm@datamite-quality    |          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
datamite-demo_quality-evaluator.1.rm1nbxe4ltkm@datamite-quality    | FileNotFoundError: [Errno 2] No such file or directory: '/app/spark_jobs/data/string_RNT Connection-Bus bar 60kV (Forecast)_received_rules.json'

Dataset used to test:

E-REDES-SubstationCapacity-DS_.csv

/cc @jpla