Add custom metrics & grafana alerts

I would like to suggest to leverage the use of some Grafana Alerting rules to get notified under a certain scenario, implementing some metrics that would register events happening under GIT ECA. This approach would enable us to have a more flexible solution for notification setup, allowing us to configure multiple Contact Points in a straightforward manner. By doing so, we could avoid the need to maintain custom code for each individual configuration, which not only saves time but also reduces the likelihood of errors and inconsistencies.

This change would supersede #149 (closed) implementation.

We might want to think about what metrics could be registered, so that we could potentially create more alerts.

added Feature Request statein discussion team:webdev webdev:java labels

Could you please give your point of view around this topic @malowe

CC: @cguindon

+1 from me — I really like this idea! I think this notification approach is something we should consider adopting for more projects in the future, and this would be a great first step to get us started.

It would also allow us to generate charts to better understand how often and when these errors occur, and make this information more accessible beyond just the webdev team.