We had multiple events where our applications sent alerts to PagerDuty through the Icinga2 integration, but the PagerDuty team, service, schedule, or escalation policy wasn’t configured properly.
So we decided to monitor the target of each alert in Icinga in PagerDuty, which means that for every Icinga2 User with an Integration Key we will run a check in front of PagerDuty API to see if a recipient for that Integration Key exists and if it’s properly configured.
We are having trouble implementing this with PagerDuty API and would love some assistance.
The best way to do this is to get the service name by the integration key from PagerDuty API.
I didn’t find a way to do that, tried to look for an integration endpoint, and tried going over all the services to find an integration with my integration key, but the API doesn’t supply a proper result for either.
I even considered creating an Event by the integration key with a defined incident key, to later query the service name of the created incident, or just to try an see if an incident were created, which means the service is properly configured and someone is on call, however, this shoots alerts to my development teams which is very problematic.
I would appreciate any idea or suggestion,
Thank You in advanced!