Skip to main content

Hey there,


We’re working on moving to zabbix from an old legacy monitoring system, and one thing that was essential for us was having Acknowledgements in PagerDuty show up in the monitoring system.


So we wrote it: https://github.com/sonic-com/pagerduty2zabbix


Handles:



  • PD incident ack -> zabbix event ack

  • PD incident un-ack -> zabbix event un-ack

  • PD trigger (creation) -> zabbix comment of link to the incident in PD

  • PD incident note -> zabbix event comment

  • PD incident resolved -> zabbix event close attempt

  • PD incident priority update -> zabbix event severityupdate


Anybody want to help test it out?


Caveat: not designed for super high alert volumes (it’s a perl CGI).


Also, in my testing it only works if the PD integration is the generic “events api v2”; the “zabbix” branded integration seems to make the dedup_key silently vanish and it’s needed for this.

FYI, 2-way integrations that depend on the deduplication key to identify alerts in the upstream monitoring system have historically only worked if “create alerts and incidents” is disabled for the service, because the dedup_key on the incident will be blank (instead, the alerts on the incident have the deduplication keys).


I see however the source code is already halfway to a workaround: retrieving the incident record from the REST API (in subroutine pagerduty_get_event_details). All you need to do is add the query parameter includel]=alerts to the URL and then, from the alerts property in the incident model in the response schema, the subroutine can put together a list of dedup keys from each alert associated with the incident, which can then be used to find and ack any Zabbix alerts that match them.


Good luck and thank you for contributing!

I’ve got it working for my stuff with just included]=body, but it was definitely finicky (some PD-side configs broke it, such as using the “Zabbix” branded integration)


I have noticed that merging breaks things (“resolves” most of the alerts and only updates the “parent” item after), though we don’t merge often. Sounds like using data fetched via includei]=alerts might get me to a fix for that issue…


Thanks for the info!

FYI,


This got some major updates… With a PD API key, it will fetch the data it needs (dedup_key) from the PD API.


Also does more than ACKs now:



  • reasonably configurable behaviors

  • Adds a comment and “signs” it as the PD user that did the action for most actions.

  • Can handle incidents and alerts. Can handle multi-problem PD WebHooks.

  • Uses a PD API key to get additional info (dedup_key/problem-id)

  • when PD gets the alert, add a comment in Zabbix linking to the PD alert/incident

  • ACKs in Zabbix when ACK’d in PD

  • un-acknowledging in PD un-acks in Zabbix

  • any comment (“annotation”) added in PD added as comment in Zabbix

  • If resolved in PD, it will try to resolve the problem in Zabbix (sends the API attempt, and only works on manually-closable events)

  • If PD “priority” updated, can change the Zabbix problem severity

  • If alerts are merged in PD, it sets the “children” as “symptoms” of the “parent” (a “cause”)

Reply


Terms & ConditionsCookie settings