Getting custom details through v2 webhooks


(Demitri Morgan) #1

Did you know, it’s possible to get the data from the event that triggered an incident, including the custom details, through v2 webhooks? This allows you to collect data from your upstream monitoring tools into your system(s) of record that receive data through PagerDuty webhooks.

How it works

Per the documentation, similar to the original webhook format, each POST request contains a JSON-encoded object with a single messages property, containing an array of message objects, each one representing an incident lifecycle event that recently occurred.

One of the new properties of each message object is log_entries, which in this context is an array of log entry type objects that are relevant to the particular incident lifecycle event. The schema of these log_entry objects is the same as described in the Request Schema of the GET /log_entries/{id} documentation.

Here’s where things get interesting: the channel property is included in the log entry objects in v2 webhooks. When the event is a trigger (type trigger_log_entry), it will have the triggering event data that we’re after. See “The channel property of log entries” in our Knowledge Base article on how to retrieve incident details through the REST API:

So, putting it all together:

  1. When an incident is triggered, a v2 webhook will send a payload containing an incident.trigger type message
  2. In the log_entries array property of the message, we will find a log entry object whose type property is trigger_log_entry
  3. The channel property of the log entry object has the trigger event data

Example: raw log data from Splunk into ServiceNow incidents

In this hypothetical example, we have a ServiceNow-integrated service (with v4 of our integration, which utilizes our new v2 webhooks architecture) that also has a Splunk integration on it. Splunk alerts will thus trigger incidents on the PagerDuty service, which will then open incidents in ServiceNow.

Our goal is to set the Description field in the ServiceNow incident to the raw log data from Splunk.

We’ll be using the Inbound Field Rules feature (new in v4 of the PagerDuty and ServiceNow integration), and in particular are interested in setting fields of the incident based on the webhook payload.

The Splunk webhooks integration produces incidents with details structured as follows, with the original log events’ full contents in the _raw property:

You’d then need to create an inbound field rule as follows:

  1. For PagerDuty Webhook Type select Trigger
  2. For the ServiceNow Incident Field we’ll choose our destination field, Description
  3. For the Type we need to select Set From PagerDuty Webhook Payload
  4. The PagerDuty Webhook Payload to use should be

:information_source: Special feature of ServiceNow webhooks

ServiceNow webhooks are sent one message / event per payload, and within inbound field rules (per the documentation on setting based on payload values) the object namespace path to the channel object is simplified to from messages[n].log_entries[m].channel (what you’d otherwise need to use), where

  • n is the index of the incident.trigger event and
  • m is the index of the trigger_log_entry log entry, usually zero

The log_entries property is in this case the log entry, rather than an array containing a log entry.

Creating incidents
(system) #2