ServiceNow integration stopped working

#1

I have an odd problem that started this morning between about 06:30 and 08:00 GMT.

I have three Pager Duty services integrated with a single Assignment group in ServiceNow. Up until this morning, all three would raise incidents into ServiceNow and there would be comms between both the systems.

As of this morning, two of the Services have stopped working. One writes nothing to ServiceNow at all, the other is creating empty incidents. The third continue to work fine.

Does anyone have an inkling as to what might have changed this morning? Has there been some sort of PagerDuty release? I’ve looked but can’t find anything.

Has anyone else had any similar problems today?

Any help or direction would be appreciated!

#2

Hi Spike,

I’ve never seen anything like that.
You may want to start with the basics, from the Pagerduty perspective.

1.Since you said, the Pagerduty Services are OK, I suspect you don’t need to check them?

-Otherwise, make sure these are not disabled in Pagerduty.
-If using an email integration on these PD services, send an email, verify a new PD incident is created without any involvement from ServiceNow.

Assuming your PD services are fine, follow up with you ServiceNow team.

2.From the ServiceNow end, ask your ServiceNow admin to login, take a look at the Pagerduty config object.

Typically this will list the ServiceNow url and acct information, it will also have a pagerduty url info and pagerduty API key.

I would also suggest, enabling ‘debug’ on the Pagerduty config object in ServiceNow, this is an option at the very bottom of the Pagerduty config object in ServiceNow.

This may help you and the SNOW team diagnose the problem.

#3

link here may also help

Make sure no one changed your servicenow integration on your service, check the extension version on the Pagerduty service, etc. There are multiple versions of the ServiceNow integration available for a Pagerduty service.

See here,

Login to Pagerduty -> Integrations -> Extensions …url is typically, https://yourdomain.pagerduty.com/extensions

It will list the ServiceNow version for your specific services…

I doubt your versions have changed, unless, someone other than you, is making changes in your environment.

#4

I raised a ticket with PagerDuty about it and they came back asking for some more information.

The following morning as I was gathering it, I could see things had started to work again about 12 hours after it stopped - with no intervention from me on either PD or ServiceNow.

Given that only part of the process was broken, it’s hard to pin where the fault might have been. Knowing nothing had changed in ServiceNow, my suspicion is PD. IF only there were some sort of logs that you could look at to see what goes on behind the scenes. I’ve asked support about this but have yet to hear anything back.