ServiceNow integration stopped working

I have an odd problem that started this morning between about 06:30 and 08:00 GMT.

I have three Pager Duty services integrated with a single Assignment group in ServiceNow. Up until this morning, all three would raise incidents into ServiceNow and there would be comms between both the systems.

As of this morning, two of the Services have stopped working. One writes nothing to ServiceNow at all, the other is creating empty incidents. The third continue to work fine.

Does anyone have an inkling as to what might have changed this morning? Has there been some sort of PagerDuty release? I’ve looked but can’t find anything.

Has anyone else had any similar problems today?

Any help or direction would be appreciated!

Hi Spike,

I’ve never seen anything like that.
You may want to start with the basics, from the Pagerduty perspective.

1.Since you said, the Pagerduty Services are OK, I suspect you don’t need to check them?

-Otherwise, make sure these are not disabled in Pagerduty.
-If using an email integration on these PD services, send an email, verify a new PD incident is created without any involvement from ServiceNow.

Assuming your PD services are fine, follow up with you ServiceNow team.

2.From the ServiceNow end, ask your ServiceNow admin to login, take a look at the Pagerduty config object.

Typically this will list the ServiceNow url and acct information, it will also have a pagerduty url info and pagerduty API key.

I would also suggest, enabling ‘debug’ on the Pagerduty config object in ServiceNow, this is an option at the very bottom of the Pagerduty config object in ServiceNow.

This may help you and the SNOW team diagnose the problem.

link here may also help

Make sure no one changed your servicenow integration on your service, check the extension version on the Pagerduty service, etc. There are multiple versions of the ServiceNow integration available for a Pagerduty service.

See here,

Login to Pagerduty -> Integrations -> Extensions …url is typically,

It will list the ServiceNow version for your specific services…

I doubt your versions have changed, unless, someone other than you, is making changes in your environment.

I raised a ticket with PagerDuty about it and they came back asking for some more information.

The following morning as I was gathering it, I could see things had started to work again about 12 hours after it stopped - with no intervention from me on either PD or ServiceNow.

Given that only part of the process was broken, it’s hard to pin where the fault might have been. Knowing nothing had changed in ServiceNow, my suspicion is PD. IF only there were some sort of logs that you could look at to see what goes on behind the scenes. I’ve asked support about this but have yet to hear anything back.

Hi Spike,

Noticed this post, yes, realize it’s May, you raised back In Feb.
Considering we use integration heavily.
if it stopped working for you? you would need to be more specific of the problem?

Within ServiceNow, you have the

1.Pagerduty Config/Settings in ServiceNow (pd url, pd API key; run the 3 tests, make sure http200s)
If something stopped working, this is one place to check

If all of that is good
2.Issue may be with a specific Assignment group within ServiceNow, check the group, verify the Pagerduty IDs on the group (Service, Escalation policy and webhook IDs) are valid
3.Verify if the issue was seen across multiple assignment groups? yes/no? or specific assignment group?
4.What’s not working? new PD incident, but no SNOW incident? the webhook disabled? checked the Extensions.
6.There is an additional logging feature, ‘debug’ on the pagerduty config in ServiceNow.

Some other ideas are here

Pretty sure, you resolved this by now!

I’m the sole ServiceNow and PD Admin and had already looked at all of the things you suggested. And yes it is working now, as my previous post said.
I never did hear back from PD support on some sort of log output.
Thanks for getting back to me.