SolarWinds Alerts Not Being Sent to PagerDuty After Upgrade

solarwinds
official

(Jonathan Curry) #1

If you’ve recently upgraded SolarWinds Orion products - like NPM, SAM, or WPM - to the new 2016 version, and you use the VBScript-based integration to send alerts to PagerDuty, you may have run in to an issue with alerts being stuck in the C:\PagerDuty\Queue directory and never making their way to PagerDuty.

This happens because the latest version of SolarWinds now saves its alert files with a UTF-8 BOM, which the original VBScript tries to send to our API as-is, however the BOM characters make the API call invalid and result in PagerDuty rejecting the event data.

Here’s an example log from an alert file with a UTF-8 BOM that was read by the VBScript and sent to PagerDuty:

PagerDuty did not accept event with data:

{
	"service_key": "super-secret-integration-key",
	"incident_key": "server123 Down",
	"event_type": "trigger",
	"description": "server123 is Down",
	"client": "SolarWinds",
	"client_url": "http://orion.example.com:8787/Orion/NetPerfMon/NodeDetails.aspx?NetObject=N:1"
}

Response was:

[400] {"status":"unrecognized object","message":"Event object format is unrecognized","errors":["JSON parse error"]}

Previous versions of SolarWinds saved alerts in plain ASCII, and VBScript opens files as ASCII unless otherwise specified, so the  you see before the opening brace { is visible to the VBScript in this mode even though they are not visible in UTF-8 aware apps like Notepad.

To resolve this, we’ve updated the VBScript for the new version of Orion so that the script reads alert files generated by SolarWinds in UTF-8 mode rather than ASCII mode. Simply replace the old script with this new one and you should be good to go.

Note: You may want to clear the alert files that have accumulated in C:\PagerDuty\Queue first, otherwise these will be sent to PagerDuty the next time the VBScript is executed after you download the updated version, regardless of how old the alerts are.


(Diego Faria) #2

Hi Jonathan,

I used your script, but i’m receiving the same error.

I’m using version 2017.1.3


(Ashley Brooks) #3

Hey Diego - My colleague tells me he’s currently working with you to resolve this issue via chat, but I wanted to follow up here as well for anyone else watching:

It looks like the JSON payload sent through the VBScript is invalid. "Last Time \\"Up\\"": has escaped backslashes which prematurely terminate the string. I would recommend removing the backslashes and double quotes around “Up” entirely and the JSON should be valid.

Let us know if you have any other trouble and we’d be happy to take a look!


(system) #4