Skip to main content

We have installed and configured the PagerDuty App for Splunk version 4.0.1 and it appears to be working properly. I am a Splunk administrator and have created a test alert that leverages the PagerDuty integration and it works flawlessly. I am able to see the new incident in PagerDuty as well as receive a text on my phone.


We have granted a few other teams access to add PagerDuty as a Trigger Action in Splunk Alerts, however when triggered there are no new incidents in PagerDuty and no phone texts. We know that their alert is triggering as they are receiving emails showing that the alert has fired.


I checked in the _internal index in Splunk and I see the following error occurs when the alert fires:


04-25-2023 17:06:20.002 +0000 ERROR sendmodalert [2351978 AlertNotifierWorker-0] - action=pagerduty STDERR - splunklib.binding.HTTPError: HTTP 403 Forbidden – You (user=xxxxxxxxxx@costcotravel.com) do not have permission to perform this operation (requires capability: list_storage_passwords OR edit_storage_passwords OR admin_all_objects).


I assume that Splunk grabs the hash key from storage/passwords to make the API call to PagerDuty. Why is the capability set at the user level and not at the app level? We do not want to grant any of these capabilities to our users. Is there a way to get around this?

Hi Tom. Our Engineering team is working on this and waiting for info back from Splunk.


They’ll be updating our integration docs soon, with the permissions requirement:



For the latest version 4.0.1, any user that will setup Alerts to notify PagerDuty require the necessary Splunk permission list_storage_passwords to be able to retrieve the App secret from secret storage, according to the official documentation.


Thank you Mandi, however that doesn’t fix our issue since we do not want to grant users the list_storage_paswords capability in Splunk. Per our conversation with our Splunk TAM, he advises us not to grant that capability to users since they could potentially view password data. IMO, the ability to obtain the password / hash should be set at the Splunk App level and not the User Role level.

Hi Tom,


The current process was developed under the guidance given by Splunk at the time. We are more than happy to meet with you and the Splunk TAM to clear this.

Reply


Terms & ConditionsCookie settings