Hi, my current problem is that once an incident is acknowledged, new incidents of the same severity (though different incidents) will no longer trigger an alert.
So, say, if I am alerted with a high severity alert and I acknowledge it instead of resolving, no new high severity incidents will alert.
Is there a way to change this such that each new incident will create an alert, as long as it’s not identical to an already acknowledged incident?
Thank you!
Hi @daniel s
Check the alert grouping settings on your service. It sounds like you want them to be less-groupy. Just to clarify, so you can find the right things in our docs, PagerDuty categorizes “alerts” as the data coming in from other sources; an alert can spawn an “incident”, which is really the primary object for most PagerDuty users or be added to an existing incident; and when an incident is created, “notifications” are sent to responders. It can definitely get confusing!
There’s a couple of things you can try, and you might want to experiment with some of the settings until you find a combination that is getting you what you want. Most folks prefer fewer incidents, so the tool mostly trends that way. Depending on your license level, you can try these:
Alert grouping in the service. This is under the “Settings” tab for a service. Click the “Edit” button in the “Reduce Noise” section and you’ll get a new screen:
More in the docs on that here. Effectively, new alerts to a service won’t create new incidents if they align with the grouping of an existing incident. If no new incidents are getting created you may be using the time-based alert grouping and you probably want to switch to Content or Intelligent grouping.
You may also be seeing the effects of Global Alert Grouping, which allows you to group incidents across services and reduce the total number of incidents. You can turn that off by removing any additional services from the grouping on the top of the “Reduce Noise” screen.
⚠️ there is also a lesser-known setting on each service to force the creation of incidents only. Most folks will not want to do this! but it is available. That setting is in the “Integrations” tab, at the bottom under “Alert and Incident Settings”. If you have integrations set up it will warn you about what’s not available if you disable alerts. I wouldn’t recommend doing this unless you’re in a place where you really, really don’t think you’re seeing the incidents you need to be seeing, in which case you can try it out and see what happens.
HTH,
–mandi
No account yet? Create an account
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.