Access token used for Service Extension no longer accessible from REST API

Hi,

we use Slack extensions on more than a hundred Pagerduty services and up to now were managing them using the method described here:

We used this method through the Terraform Pagerduty provider as described here: https://github.com/terraform-providers/terraform-provider-pagerduty/issues/94#issuecomment-449075707

The issue is this does not seem to be working any more since yesterday as the token is now redacted from the REST API responses:

From the terraform point of view, this triggers permanent diff:

# terraform plan

  # module.pagerduty.pagerduty_extension.slack_nbh["MyService103"] will be updated in-place
  ~ resource "pagerduty_extension" "slack_nbh" {
      ~ config            = jsonencode(
          ~ {
              ~ access_token     = "redacted" -> "xoxp-******-******-******-******"
                bot              = {
                    bot_user_id = "U******1"
                }
                enterprise_id    = null
                incoming_webhook = {
                    channel           = "#fr-******-pagerduty"
                    channel_id        = "C******X"
                    configuration_url = "https://******.slack.com/services/******"
                    url               = "https://hooks.slack.com/services/******/******/******"
                }
                notify_types     = {
                    acknowledge = false
                    annotate    = true
                    assignments = false
                    escalate    = true
                    resolve     = false
                    trigger     = true
                }
                ok               = true
                referer          = "https://******.pagerduty.com/services/******/integrations"
                scope            = "identify,bot,commands,incoming-webhook,channels:read,groups:read,im:read,team:read,users:read,users:read.email,channels:write,chat:write:user,chat:write:bot,groups:write"
                team_id          = "T******Y"
                team_name        = "******"
                urgency          = {
                    high = true
                    low  = true
                }
                user_id          = "U******6"
            }
        )
        extension_objects = [
            "P******7",
        ]
        extension_schema  = "P******R"
        id                = "P******P"
        name              = "MyService103 - #fr-******-pagerduty in ******.slack.com"
    }

Plan: 0 to add, 103 to change, 0 to destroy.

Sadly, all our Slack V2 extensions that shared a previously valid access_token stopped working all at once and we cannot re-authorize them through the API.

Is there any other known method to retrieve an access token?

Alright, the new method to retrieve this access token is during the OAuth re-authentication phase in an actual browser:

  1. In Firefox, browse to a service integrations page, e.g. https://******.pagerduty.com/services/P*****C/integrations,
  2. On a Slack V2 extension, open the drop down menu by clicking the gear icon,
  3. Click Re-authorize,
  4. Open the Developer Tools with F12,
  5. Execute this in the Developer Tools Console: document.body.getElementsByTagName("script")[1].text
  6. Extract the token value from the text, it should look like xoxs-************-************-************-****************************************************************
  7. Select the target Slack channel and click Allow.
2 Likes

I ran into the exact same problem. What should be done with the xoxs token then?

I mean, did you find a way to use this new token to re-authorize all the extensions using the API/Terraform?

Thanks a lot for posting this. It’s very helpful.

My understanding is that you should replace “access_token” with the new “xoxs-*” token inside the config section, but I couldn’t get it to work myself.

We have faced exactly the same issue on Feb 6th, I have manually re-authorised extension for one of the services while getting new token. However, when I use that token as “access_token” to propagate config via Terraform for other services it gets applied successfully, but not Slack alerts are created when incident is triggered.

Obscuring “access_token” value makes it very difficult to troubleshoot any further, so help will be much appreciated.

Thanks.

Hello all,

Thank you for bringing this to our attention! If you haven’t already, could you open tickets with us in Support by emailing support@pagerduty.com and referencing this community post?

We’re happy to investigate here!

Nick - Support

Please see #240748. Many thanks.

When I tried on Feb 7th, it worked for some but not all, so I ended up re-authorizing all slack extensions by hand as I was leaving for a one week vacation.

Got the information something was fixed by Pagerduty engineering team during my vacation (ticket #240113).

I’m planning to perform more tests soon.

One thing to expect with terraform for now until some better solution is implemented is permanent diff as the token is redacted.