In the configuration as code space, I’m building infra structure as code and configuration that will manage and control our config in PD (as much as possible).  The main driver is to manage the configuration of this critical piece of operations more securely and without the human-in-the-loop ClickOps via the PD portal.

I’m running into trouble using the PagerDuty Terraform provider when you use it to create multiple  PD resources (users, services, schedules,etc) in bulk (i.e. using for-each in the terraform resource)

We get failures back from the PD provider along the lines of:

│ Error: Provider produced inconsistent result after apply
│
│ When applying changes to
│ pagerduty_user.service_usersr"<redacted>@<email>"], provider
│ "providerr\"registry.terraform.io/pagerduty/pagerduty\"]" produced an
│ unexpected new value: Root object was present, but now absent.
│
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.

Question is - has anyone experienced something similar using the PagerDuty Terraform Provider and how did you work around it?

Sample Terraform Code

terraform {
  required_providers {
    pagerduty = {
      source  = "pagerduty/pagerduty"
      version = ">= 2.2.1"
    }
  }

  backend "azurerm" {
    use_msi = true
  }
}

variable "<redacted>_config" {
  type = string
}

variable "pagerduty_token" {
  type = string
}

provider "pagerduty" {
  token = var.pagerduty_token
}

locals {
  all_service_data = jsondecode(file(var.<redacted>_config))
  dev_oncall_users = local.all_service_data.serviceConfiguration.nonProduction.dev.onCallSchedule
  qa_oncall_users  = local.all_service_data.serviceConfiguration.nonProduction.qa.onCallSchedule
  service_oncall_users = setunion(
    local.dev_oncall_users,
    local.qa_oncall_users
  )
}

# Create a PagerDuty user
resource "pagerduty_user" "service_users" {
  for_each = { for user in local.service_oncall_users : user.email => user }
  name     = each.value.name
  email    = each.value.email
}

I note that all the PD terraform examples create resources one by one, which obviously can’t be used at scale in an enterprise.