From calling the RestAPI https://api.pagerduty.com/users
I can only get the first 25 users, how can I get all the users from via calling api?
How can I get all the users from calling the API?
Hello,
Thanks for reaching out on our Community. Sure thing, you can read up more on why this is happening here. Many index endpoints will be paginated, meaning that only a certain number of resources will be returned at a time. The default is 25. You can override this by passing a limit parameter to set the maximum number of results, but cannot exceed 100.
Hope this makes sense, let us know if not.
John
Thanks John, added the parameters and got the list.
Brilliant!
You’re very welcome !
Hi Gongjunhuang,
In the spirit of community, could you share the approach you took with your script/code so others may benefit in the future? (be sure to not include any API tokens, etc.)
Thanks!
Hi Doug,
The code is:
def writeToExcel():
headers = {
"Accept": "application/vnd.pagerduty+json;version=2",
"Authorization": "Token token=**token**"
}
excel = openpyxl.Workbook()
sheet = excel.get_active_sheet()
sheet.column_dimensions['A'].width = 100
sheet.column_dimensions['B'].width = 100
sheet.column_dimensions['C'].width = 100
sheet['A1'] = 'Name'
sheet['B1'] = 'Email'
sheet['C1'] = 'Avatar_url'
for offset in range(0, 2000, 100):
url = 'https://api.pagerduty.com/users?limit=100&offset={0}'.format(offset)
r = requests.get(url, headers=headers)
json = r.json()['users']
for each in range(len(json)):
sheet['A' + str(each + offset + 2)] = str(json[each]['name'])
sheet['B' + str(each + offset + 2)] = str(json[each]['email'])
sheet['C' + str(each + offset + 2)] = str(json[each]['avatar_url'])
excel.save('PageDuty.xlsx')
Note that the talented @demitri has written a Session-aware PagerDuty API SDK in Python which can automatically handle the pagination with functions like iter_all
.
Available here https://pagerduty.github.io/pdpyras/
Installable via pip: pip install pdpyras
I had previously written a custom class to (poorly) implement the pagination and some other helper functions, all of which were replaced simply and efficiently by including pdpyras!
Hope that helps,
@simonfiddaman
Just to add a few extra side notes here on pagination
All the currently documented index endpoints in the PagerDuty REST API should both accept a total
parameter in the request, and respond with a more
parameter in the body of the response. See: Pagination
When implementing PagerDuty REST API pagination in any language, either of these can be used to automatically stop requesting pages when reaching the end of the dataset. PDPYRAS uses the more
parameter to figure out when it has reached the end (it should be false
on the last page)
An edge case to watch out for: if there are more than 10,000 records in the query. When this happens, one cannot increment
offset
such thatoffset
+limit
exceeds 10k; a status 400 will result.If operating on the data in such a way that the action taken on each result removes it from the dataset (i.e. resolving >10k incidents, and filtering for unresolved incidents) an approach to circumvent this is to pre-fetch a few thousand at a time, act on them, and then start over until the condition
more=false
is encountered before the arbitrary few-thousand results limit.
Thanks @simonfiddaman for the shout-out!