Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.
Hi There,
I have been using the following tutorials to pull all paginated records from a Web API.
Blog: https://www.mattmasson.com/2014/11/iterating-over-an-unknown-number-of-pages-in-power-query/
Video: https://www.youtube.com/watch?v=vhr4w5G8bRA&t=6s
My code:
GetData Function:
(Page as number) => let Source = Json.Document(Web.Contents("https://churchmetrics.com/api/v1/records.json?page=" & Number.ToText(Page) & "&per_page=100" , [Headers=[#"X-Auth-User"="xxxxxxxxxxxxxxxxxx", #"X-Auth-Key"="xxxxxxxxxxxxxxxx"]])) in Source
This invokes fine with page 1 (data) and returns records, and page 4 (no data) returns list with no rows.
Query to pull all pages together:
= List.Generate( ()=> [Result= try GetData(1) otherwise null, Page = 1], each [Result] <> null, each [Result = try GetData([Page]+1) otherwise null, Page =[Page]+1], each [Result])
Now when I refresh thi,s it takes a long time, but then produces - what seems like infinate lists (over 1,000). As tested in the function lists over 3 are empty. The tutorials I used used the query to, basciallly, if error then null, if null then stop. But the issue I am facing is that the query doesn't throw up an error it just provides a blank list. This seems like an easy fix but I cant figure it out.
All help would be apreciated.
Thanks in advance.
Hi @DesignerMat,
For any concern about the used M function in the tutorial, you could add comments and ask questions at the bottom, I think the author should be more familar with it and give you advice.
Regards,
Yuliana Gu
Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City
Check out the April 2024 Power BI update to learn about new features.
User | Count |
---|---|
109 | |
98 | |
77 | |
66 | |
54 |
User | Count |
---|---|
144 | |
104 | |
100 | |
86 | |
64 |