cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Highlighted
Microsoft
Microsoft

Using a Loop to Get Paginated Data in a Non-ODATA Rest API in Power Query

I am a newbie in the API space, and I am trying to pull all the paginated data from UserVoice.  They have a limit of 100 results per page, and I can't use the ?limit syntax to increase the limit.  I have this so far in Power Query, but it is giving me the same results over and over again.  Can anybody assist?

 

let

url = "https://SUBSITE.uservoice.com/",

Header=[Authorization="TOKEN"],

RelativePathing = "api/v2/admin/suggestions",

Source1 = Json.Document(Web.Contents(url, [RelativePath=RelativePathing, Headers=Header])),

iterations = Source1[pagination][total_pages],

FullURL = url & RelativePathing,

 

FnGetOnePage =

(FullURL) as record =>

let

Source2 = Json.Document(Web.Contents(url, [RelativePath=RelativePathing, Headers=Header])),

data = try Source2[suggestions] otherwise null,

next = try Source2[pagination][cursor] otherwise null,

res = [Data=data, Next=FullURL & "?cursor=" & next]

in

res,

 

GeneratedList =

List.Generate(

()=>[i=0, res = FnGetOnePage(FullURL)],

each [i]<iterations and [res][Data]<>null,

each [i=[i]+1, res = FnGetOnePage([res][Next])],

each [res][Data])

in

   GeneratedList

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted
Super User IV
Super User IV

Re: Using a Loop to Get Paginated Data in a Non-ODATA Rest API in Power Query

@glove80 hope this post helps.

 

I would 💖Kudos 🙂 if my solution helped. If you can spend time posting the question, you can also make efforts to give Kudos whoever helped to solve your problem. It is a token of appreciation!

 

 






Did I answer your question? Mark my post as a solution.

Proud to be a Super User! Appreciate your Kudos 🙂
Feel free to email me with any of your BI needs.





View solution in original post

2 REPLIES 2
Highlighted
Super User IV
Super User IV

Re: Using a Loop to Get Paginated Data in a Non-ODATA Rest API in Power Query

@glove80 hope this post helps.

 

I would 💖Kudos 🙂 if my solution helped. If you can spend time posting the question, you can also make efforts to give Kudos whoever helped to solve your problem. It is a token of appreciation!

 

 






Did I answer your question? Mark my post as a solution.

Proud to be a Super User! Appreciate your Kudos 🙂
Feel free to email me with any of your BI needs.





View solution in original post

Highlighted
Community Support
Community Support

Re: Using a Loop to Get Paginated Data in a Non-ODATA Rest API in Power Query

HI @glove80,

You can also take a look at sample helper functions that shared on official documents:

Helper Functions#tablegeneratebypage 

Regards,

Xiaoxin Sheng

Community Support Team _ Xiaoxin
If this post helps, please consider Accept it as the solution to help the other members find it more quickly.

Helpful resources

Announcements
Community Blog

Community Blog

Visit our Community Blog for articles, guides, and information created by fellow community members.

Using the Community

Using the Community

Need help with the Power BI Community? Our 'Using the Community' support articles are a great place to start.

Community Summit North America

Community Summit North America

Innovate, Collaborate, Grow. The top training and networking event across the globe for Microsoft Business Applications

Power Platform 2020 release wave 2 plan

Power Platform 2020 release wave 2 plan

Features releasing from October 2020 through March 2021

Top Solution Authors
Top Kudoed Authors