Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.

Reply
ek2112
Advocate II
Advocate II

Retrieving data by working around REST API limitations through dynamic web query

Hi all,

 

I have a data source with timelog entries, with about 30,000 records. Until now, I was able to get the data out of the data source through direct authentication that fetched all the records through a single query that returns values as JSON (Get data -> From web option). 

 

Now, the data source is changing their API and limiting the records fetched through a single REST call to 1000 and they are rolling this out starting 1st week of June.

 

I can get all the records and load them into Power BI through, say, 6/4 using the existing authentication. My question (or rather questions) as follows:

 

a) Going forward, once the authentication changes with the aforementioned limiting effect of fetching only 1000 records at a time, how do I dynamically modify the source url to get worklogs on a weekly basis, say from 6/4 to 6/10 and append them to the existing data?

b) How to do this on a weekly basis?

 

Is there a way to dynamically update the Web.Contents() URI and append the resultset to the table? Or through parameters or both? Thank you for your time.

1 ACCEPTED SOLUTION
v-chuncz-msft
Community Support
Community Support

@ek2112,

 

You may take a look at the post below.

https://community.powerbi.com/t5/Desktop/Stopping-when-looping-through-a-paginated-API-feed/m-p/3103...

Community Support Team _ Sam Zha
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

2 REPLIES 2
v-chuncz-msft
Community Support
Community Support

@ek2112,

 

You may take a look at the post below.

https://community.powerbi.com/t5/Desktop/Stopping-when-looping-through-a-paginated-API-feed/m-p/3103...

Community Support Team _ Sam Zha
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

@v-chuncz-msft This got me thinking...and helped. Thanks.

Helpful resources

Announcements
Microsoft Fabric Learn Together

Microsoft Fabric Learn Together

Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City

PBI_APRIL_CAROUSEL1

Power BI Monthly Update - April 2024

Check out the April 2024 Power BI update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.