cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Highlighted
Frequent Visitor

Azure server running Power BI: Export / import data tables to files on server to reduce API traffic?

Hello folks,

 

Can you please tell me if the following is possible in Power BI?

My situation:
We're using PBI to develop reports for our customers based on a number of data streams coming form physical sensors. This data is stored in a database which we access througfh an API. The amount of data grows (at least) linearly with time, and our customers need to be able to view all data through the reports. This is a huge problem, as the workload on the database server and the refresh lag increases steadily over time and will eventually exceed the physical capabilities of the server or the patience of our customers. Surely, others must already have experienced or are anticipating a similar challenge?

An attempt at a solution:
After each refresh, a query saves the content of the data tables locally in a .csv or excel file on the report server. The content of this file is then loaded into the tables on refresh, and a query is sent to the API for just the newest data not already in the files. So, effectively, this would work as an incremental refresh, heavily reducing the workload on the database server.

 

Could this work? Is there already an implemented solution for this challenge that I might have overlooked?

 

Cheers, Mike

Helpful resources

Announcements
August 2020 Community Challenge: Can You Solve These?

August 2020 Community Challenge: Can You Solve These?

We're excited to announce our first cross-community 'Can You Solve These?' challenge!

Community Blog

Community Blog

Visit our Community Blog for articles, guides, and information created by fellow community members.

Upcoming Events

Upcoming Events

Wondering what events you could join or have an event to promote yourself? Check out our Upcoming Events.

Get Ready for Power BI Dev Camp

Get Ready for Power BI Dev Camp

We are thrilled to announce we will begin running a monthly webinar series named Power BI Dev Camp.

Top Solution Authors