cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
gmckinnon
Frequent Visitor

Power BI premium Large Dataset performance

Hi, we have a large dataset (around 8gb compressed) hosted in a P1 capacity still currently in premium Gen1. Currently we load a subset of the data into pbi desktop and use incremental refresh to load the historical data and new data. 

 

The dataset will be growing and we want to work out how scaleable this solution is, esspecailly when we move to premium gen 2 and memory is less of an issue and we just need to worry about CPU.

 

The biggest issue we see right now is when the dataset needs to be loaded into memory which currently takes about 10-15 seconds and seeing spikes in the CPU useage of around 80% during this time.

 

If the dataset was to grow, how would it perform? Would we need to revert to filtering the data or using direct query (which was terribly slow). If the dataset grew to say 25gb (the max for a p1) could it overload the CPU when moving in and out of memory in the service?

1 ACCEPTED SOLUTION
TomMartens
Super User II
Super User II

Hey @gmckinnon ,

 

this is a difficult question, but there is at least one very simple answer.

Keep in mind that you can't leverage the full 25GB of the available dataset size, as there is extra memory needed during processing.

 

Behavior will change with Premium Gen 2 (What is Microsoft Power BI Premium? - Power BI | Microsoft Docs), i recommend to turn on the preview feature, and gather your experiences, before you migrate to PE2.

 

Regards,

Tom



Did I answer your question? Mark my post as a solution, this will help others!

Proud to be a Super User!
I accept Kudos 😉
Hamburg, Germany

View solution in original post

1 REPLY 1
TomMartens
Super User II
Super User II

Hey @gmckinnon ,

 

this is a difficult question, but there is at least one very simple answer.

Keep in mind that you can't leverage the full 25GB of the available dataset size, as there is extra memory needed during processing.

 

Behavior will change with Premium Gen 2 (What is Microsoft Power BI Premium? - Power BI | Microsoft Docs), i recommend to turn on the preview feature, and gather your experiences, before you migrate to PE2.

 

Regards,

Tom



Did I answer your question? Mark my post as a solution, this will help others!

Proud to be a Super User!
I accept Kudos 😉
Hamburg, Germany

View solution in original post

Helpful resources

Announcements
PBI User Groups

Welcome to the User Group Public Preview

Check out new user group experience and if you are a leader please create your group!

MBAS on Demand

Microsoft Business Applications Summit sessions

On-demand access to all the great content presented by the product teams and community members! #MSBizAppsSummit #CommunityRocks

MBAS Attendee Badge

Claim Your Badge & Digital Swag!

Check out how to claim yours today!

Top Solution Authors