Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
Anonymous
Not applicable

All tables to dataflows

Hi, 

 

We have mulitple reports using some similar tables and we are thinking about using dataflows as source for the reports.

Although the tables are the same, some parts of the queries are different for different reports. We are thinking about making mulitple dataflows of one table for this reason.

Should I make multiple dataflows with different queries for one table and use the dataflow that is specific for the report?
Does the make of different/mulitple dataflows affect capacity, or more than without dataflows???


1 ACCEPTED SOLUTION
v-eachen-msft
Community Support
Community Support

Hi @Anonymous ,

 

1. Different queries need different dataflows even though you used the same source.

2. When refreshing a dataflow, the dataflow workload spawns a container for each entity in the dataflow. Each container can take memory up to the volume specified in the Container Size setting. The default for all SKUs is 700 MB. You might want to change this setting if:

  • Dataflows take too long to refresh, or dataflow refresh fails on a timeout.
  • Dataflow entities include computation steps, for example, a join.

It's recommended you use the Power BI Premium Capacity Metrics app to analyze Dataflow workload performance.

In some cases, increasing container size may not improve performance. For example, if the dataflow is getting data only from a source without performing significant calculations, changing container size probably won't help. Increasing container size might help if it will enable the Dataflow workload to allocate more memory for entity refresh operations. By having more memory allocated, it can reduce the time it takes to refresh heavily computed entities.

The Container Size value can't exceed the maximum memory for the Dataflows workload. For example, a P1 capacity has 25GB of memory. If the Dataflow workload Max Memory (%) is set to 20%, Container Size (MB) cannot exceed 5000. In all cases, the Container Size cannot exceed the Max Memory, even if you set a higher value.

If you'd like to refresh the dataset with a large model, you might refer to the official document: You can use incremental refresh to configure a dataset to grow beyond 10 GB

https://docs.microsoft.com/en-us/power-bi/service-premium-large-models 

Also, if you'd like to split the dataset into several dataflow and run the number of parallel model refreshes. Kindly note the below limitations:

https://docs.microsoft.com/en-us/power-bi/service-premium-what-is#capacity-nodes 

 

Community Support Team _ Eads
If this post helps, then please consider Accept it as the solution to help the other members find it.

View solution in original post

1 REPLY 1
v-eachen-msft
Community Support
Community Support

Hi @Anonymous ,

 

1. Different queries need different dataflows even though you used the same source.

2. When refreshing a dataflow, the dataflow workload spawns a container for each entity in the dataflow. Each container can take memory up to the volume specified in the Container Size setting. The default for all SKUs is 700 MB. You might want to change this setting if:

  • Dataflows take too long to refresh, or dataflow refresh fails on a timeout.
  • Dataflow entities include computation steps, for example, a join.

It's recommended you use the Power BI Premium Capacity Metrics app to analyze Dataflow workload performance.

In some cases, increasing container size may not improve performance. For example, if the dataflow is getting data only from a source without performing significant calculations, changing container size probably won't help. Increasing container size might help if it will enable the Dataflow workload to allocate more memory for entity refresh operations. By having more memory allocated, it can reduce the time it takes to refresh heavily computed entities.

The Container Size value can't exceed the maximum memory for the Dataflows workload. For example, a P1 capacity has 25GB of memory. If the Dataflow workload Max Memory (%) is set to 20%, Container Size (MB) cannot exceed 5000. In all cases, the Container Size cannot exceed the Max Memory, even if you set a higher value.

If you'd like to refresh the dataset with a large model, you might refer to the official document: You can use incremental refresh to configure a dataset to grow beyond 10 GB

https://docs.microsoft.com/en-us/power-bi/service-premium-large-models 

Also, if you'd like to split the dataset into several dataflow and run the number of parallel model refreshes. Kindly note the below limitations:

https://docs.microsoft.com/en-us/power-bi/service-premium-what-is#capacity-nodes 

 

Community Support Team _ Eads
If this post helps, then please consider Accept it as the solution to help the other members find it.

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors