Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
MalJan19
Frequent Visitor

Dataflow Error in Premium Workspaces

We had a little bit of an emergency yesterday concerning our dataflows. Here's the chain of events:
1. We recently got Power BI Premium, and I put one of our workspaces containing many dataflows in a Premium capacity. Everything was fine for a couple days.

2. I went into the capacity settings and adjusted Max Memory in the capacity settings for dataflows and datasets.

3. A few hours later, datasets using these dataflows failed to refresh. We also noticed that we could not connect to any of these dataflows in a new .pbix file (Error: Dataflow Internal Error). 

4. I changed the capacity settings back to what they had been. The error persisted.

5. I took the workspace off Premium capacity. The error persisted. 

6. I tried refreshing the dataflows once the workspace was no longer premium, and they all failed to refresh (Error: There was a problem refreshing your dataflow).

7. Since the Premium capacity was only a test capacity, I deleted the capacity itself. When we tried to refresh the dataflow, it threw an error message saying we needed to assign a capacity to the workspace.

8. We created a new premium capacity and assigned it to the workspace. Everything worked fine.

 

Why did this happen??? It makes me nervous to use Power BI Premium, and especially to adjust the workloads in the capacity settings. Has this happened to anyone else?

3 REPLIES 3
jeffshieldsdev
Solution Sage
Solution Sage

Are you using the Enhanced Dataflows Compute Engine?  I've had issues with large queries failing so I've disabled this preview feature.

 

You can use this calculator to see how many containers you can get with allocated memory to dataflows and to containers: http://petcu40.blogspot.com/2019/06/premium-dataflow-workload-parameters.html

 

We're not using many datasets in Premium, so I've dialed that down to 80%, increased dataflows to 80%, disabled EDCE and set container size to 2400 Mb for (hopefully) 8 max containers.

Anonymous
Not applicable

Certain dataflow features aren't supported when dataflow storage is in Azure Data Lake Storage Gen2:

Power BI Pro, Premium and Embedded workspaces:

  • The linked entities feature is only supported among workspaces in the same storage account
  • Workspace permissions do not apply to dataflows stored in Azure Data Lake Storage Gen2; only the owner of the dataflow can access it.
  • Otherwise, all data prep features are the same as for dataflows stored in Power BI storage

There are some additional considerations as well, described in the following list:

  • Once a dataflow storage location is configured, it cannot be changed.
  • Only the owner of a dataflow stored in Azure Data Lake Storage Gen2 can access its data.
  • On-premises data sources, in Power BI Shared capacities, are not supported in dataflows stored in your organization’s Azure Data Lake Storage Gen2.

Thanks for the reply, Lewis-H. Some of your suggestions make sense, but none of them fit exactly:

1. We aren't using any linked entities.

2. I am the owner of the dataflows, and during this period I could not even open the dataflow to see the list of columns in each table.

3. What is meant by the dataflow storage location cannot be changed? I don't think this is the issue, because the dataflows were fine for a couple days after putting the workspace in a premium capacity, but broke only after I adjusted the dataflow settings in the capacity.

4. Could you clarify your last point? Our dataflows do connect to an on-prem data warehouse, but they worked completely fine when on a shared capacity and a premium capacity until I adjusted the dataflow workload settings. 

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors