Power BI Service, Dataflows, Dataset: Data sometimes partly missing

[lukmtb08]

Hi, 

we have a dataflow architecture, factory dataflows are all referenced in major dataflows. So all factory can simply refresh their data as soon as they have entered it in Excel files stored in SharePoint. 

Sometimes my users are complaining that data is partly missing thta has already been available. But I am not understanding why. After refreshing a flow it is visible again. It seems that sometimes the data of the whole flow get´s lost without any new refresh. This really big problem. After refreshing the dataflow, is is visible again. I thought that the data is kind of stored within a dataflow, as long as the flow will be refreshed.

Another problem which might be related to this. If I publish a new report with the same name, I got the following error: 

The refresh of the dataset works, but only about 30 min after publishing. 

But the report does not open at all:

Do you know if that is a known issue? 

BR

[lukmtb08]

Thank you very much for your support. 

By the way: I totally understand your point and I a totally agree with you. If central IT has DB migrations ongoin for years and now data lake connection is possible, Excel & Dataflows & Power BI are a very cool alternative 🙂

[lukmtb08]

Thank you four answer! Scheduled refresh in disabled. We are having a premium workspace. The Users have a Pro licencnse. About 20 people are refreshing dataflows and the dataset manually as soon as they need to show actual figures. 

Unfortunatetly, I can not schedule the dataflow refresh automatically, as they then fail. All flows are referenced and as soon as two start at the same time, the second fails. 

[lbendlin]

Do you have multiple developers refreshing the dataflows?  Do you have incremental refresh enabled?

Most of the time these report  issues are caused by meta data mismatch between the workspace and the app in a Premium environment.  The remedy is to update the app.

What environment are you working in, shared or Premium?

[lukmtb08]

Thank you for your answer. About 20 people are refreshing the flows. 

My report is part of premium workspace. 

What do you mean by "App" the report or an Power BI App? Should I delete the dataset and republish it again? This would mean that my users would need a new link every time as they are not all part of the workspace but have access via the link. 

Thank you for your support.

[lbendlin]

About 20 people are refreshing dataflows

Hopefully this is only a terminology issue.  I was asking how many people are updating the data in the dataflow, not how many people are consuming the data.

What do you mean by "App" the report or an Power BI App?

I mean the app. No GUIDs will be harmed in the process of updating the app.

[lukmtb08]

We have 13 factories, every factory has two files, which are collecting data from Excel files. 
All flows are referenced at one major flow. This major flow is the fact table of my report.

About 20 different people are able to refresh the diferrent flows individually, to bring new data to our dataset.  

Except the "metadata"-issue, everything is working perfectly. 

[lbendlin · Accepted solution]

wow, that is ... inventive.   Certainly pushing the boundaries.

Anyway - the metadata issue is likely unrelated, and should be addressable with app updates.

Beware - every time you change the structure of one of your upstream data sources* this will require another meta data re-sync.

* Especially when you remove or rename columns. Adding columns _sometimes_ works without the need to update.