Dataflow to Dataset Refresh - Extra Entity Column Not Automatically Included in Dataset

[Daryl-Lynch-Bzy]

Question - When a Dataset based on Dataflow is refreshed in the App.powerbi.com, should it automatically include changes like the addition of a new column on the Dataflow entity table? 

I have created a series of Dataflows to create a Star Schema from a Flatten Table.  The Dataflows work in the the following manner:

1. Extract - Read the series of Excel Tables into a temp storage dataflow
2. Transform Dimensions - creates several Entities that represent the dimension in star schema with the unique members, and a key is added.  for example Product, Office, Colour.
3. Transform Fact - removes the Dimension columns and replaces values with the Key identified in the Transform Dimension stage.

The Dataset then imports the Dimension and Fact dataflows to create the Data Model.

We had identified a new requirement to add a True / False flag to one of the Dimension entities.  I thought I could simply make this update to the Dimension Entity and then refresh the Dataset normally.  My dimension tables are created and used, so new Entity columns should be visible to Report Builders or people using the Dataset.

However, when I refreshed the Dataset on app.powerbi.com, it did not recognise the additional column.  So, I downloaded the PBIX file and refreshed it manually.  At this stage the Dataset did include the additional column on the entity table.  I then re-published the Dataset.

[Anonymous]

Hi @Daryl-Lynch-Bzy,

I would just like to know if you have found a way to do the above without republishig the pbix file? I am encountering the same problem right now and will appreciate it if you can share some insights. Thanks.

[Daryl-Lynch-Bzy]

Unfortunately no I did not find a solution that didn't involve repubishing the PBIX file.  I was hoping that Power BI would be more Dynamic and pull through the additional dimension data automatically.  However most of the time the additional column may require changes in the PBIX file anyway.  For example, the  Formatting or Sort Order setting, or prehaps the data would require a new measure.

[v-stephen-msft]

Hi @Daryl-Lynch-Bzy ,

Could you tell me if your problem has been solved?
If it is, kindly Accept it as the solution. More people will benefit from it.
Or you are still confused about it, I found two similar posts on the Issues forum. There are some solutions for you to refer to

https://community.powerbi.com/t5/Issues/Dataflow-error-Unable-to-create-new-Dataflow-or-edit-update/...

https://community.powerbi.com/t5/Issues/Dataflows-No-entity-after-refresh/idi-p/848917/page/2#commen...

Best Regards,
Stephen Tao

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[Daryl-Lynch-Bzy]

Hi Stephen,

No, it is not resolved, but it is not really an issue.  It just me want to be lazy and efficient as possible.  It just seem possible.  If changes made to the data contain in dataflow are made available when the subsequent dataset is refreshed, it would make sense that changes to the structure and data would be made available.  I just found it unusual that the new column added to a dataflow table, would not be automatically added when the dataset is refreshed.

However, the issue is avoided by exporting the PBIX to desktop, then refreshing, and then republishing the PBIX file.

I would like it to be more dynamic (one small change automatically deploys to the end), but i am just being lazy and efficient.

[v-stephen-msft]

Hi @Daryl-Lynch-Bzy ,

Maybe this article will help you to understand the process of dataflow refreshing.

Reference: Understanding and optimizing dataflows refresh

                  Dataflows limitations and considerations

Best Regards,

Stephen Tao

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[lbendlin]

"I have created a series of Dataflows to create a Star Schema from a Flatten Table "

I would recommend you read again what dataflows are for, how they are represented internally in the service, and how that contrasts with your normalization efforts.