New column added to Dataflow not displaying in Dataset

[S4CSurrey]

I have a pre-existing Dataflow that is calling data over REST. I have a Dataset created for this Dataflow, and a connected Report displaying data correctly.

Today, I edited the query in the Dataflow to expand out a new piece of data, a basic string type column. I refreshed the data, and the Power Query Editor is now displaying my table of data with the new column, and valid entries in the rows of this new column. 
After this was done, I saved in Power Query Editor, and no errors came up. I refreshed my Dataflow, and once completed, I refreshed the Dataset.

Opening the Dataset in Excel for analysis, I cannot see my my column from the Dataflow's data query.

What might be causing this. Any help is appreciated.

[denpries]

I have an exact similar issue to the one you describe.

I have added in a dataflow 3 new columns.

Aother, existing dataflow, combines this dataflow with another flow.

In the flow editor on the service you see the three new columns as last step. When refreshing, all goes well.

PBI desktop however cant see the new 3 columns, also not when you clear cache.

The only resolution is to e.g. add a useless replace operation in the query of the flow on an arbitrary column, not per se the 3 missing ones, 'e.g. replace X with identical X' . After refreshing the new columns also appear in powerbi desktop.

Strangely enough, when removing the useless replace again, the columns also stay.

What i think, is that something in the underlying scheme is not updated, until you force it to. (or something like that)

[retailbusiness]

the same issue here, I have to open the dataset on Desktop PBI and refresh from table "refresh data" option to display new columns from dataflow. Online dataset refresh won't update table schema! I don't know if it's normal behavior.

[chaz2jerry]

I think I have a similar issue and found the root cause.  In my scenario, I created a custom column in Decimal format.  Preview data in dataflow looks good, but the new column does not show up in the entity/table after refresh (as confirmed by connecting to this Dataflow table from PBI Desktop).  I had this issue a couple months back, and could not confirm why it was happening.  Then today I looked into the Advanced Editor (M Query) and found that the final "in" statement points to not the last query step as usual, but points to the step before I made the latest custom column.  Example of the M Query having the issue.  I assume this issue will be fixed when I adjust the M Query manually, not sure why it occurred in the first place.  

  #"step 6" = Table.TransformColumnTypes(#"Changed Type", {"SOLD_TO", type text}),

  #"step 7" = Table.AddColumn(#"Changed Type1", "Sales incl Tariff", each [Sales] + [TARRIF]),

  #"step 8" = Table.TransformColumnTypes(#"step 7", {{"Sales incl Tariff", Currency.Type}})

in

  #"step 6"

[tmason68]

You are a genius!

[v-lid-msft]

Hi @S4CSurrey ,

Could you please check if you can see the new column in a blank report based on the dataset? What is the data type of this new column , based on this document: A dataflow must output one of the following types: Date/Time, Decimal Number, Text, Whole number, Date/Time/Zone, True/False, Date, Time

Best regards,

[S4CSurrey]

I tried in a new report, not there still.

In Power Query, it is currently a Text type column. The source data is just a String type from my online database. Other string type fields are coming in and set to be a Text type column, and are working.

As a test, I set the column type to be Decimal, and the column's data still displayed properly. Switched back to Text, refreshed the data, and the column's data still displayed properly. Refreshed the Dataflow and Dataset, re-checked the new test report and analyzed it in Excel, and the column of data is still not accessible.

[v-lid-msft · Accepted solution]

Hi @S4CSurrey ,

Could you please check if you can see the new column when using the same query in Power BI Desktop?

Best regards,

[S4CSurrey]

I just tried with Power BI Desktop, and it does show up there! I was able to re-publish the report and it has updated the dataset as well.

Why is this required? Can't it all be edited online?

[maxcolaceci]

Hi All,

As far as I know, in order to have a new table/column available in a dataset, an user must first import the dataflow from Power BI Desktop, then publish the report to the Service. I.e., Power BI Desktop is the mean to turn dataset schema update (I don't know if there are others means).

I'm experimenting a similar issue:

1) the IT staff added a column to an existing table of a dataflow already used by a published dashboard

2) I can see the new column in the dataflow query editor and run a refresh successfully but...

3) ...when I import the table in PBI Desktop, the new column is not available

I did some testes:

a) I created a new dataflow, add the table, saved, refreshed and imported in a PBI Desktop file. However, I don't want to use a brand new dataflow for each change of table/column, so...

b) ...so I edited the existing dataflow, added the table (which stays beside the existing old one), replaced the code of existing table with the new one. Kind ok work around, I'd like to have some more straightforward. But it works.

Thanks

[v-lid-msft]

Hi @S4CSurrey ,

We want to check if the error is exist only on the old dataflow, If the same query works in Power BI Desktop, we think if using the same query in a new data flow entity, it will show the new column. Could you please check if this issue only exist in the spetific old dataflow?

Best regards,

[S4CSurrey]

Sorry, I am unsure how to check this. I only am using a single dataflow.

[BSLATTER]

Hello, Were you able to resolve this issue? My team is currently experiencing the exact same scenario you were and the provided "Solution" doesn't really seem to be solving it.