New column in source not appearing in dataset

[nephologist]

I recently added another column to the .csv files I'm using as source for a dataset, and Power BI will not read the new column.

It's a similar problem with the one posted over here:

https://community.powerbi.com/t5/Desktop/New-column-in-source-table-not-appearing-in-dataset-after/t...

Unfortunately, I'm drawing the .csv files from an entire folder, so my Advanced Editor starts off with something like this:

Source = Folder.Files("some path"),

so the solution in the post doesn't really work for me.

How can I make Power BI read all the columns in each .csv file?

Thanks in advance.

[parry2k]

If I understood your question correclty, you have multiple csv files with same data structure and you want to read all those files, correct?

[nephologist]

Correct.

And the .csv files were being read before just fine, until I added a new column to each of them. Now the new column is not being read.

[Greg_Deckler]

Open up Advanced Editor in Query Editor, find the Source line and change the number of columns that are specified.

So, if your Source line is this:

    Source = Csv.Document(File.Contents(OneSource1),[Delimiter=",", Columns=4, Encoding=1252, QuoteStyle=QuoteStyle.None]),

And you add a column, change it to this:

    Source = Csv.Document(File.Contents(OneSource1),[Delimiter=",", Columns=5, Encoding=1252, QuoteStyle=QuoteStyle.None]),

[parry2k]

do you have remove column or select column steps in your query editor? May be you need to revisit one of those steps.

[nephologist]

@parry2k Yes.

The steps I had before are as follows:

Source = Folder.Files("D:\file path\FILE NAME"),
#"Invoke Custom Function" = Table.AddColumn(Source, "Transform File from FILE NAME", each #"Transform File from FILE NAME"([Content])),
#"Renamed Columns" = Table.RenameColumns(#"Invoke Custom Function", {"Name", "Source.Name"}),
#"Removed Other Columns" = Table.SelectColumns(#"Renamed Columns",{"Source.Name", "Transform File from FILE NAME"}),

I don't see if the select function is somehow choosing or limiting the number of columns in the source...

@Greg_Deckler I can't use that solution since I'm extracting the files from an entire folder, and the Advanced Editor doesn't start with "Source = Csv.Document"

[parry2k]

So from your query it looks like you are calling a function to read data from each file. You need to check that function to make sure it is reading all the columns, and change the query in the function as @Greg_Deckler suggested or look if you remove/selected column step in that function.

[Greg_Deckler · Accepted solution]

OK, when you created your Folder query, it should have created a folder in Query Editor called something like "Transform File from csvs". In that folder should be a "Transform Sample File from csvs". Click on that and then click on Advanced Editor and edit the Source line in that query as specified above.

[nephologist]

Sweet, that worked. Thank you.

So I guess every time I add something to the source I have to do this all over again, right? Is there a way to automatize it? I'm thinking, instead of "Columns=5" or any number, "Columns=All"?

[MarcelBeug]

"Columns" is optional, so you can leave it out completely.