Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Grow your Fabric skills and prepare for the DP-600 certification exam by completing the latest Microsoft Fabric challenge.

Reply
Ria_
Frequent Visitor

Adding columns to some, but not all , files in a folder

Hello,

 

I connect my PBI desktop to a folder where files are delivered daily. For this example lets say files 1,2 and 3 all have columns A,B and C on them. 

 

Now the client wants a new KPI displayed and added Column D on to the new files they deliver daily to the connected folder. 

 

So Files 1, 2 and 3 have columns A, B and C. 

And files 4, 5 and 6 have columns A, B, C and D. 

 

How do I get column D to show up so that I can begin building my queries and displaying the new KPI they want? 

 

I opened the advanced editor and tried simply adding it to the end of the script but that does not work, I get an error that the new cloumn "D" is not found. Here is the script:

 

let
Source = Folder.Files("\\Webdrive\directcallsolution.iqstorage.com\PlanPlusDaily"),
#"Filtered Hidden Files1" = Table.SelectRows(Source, each [Attributes]?[Hidden]? <> true),
#"Invoke Custom Function1" = Table.AddColumn(#"Filtered Hidden Files1", "Transform File", each #"Transform File"([Content])),
#"Renamed Columns1" = Table.RenameColumns(#"Invoke Custom Function1", {"Name", "Source.Name"}),
#"Removed Other Columns1" = Table.SelectColumns(#"Renamed Columns1", {"Source.Name", "Transform File"}),
#"Expanded Table Column1" = Table.ExpandTableColumn(#"Removed Other Columns1", "Transform File", Table.ColumnNames(#"Transform File"(#"Sample File"))),
#"Changed Type" = Table.TransformColumnTypes(#"Expanded Table Column1",{{"Source.Name", type text}, {"A", type text}, {"B", type text}, {"C", type text}})

5 REPLIES 5
Greg_Deckler
Super User
Super User

@Ria_ You probably have to modify your sample file transformation. If these are CSV files for example, they often have a specified number of columns and you just need to remove that.


Follow on LinkedIn
@ me in replies or I'll lose your thread!!!
Instead of a Kudo, please vote for this idea
Become an expert!: Enterprise DNA
External Tools: MSHGQM
YouTube Channel!: Microsoft Hates Greg
Latest book!:
The Definitive Guide to Power Query (M)

DAX is easy, CALCULATE makes DAX hard...

@Greg_Deckler Thank you for replying so fast! I found the Transform sample file and went to the advanced editor:

 

let
Source = Csv.Document(Parameter1,[Delimiter=",", Columns=39, Encoding=1252, QuoteStyle=QuoteStyle.None]),
#"Promoted Headers" = Table.PromoteHeaders(Source, [PromoteAllScalars=true])
in
#"Promoted Headers"

 

 

I am not sure how to remove the columns requirement like you are suggesting - I did change the column number from 39 to 40 and a blank column showed up in the data instead of the new column 'D' and its data. 

The Columns argument is optional. You can omit ", Columns=39" entirely.

@AlexisOlson @Greg_Deckler  Ok guys Im there with you now. So I have deleted the columns argument. Here is what the scirpt looks like now:

 

let
Source = Csv.Document(Parameter1,[Delimiter=",", Encoding=1252, QuoteStyle=QuoteStyle.None]),
#"Promoted Headers" = Table.PromoteHeaders(Source, [PromoteAllScalars=true])
in
#"Promoted Headers"

 

I let that change load and then refreshed the power BI desktop file. The new column is not showing up at all still. Any ideas on what to do next? 

@Ria_ Do you have any other transformations going on such as a "remove other columns" step? 


Follow on LinkedIn
@ me in replies or I'll lose your thread!!!
Instead of a Kudo, please vote for this idea
Become an expert!: Enterprise DNA
External Tools: MSHGQM
YouTube Channel!: Microsoft Hates Greg
Latest book!:
The Definitive Guide to Power Query (M)

DAX is easy, CALCULATE makes DAX hard...

Helpful resources

Announcements
RTI Forums Carousel3

New forum boards available in Real-Time Intelligence.

Ask questions in Eventhouse and KQL, Eventstream, and Reflex.

MayPowerBICarousel1

Power BI Monthly Update - May 2024

Check out the May 2024 Power BI update to learn about new features.