Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!
Hi,
I would have thourght this is a common problem but I can't find any thing here.
Enviroment:
Pulling hundreds of CSV files from Sharepoint that have over time had some of the Column Names changed.
Requirement:
Whilst importing each file update any of the old column headings to the current headings.
Issue
I have a solution below but it is VERY VERY VERY slow when it executes and looking at Resource Monitor Microsoft.Mashup is hammering my memory with hard faults.
Question:
Is there a better way to do this that would be more efficient?
Current Code
#"Transform to CSV: Create import column" = Table.AddColumn(#"Removed Columns", "Transform File from Binary", each let Source = Csv.Document([Content],[Delimiter=",", Columns=85, Encoding=1252, QuoteStyle=QuoteStyle.None]), #"Promoted Headers" = Table.PromoteHeaders(Source, [PromoteAllScalars=true]), #"Rename Column" = Table.RenameColumns(#"Promoted Headers",{ {"Thick1","S1_THICK1_RATE"}, {"Thick2","S1_THICK2_RATE"}, {"Thick3","S1_THICK3_RATE"}, {"Thick4","S1_THICK4_RATE"}, {"Length1","S1_LENGTH1_RATE"}, {"Length2","S1_LENGTH2_RATE"}, {"Length3","S1_LENGTH3_RATE"}, {"Length4","S1_LENGTH4_RATE"}, {"PickFail1","S1_PICKFAIL1_RATE"}, {"PickFail2","S1_PICKFAIL2_RATE"}, {"PickFail3","S1_PICKFAIL3_RATE"}, {"PickFail4","S1_PICKFAIL4_RATE"}, {"Purge","S1_PURGE_RATE"}, {"Shutter","S1_SHUTTER_RATE"} },MissingField.Ignore) in #"Rename Column"), #"Expanded Transform File from Binary" = Table.ExpandTableColumn(#"Transform to CSV: Create import column", "Transform File from Binary", {"CustomerID", "ETC ETC ETC"}),
Regards
Andrew
Hi @Timaru_Golf
It seems you may delete changed type under applied steps in query editor window. If it is not your case, please check if this post can help you.
Regards,
Cherie
Hi @Timaru_Golf
Could you tell me if your problem has been solved? If it is,kindly mark the helpful answer as a solution and welcome to share your own solution. More people will benefit from here. If not, please share more details for us so that we could help further on it.
Regards,
Cherie
User | Count |
---|---|
140 | |
113 | |
104 | |
77 | |
64 |
User | Count |
---|---|
135 | |
122 | |
101 | |
71 | |
61 |