Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!
I'm using folder as a source where csv files get saved daily, and is not uncommon to have new columns added to the csv file. I would like to figure out a way that my query will account for them. Is it possible?
File Sample 1:
Timeslice | api | computenode | cris | datapower | dcf |
02/17/2019 12:00:00 AM EST | 4554.465 | 7.004469 | 1.877915 | 30.03808 | 26.07254 |
File Sample 2:
Timeslice | aad | api | computenode | cris | datapower |
02/18/2019 12:00:00 AM EST | 0.181887 | 5611.143 | 13.32732 | 1.88031 | 30.31816 |
I would like to account for the addition of the aad column and have it added to my data automatically. I even don't mind adding another sample file if that is an option. Any options would be great.
Thanks for the help!
Hi @Anonymous ,
There could be a solution. Please download the demo from the attachment. It will detect all the columns in theory.
1. Add one custom column.
List.Accumulate(#"Removed Other Columns1"[Transform File from FolderSource], {}, (seed, value)
=> List.Union({seed, Table.ColumnNames(value)}))
2. Modify the M code in the Advanced Editor.
#"Expanded Table Column1" = Table.ExpandTableColumn(#"Added Custom", "Transform File from FolderSource", #"Added Custom"{0}[Custom])
3. Delete the Columns limitation in the Sample query.
Best Regards,
I tried and tried and couldn't get this to work. I have many files and wasn't able to get the new columns to be seen. Is it possible to add another sample file with the additional columns?
Hi @Anonymous ,
Could you please mark the proper answers as solutions?
Best Regards,
It didn't work for me and I haven't had time to revisit it. I can close this but not really as solved.
No hurry, please take your time. I hope you can make it work. Appreciate it if you can share the solution here.
Best Regards,
Hi @Anonymous ,
Actually, the first step retrieves all the column names.
Did you try my demo?
Best Regards,
User | Count |
---|---|
130 | |
108 | |
101 | |
67 | |
62 |
User | Count |
---|---|
137 | |
115 | |
102 | |
71 | |
61 |