Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
Anonymous
Not applicable

Need query to account for newly added column in dataset

I'm using folder as a source where csv files get saved daily, and  is not uncommon to have new columns added to the csv file. I would like to figure out a way that my query will account for them. Is it possible? 

 

File Sample 1:

Timesliceapicomputenodecrisdatapowerdcf
02/17/2019 12:00:00 AM EST4554.4657.0044691.87791530.0380826.07254

File Sample 2:

 

Timesliceaadapicomputenodecrisdatapower
02/18/2019 12:00:00 AM EST0.1818875611.14313.327321.8803130.31816

 

I would like to account for the addition of the aad column and have it added to my data automatically. I even don't mind adding another sample file if that is an option. Any options would be great.

 

Thanks for the help!  

 

 

6 REPLIES 6
v-jiascu-msft
Employee
Employee

Hi @Anonymous ,

 

There could be a solution. Please download the demo from the attachment. It will detect all the columns in theory.

1. Add one custom column. 

List.Accumulate(#"Removed Other Columns1"[Transform File from FolderSource], {}, (seed, value) 
=> List.Union({seed, Table.ColumnNames(value)}))

2. Modify the M code in the Advanced Editor. 

   #"Expanded Table Column1" = Table.ExpandTableColumn(#"Added Custom", "Transform File from FolderSource", #"Added Custom"{0}[Custom])

3. Delete the Columns limitation in the Sample query.

Need-query-to-account-for-newly-added-column-in-dataset

 

 

Best Regards,

Community Support Team _ Dale
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Anonymous
Not applicable

I tried and tried and couldn't get this to work. I have many files and wasn't able to get the new columns to be seen. Is it possible to add another sample file with the additional columns?

Hi @Anonymous ,

 

Could you please mark the proper answers as solutions?

 

 

Best Regards,

Community Support Team _ Dale
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Anonymous
Not applicable

It didn't work for me and I haven't had time to revisit it. I can close this but not really as solved.

No hurry, please take your time. I hope you can make it work. Appreciate it if you can share the solution here.

 

 

Best Regards,

Community Support Team _ Dale
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Hi @Anonymous ,

 

Actually, the first step retrieves all the column names. 

Did you try my demo?

 

 

Best Regards,

Community Support Team _ Dale
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.