Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more.
Get startedGrow your Fabric skills and prepare for the DP-600 certification exam by completing the latest Microsoft Fabric challenge.
Hello
I'm on a MC fabric project and i use notebook to concatenate severals csv. I would have an other way to do this.
I want ton use pipeline and the get metadata activity.
The folder architecture is on 3 folder level.
Example
Parent folder / child folder / csv
How should i use a pipeline and a for each activity to get the metadata and could copy the csv ?
Thanks for help
Solved! Go to Solution.
Hi @Bendelyon,
In that case you could also use a Get metadata activity before the for each to retrieve your structure first and use that as items in your for each loop instead of the variable I proposed.
Here is a YouTube video which shares such an approach: https://www.youtube.com/watch?v=EU6Ex9EhWyA
Hi @Bendelyon,
With in the for each activity you can use an array. With in the for each you can reference to items of that array via the Dynamic content. A reference would be:
@item()
Below is an example to demonstrate this with the Get metadata activity
1. Example Folder structure
2. Array variable
3. Usage of variable in For Each Items property
@variables('CSVArray')
4. Reference to item of array in GetMetadata activity
@item()
5. Test run pipeline
Hello
Thanks for your help and for five me the "logical" to think.
However i have many parent folder.
Example
Parent folder a / child folder a / csv
Parent folder b / child folder a / csv
Etc
I need to iterate on thé parent folder to enter on the child folder ans get the csv whitch is on.
Thanks for your help
Hi @Bendelyon,
In that case you could also use a Get metadata activity before the for each to retrieve your structure first and use that as items in your for each loop instead of the variable I proposed.
Here is a YouTube video which shares such an approach: https://www.youtube.com/watch?v=EU6Ex9EhWyA
Hi @Bendelyon ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .
Otherwise, will respond back with the more details and we will try to help .
Hi @Bendelyon ,
Glad to know that your query got resolved. Please continue using Fabric Community on your further queries.
Do you mean to concatenate all the CSV files into a single CSV file ?
Then the below blog acts as a sample reference :
https://www.c-sharpcorner.com/article/merge-multiple-json-files-via-synapse-data-factory-pipelines/
wherein you can use the Copy activity recursive and merge into single property
Hello
My principal problem is to get metadata to navigate onto the folder structure.
Thanks
Ask questions in Data Engineering, Data Science, Data Warehouse and General Discussion.
Ask questions in Eventhouse and KQL, Eventstream, and Reflex.
User | Count |
---|---|
5 | |
2 | |
1 | |
1 | |
1 |