Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
schau
Regular Visitor

Transforming CSV file from SharePoint folder

Hi all,

 

I have a SharePoint folder which holds a number of large csv files. I want to set up a query which opens all the files in the folder and puts the data into a single view. I was wondering if this is possible and how I can achieve this, noting that:

 

  • The number of CSV files in the SharePoint folder will increase over time
  • All the CSV files will have several header rows (First 8 rows of each CSV file need to be removed)

I've been looking around the community and I've seen solutions when using a folder as the data source, however I haven't been able to find anything for SharePoint folders. Could somebody please point me in the right direction? 

1 ACCEPTED SOLUTION

Hey Lydia,

 

Thanks for the reply. The headers in the CSV are not exactly identical as it also contains some additional metadata. I think because of this, it's not able to automatically detect that they are headers and end up leaving them in the dataset as well.

 

I've figured it out however. After combining the files, I needed to remove the headers from the pre-generated transfomration sample file to get rid of all the header information.

 

Thanks

View solution in original post

3 REPLIES 3
v-yuezhe-msft
Employee
Employee

@schau,

Do all the CSV files have same data columns and same column name? If so, enter the root URL of your SharePoint site in the SharePoint Folder connector, then just click the double arrows in Query Editor to combine the CSV files and put data into a single view. And once you add new CSV files in the SharePoint folder, just click "Refresh Preview" button in Query Editor to bring new data.

1.JPG2.JPG3.JPG

Regards,
Lydia

Community Support Team _ Lydia Zhang
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Hey Lydia,

 

Thanks for the reply. The headers in the CSV are not exactly identical as it also contains some additional metadata. I think because of this, it's not able to automatically detect that they are headers and end up leaving them in the dataset as well.

 

I've figured it out however. After combining the files, I needed to remove the headers from the pre-generated transfomration sample file to get rid of all the header information.

 

Thanks

@schau,

Glad to hear that the issue is solved. Please mark appropriate replies as solutions, that way, other community members would easily find the solution when they get same issues.

Regards,
Lydia

Community Support Team _ Lydia Zhang
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.