cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
dancarr22 Member
Member

Connecting to 'Folder' from desktop just to get info on folder contents - extremely slow

Hello,

 

We are testing Power BI to see what details it can provide about folder/subfolder/files.

So, pretty simply, pointing to a folder (in Get Data) and trying to get info about the contents of those folders.  We do not need to interrogate the contents of those underlying files.  Just need to know attributes/metadata about the file - name, folder, extension, size, last time accessed, etc.

This is extremely slow and takes several hours to run.  All we want is the metadata/properties of each file.  Is there a setting or way to do this so it runs quicker?  We do have ~40k underlying files but it took ~6 hours to run and load.  This was just one folder structure.  

 

Thanks for any help you can provided,

Dan

1 ACCEPTED SOLUTION

Accepted Solutions
Super User
Super User

Re: Connecting to 'Folder' from desktop just to get info on folder contents - extremely slow

Use PowerShell, this isn't really a use case Power BI was designed for. I've written multi-threaded PowerShell to scan folder structures and that's a much better tool for the job. User PowerShell to generate a CSV file of the metadata and import that into Power BI. You are trying to use a miter saw to hammer a nail.


I have book! Learn Power BI from Packt


Did I answer your question? Mark my post as a solution!

Proud to be a Datanaut!

View solution in original post

2 REPLIES 2
Super User
Super User

Re: Connecting to 'Folder' from desktop just to get info on folder contents - extremely slow

Use PowerShell, this isn't really a use case Power BI was designed for. I've written multi-threaded PowerShell to scan folder structures and that's a much better tool for the job. User PowerShell to generate a CSV file of the metadata and import that into Power BI. You are trying to use a miter saw to hammer a nail.


I have book! Learn Power BI from Packt


Did I answer your question? Mark my post as a solution!

Proud to be a Datanaut!

View solution in original post

minhvuong93 Regular Visitor
Regular Visitor

Re: Connecting to 'Folder' from desktop just to get info on folder contents - extremely slow

@Greg_Deckler  Do you mean using PowerShell to scan Excel files and convert into metadata CSV files?

Because I have the exact  issue. Huge Excel files as data, updating monthly, and powerbi refreshes all the data sources from the beginning which took forever

Helpful resources

Announcements
Can You Solve These Challenge

Challenge: Can You Solve These?

Find out how to participate in the first Power BI 'Can You Solve These?' challenge.

Community News & Announcements

Community News & Announcements

Get your latest community news and announcements.

Virtual Launch Event

Microsoft Business Applications October Virtual Launch Event

Join us for an in-depth look at the new innovations across Dynamics 365 and the Microsoft Power Platform.

Community Kudopalooza

Win Power BI Swag with Community Kudopalooza!

Each week, complete activities and be qualified in the drawing for cool Power BI Swag.

Users Online
Currently online: 130 members 1,694 guests
Please welcome our newest community members: