cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Highlighted
Helper I
Helper I

Too long to Refresh - OneDrive for Business

Hello PBI Community,

 

I have multiple Excel files (6 files) and Folder (Contain 15 or so XL files) on my OneDrive for Business,

then I import them to PQ using Sharepoint.Content.

- I have a Query "Connection" which establish the OneDrive connection and navigate to the destination Folder

- I reference this query then start mapping to the XLs and the Folder

- Then I make some transformation to my data

 

When I hit Refresh the Query I'll describe take too long to refresh and here is how I processed it:

- Reference the Connection Query

- Select the XLs inside the folder to combine them

- Combine the files and apply some transformation using custom function I made then expand the tables (query1)

** to this point I refresh less than a minute which is good, but the next steps made the difference**

- I start new query referencing the previous one filter some specific data from external list (query2)

- then append the result from this Query to the previous one to a new query to avoid circular reference error.(result)

 

This steps made my refresh time more than 2 minutes.

As I see when PQ show you the refresh process, only the new table that has the appended query take long.

It's like with each step connecting to OneDrive to get the data then continue to the next step.

 

I tried Table.Buffer to the result query1 but it made the refresh longer.

My internet connection is fast and the laptop I use is high end, so no hardware issues.

 

Any Ideas 🙂

 

3 REPLIES 3
Highlighted
Community Support
Community Support

Re: Too long to Refresh - OneDrive for Business

Hi @hatahetahmad ,

 

Maybe you can use this connector:

(Use the root URL of your OneDrive for Business.)

i4.PNG

Then we can combine all the files in the OneDrive for business.

i5.PNG

 

Best regards,
Lionel Chen

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Highlighted
Helper I
Helper I

Re: Too long to Refresh - OneDrive for Business

Thank you for your comment.

 

I am using the root folder, like this thread 

https://radacad.com/power-bi-get-data-from-multiple-files-in-a-folder-on-onedrive-for-business-no-ga...

Highlighted
Frequent Visitor

Re: Too much time to update - OneDrive for Business

I have also faced this problem due to the slowness of that connector and, for the time being, I have solved it as follows:

  1. I use one parameter to direct the connection to the OneDrive with all the data or to another that contains very few rows.
  2. I perform transformations in PQ by connecting to the OneDrive with few-row files to minimize slowness.
  3. Once all the transformations have been done, I modify the parameter to point to the final OneDrive.
  4. I'm in the cloud.
  5. I'm upgrading from the cloud thereafter, as it's faster than the on-premises upgrade.
  6. If I need to make new modifications to PQ I point again to the OneDrive with small files.

It's not the ideal solution, but it can help you while finding a better way.

Best regards.

Helpful resources

Announcements
Community Blog

Community Blog

Visit our Community Blog for articles, guides, and information created by fellow community members.

Using the Community

Using the Community

Need help with the Power BI Community? Our 'Using the Community' support articles are a great place to start.

Community Summit North America

Community Summit North America

Innovate, Collaborate, Grow. The top training and networking event across the globe for Microsoft Business Applications

Power Platform 2020 release wave 2 plan

Power Platform 2020 release wave 2 plan

Features releasing from October 2020 through March 2021

Top Solution Authors
Top Kudoed Authors