cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
rfr
Frequent Visitor

Power Query performance issue

Hi All

 

I have a Power BI file/application which I amend from time to time. It draws data from several excel workbooks and the worksheets within them, and occasionally I'll make amendments in PowerQuery to include new data or calculations. The workbooks are synchronized via OneDrive to my local machine.

 

The problem I'm having now is that each transformation operation I perform (e.g. add a custom column) takes ages (5-20 minutes). Looking at the status bar (bottom left), several excel source files appear to be re-read after every operation, even though some of them aren't directly related to the dataset I'm transforming. (they are source files for the application, just not the derived table I'm working with). For example, the status bar will show "533 MB from source1.xlsx" and continually refresh; apparently switching between files). I would expect these operations to be performed 'in-memory', rather than a reload from excel.

 

It appears this is not memory problem nor processing power problem - I've moved the file to large virtual machine and my memory and CPU utilisation are low, but the problem persists. Also, I've disabled loading in the background in the Power BI options.

 

Could anyone advise on where the problem may lie and how to best resolve it?

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted
rfr
Frequent Visitor

Re: Power Query performance issue

Hi

 

Just a followup: I read the recommended posts and watched an interesting video on query referencing here. In the end, I changed my design to duplicate the table I was previously referencing. I was able to amend some steps and remove several others in my derived queries, and the performance was MUCH better i.e. less than a minute to refresh instead of 20 minutes or more.

 

Reference vesus duplicate.pngQuery Referencing v. Duplicating

 

R.

View solution in original post

2 REPLIES 2
Community Support
Community Support

Re: Power Query performance issue

Hi @rfr ,

I find some posts for this problem, you could refer to this post   for details. By the way, I am  not professional in performance, so I suggest you could @ super users in forum for more suggestions.

Best Regards,
Zoe Zhi

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

 

Highlighted
rfr
Frequent Visitor

Re: Power Query performance issue

Hi

 

Just a followup: I read the recommended posts and watched an interesting video on query referencing here. In the end, I changed my design to duplicate the table I was previously referencing. I was able to amend some steps and remove several others in my derived queries, and the performance was MUCH better i.e. less than a minute to refresh instead of 20 minutes or more.

 

Reference vesus duplicate.pngQuery Referencing v. Duplicating

 

R.

View solution in original post

Helpful resources

Announcements
Announcing the New Spanish Forum

Announcing the New Spanish Forum

Do you need help in Spanish? Check out our new Spanish community section.

April 2020 Community Highlights

April 2020 Community Highlights

Info on our Super Users, MBAS content and badges, and updates to our support articles. - Read the full Community Highlights.

MBAS Gallery 2020

MBAS Gallery 2020

Watch Microsoft Business Applications Summit sessions on-demand.

Top Solution Authors
Top Kudoed Authors