cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
IAM
Helper III
Helper III

SQL connection -> different Query stages.. Why???

Hello,

 

I am working on a report, made by someone I cannot reach anymore. I would like to know the logic behind this setup.. 

 

There are (very large) datatables connected with SQL.

In Power Query he has divided different queries into folders/sections:

 

IAM_0-1657693698610.png

In each 'stage' are some basic steps, like changing a column name and type and merging with lookup tables.

 

Could it be that he needed to make some adjustments first on two tables and then merge them to the other?

If so, is this the best approach? 

 

It takes about 20 to 30 minutes to refresh the dataset. Does it take this long because of this setup?

 

I hope someone can enlighten me 🙂

IAM_1-1657694318193.png

 

 

1 ACCEPTED SOLUTION
indkitty
Helper II
Helper II

Folder or Data source should not be affecting. The SQL data may be cleansed or Raw data, at times I have used both cleansed and raw data. 

I have data refresh running more than 30 minutes, that would be during initial stage. 

Check if the merge table is at the last process, if they are then the new table is dependent on other tables data refresh. You could also check if the data has to be refreshed for the entire tables or update /append only the new data into the table. This would use incremental refresh which is much faster. 

Guysinacube has tutorials for Query folding, incremental refresh, reduction of data by using parameters. 

 

View solution in original post

2 REPLIES 2
indkitty
Helper II
Helper II

Folder or Data source should not be affecting. The SQL data may be cleansed or Raw data, at times I have used both cleansed and raw data. 

I have data refresh running more than 30 minutes, that would be during initial stage. 

Check if the merge table is at the last process, if they are then the new table is dependent on other tables data refresh. You could also check if the data has to be refreshed for the entire tables or update /append only the new data into the table. This would use incremental refresh which is much faster. 

Guysinacube has tutorials for Query folding, incremental refresh, reduction of data by using parameters. 

 

Vijay_A_Verma
Super User
Super User

The folder like organization doesn't impact queries. It's a way to organize it visually and will not have any impact on loading time. 

Why this is taking half an hour - You will need to analyze the relevent queries and their inter-relationships (how these are placed on left side need to discarded from your consideration)

Helpful resources

Announcements
Carousel_PBI_Wave1

2023 Release Wave 1 Plans

Power BI release plans for 2023 release wave 1 describes all new features releasing from April 2023 through September 2023.

Power BI Summit Carousel 2

Global Power BI Training

Make sure you register today for the Power BI Summit 2023. Don't miss all of the great sessions and speakers!

Thank you 2022 Review

2022 Monthly Feature Releases

We had a great 2022 with a ton of feature releases to help you drive a data culture.

Top Solution Authors
Top Kudoed Authors