DB2, Incremental Refresh, Query Folding Basics

[troyhimes]

Hello,

I am querying a DB2 db but fails every so often due to a time out error.  I've never used incremental refresh but understand that this would probably be a solution and that query folding is required.  Google tells me that query folding is possible with a DB2 query but for some reason it doesn't seem to be working for me.  Anything that I'm missing?

[troyhimes]

There has to be some Db2 folders out there.....anyone?

[troyhimes]

Yolo,

1. The difference in the 'Source' step in my case (Db2) vs your example (SQL) is that the 'Database' definition in the first step is required vs optional.  That's the reason my first 'Source' step includes the 'Database' name.

2. I've tried dropping the "" from the database name and still no luck 

Source: 

Custom1

[v-xinruzhu-msft]

Hi @troyhimes 

Based on your descriprion, you can refer to the following link about the query folding.

Query folding examples in Power Query - Power Query | Microsoft Learn

Understanding query evaluation and query folding in Power Query - Power Query | Microsoft Learn

And if you  want to find which steps are preventing your query from folding, you can use the query folding indicators  to check, you can refet to the following link.

Query folding indicators in Power Query - Power Query | Microsoft Learn

And you can refer to the following link to know more about incremental refresh.

Incremental refresh for semantic models and real-time data in Power BI - Power BI | Microsoft Learn

Best Regards!

Yolo Zhu

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

 

[troyhimes]

Thanks for the response Yolo ( @v-xinruzhu-msft ) and the links.  I had already reviewed all of those in my initial quest.  Any guidance specifically in relation to query folding and Db2?  From the screenshot above query folding is dead at the 'source' step (right click on 'Source', 'View Native Query' is greyed out).  Does it have to do with using the 'Microsoft' driver rather than IBM?  The query doesn't work when using the IBM driver.

[v-xinruzhu-msft]

Hi @troyhimes 

Thanks for your quick reply.  After testing, the source step itself is not collapsible, it only acts as a connector to connect to the data source, but in order to ensure that the later steps can be collapsed， you can refer to the following steps:

1.Make sure that your connect mode is import, don't filter in advanced options. just click ok

2.Select the database , then click 'Transform data'

3.Add a new step in power query , then refer to the following code.

= Value.NativeQuery(xxxx(your database name),"select * from SQLTest",null,[EnableFolding=true])

Then you can find the query folding can use.

And you can refer to the following video.

Bing Videos

Best Regards!

Yolo Zhu

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[troyhimes]

Thanks Yolo (@v-xinruzhu-msft ), we're getting close.  I had previously watched that video too but couldn't get it to work.  I believe it may be something simple in the way I'm writing the command.  Do you see anything that may be causing the issue?

[v-xinruzhu-msft]

Hi @troyhimes 

Thanks for your quick reply, based on the picture you have offered,

You need to enter the database first, you can refer to the step I offered above.

Then you don't need to use " " for the database name, try this.

Value.NativeQuery(FXXD,"select * from jdeview",null,[EnableFolding=true])

Best Regards!

Yolo Zhu

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[troyhimes]

Yolo ( @v-xinruzhu-msft ),

1. One difference on the 'Source' step betwen mine (Db2) and your example (SQL) is that the database name is required in the first step for Db2 source vs a SQL source.

2.  With that in mind, here's how it goes down for me.... tried it a couple different ways, but it ends with an error either way

Try #1

Try #2