Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more.
Get startedGrow your Fabric skills and prepare for the DP-600 certification exam by completing the latest Microsoft Fabric challenge.
I'm loading data from a on-premise Gupta database using ODBC with DataFlow Gen2
It's working fine until I try to make it an incremental refresh using the last date from the already loaded table.
I'm extracting the data from the warehouse with:
This seems to be working fine.
I'm using a native query and I use the output of the first query as a value (VW_INCR_DATE) in the sql native query
In the dataflow (when editing) it seems to be working. The results are showing correct.
But the issue shows up when the dataflow runs.
I don't understand this issue. Is anyone able to help me with this issue?
Solved! Go to Solution.
The VW_INCR_DATE value is coming from the warehouse and is the last date for which the data is already loaded. That is not from the source.
I found the solution. The query which extracts the Date value from the warehouse should have "enable staging" false to get it working. Not sure why though :).
I found the solution. The query which extracts the Date value from the warehouse should have "enable staging" false to get it working. Not sure why though :).
Hi @SnoekL
Dosen't look like implemented refresh is implemented the right way. Please refer to this
https://learn.microsoft.com/en-us/power-bi/connect-data/incremental-refresh-overview
Regards
Amine Jerbi
If I answered your question, please mark this thread as accepted
and you can follow me on
My Website, LinkedIn and Facebook
aj1973, the topic you are referring to is actually the one I used to build my incremental refresh.
Big difference though is that the example has sql database as a source where I'm connecting via odbc to a local database.
And the example has a datekey in the source which is a integer where my source only has a datetime field to use to get the incremental data.
I tried to convert the source data to different datatypes but than I get different errors as well. So i'm stuck at the moment.
When connecting to ODBC
Add a SQL statement to convert/cast this part
to Date before loading into Power Query Online.
Power Query couldn't accept to add a step to Format that column so try with SQL statement, it should work.
Regards
Amine Jerbi
If I answered your question, please mark this thread as accepted
and you can follow me on
My Website, LinkedIn and Facebook
The VW_INCR_DATE value is coming from the warehouse and is the last date for which the data is already loaded. That is not from the source.
Join the community in Stockholm for expert Microsoft Fabric learning including a very exciting keynote from Arun Ulag, Corporate Vice President, Azure Data.
User | Count |
---|---|
55 | |
39 | |
33 | |
31 | |
25 |