Schedule Refresh do not complete when referenced / computed entity contain Incremental refresh

[jochungJE]

Please Help,

I am having issues with my dataflow. When I add add a NEW dataflow with NEW reference entities. It freshed without issues. 

When I apply incremental refresh policy on the these NEW referenced entites, the refresh just go on forever and do not complete (went for over 8 hours until I cancel it manually) 

Existing Datflow with Incremental Refresh policy set on Referenced Entities continue to refresh without issue. 

[Icey]

Hi @jochungJE ,

Hope this post could help you: How to use Incremental Refresh in Dataflows?

Best Regards,

Icey

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[jochungJE]

Thanks @Icey,

That does not explain why for the same computed entities, without incremental refresh it took 3 hours to complete, but when I set incremental refresh it just doesn't finish.. (I have to stop after 8 hours as it is impating other dataflow refrehses).

Also even though the linked entity may already have incremental refresh policy inplace, but for the computed entites, I may need different incremental policy. e.g the linked entitiy hold 3 years of data, but for the computed entity, I only need 12 months of data.

The suggestion of the computed entities contain advanced or complex query which cause the refresh time to be longer than the refresh time of the linked entites do not really apply here as the computed entity only contain further 'cleansing' steps without merging or calculation.

Would appreciate your though or suggestion to resolve this issue.

[Icey]

Hi @jochungJE ,

Sorry, I have no access to create dataflows in my premium test account. And so far, I have not found a similar case.

It is suggested to create a support ticket to get further help. If there is any workaround, please share it here.

Best Regards,

Icey

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.