Dataflow (gen1) fails after Join operation

[GonzaloB]

Hi everyone,

I pull 7 tables from our Postgres DB into a dataflow. I was able to refresh it just fine, but after resuming my ETL process, and merging 'Orders' and 'Orderline' table to get a flat fact table, and although I am able to see the data in the dataflow data view, upon saving and exiting the dataflow fails to refresh, showing this error:

'Error: On-Prem execution not supported for entity '<ccon>9ac68d65-62fe-4f31-975c-63fcca146f32/123695af-7b5c-44a6-a79c-060514043792/Final_Spf orders</ccon>' since it refers to another entity.. Param1 = <ccon>9ac68d65-62fe-4f31-975c-63fcca146f32/123695af-7b5c-44a6-a79c-060514043792/Final_Spf orders</ccon> Request ID: 53969960-39c0-6b27-9983-735323ac30dd.'

Can anyone please advise what am I doing wrong with such a simple ETL process?

Thanks,

EDIT: confirming that removing the merge step fixes the issue.

I'll keep troubleshooting, but any ideas are welcome 🙂

[GonzaloB]

Hi @mohitkumawat ,

Thanks for your assistance.

When you say dataflow settings, I cannot really see any load options here:

If you are referring to this 'enable load' then I can confirm it is now off on all tables but now the dataflow won't validate

Am I missing something?

Also are you suggesting that before every merge I create a reference table and merge those instead?

Thanks,

[GonzaloB · Accepted solution]

In case anyone stumbles on this post, I got around by having another admin of the workspace reconfigure the credentials. Apparently the connection to a google sheet was the culprit.

It has been working correctly for the last two days, fingers cross y does not randomly break.

[mohitkumawat]

Hi @GonzaloB ,

This error is triggered due to the dependencies with the source “Orders”  tables, where the latter is the one that necessitates the use of on-premises gateway, from the merged entity.

you can check below things if this is enabled 

First, navigate to the dataflow settings and ensure the “Enable load” for the linked entities’ “Orders”  checkbox is not ticked .

By refreshing the data in the cloud first, and then using the reloaded data for merging he can thus bypass the on-premises gateway.

Instead of merging the tables directly, create a reference with the original tables in the dataflow. Consequently, this will enable him to use the cloud-based source of the dataflow entities during the refresh rather than relying on on-premises gateway.