Dataflow to bring data from Google Bigquery takes forever and fails often

[sathishsam]

We have our data currently in Google BigQuery and using import model to build our reports and now want to explore Fabric to see if the reports would become more faster.

I tried to get data from BigQuery to Fabric Lakehouse using Dataflow Gen2, however, to get 30 million rows it is taking 4 to 5 hours and many a times it fails. Not sure if I am doing something incorrect.

- Using Google Bigquery connector

- No transformation at all

This is very important for us to further explore Fabric, please help

[sathishsam]

Raised a ticket and working with the engineer. however, it looks like there is nothing much the Dataflow Gen 2 will do here..
SR 2312120010002318

[v-gchenna-msft]

Hi @sathishsam ,

Apologies for the issue that you are facing.

This might require a deeper investigation from our engineering team about your workspace and the logic behind it to properly understand what might be happening. 

Please go ahead and raise a support ticket to reach our support team:

https://support.fabric.microsoft.com/support
Please provide the ticket number here as we can keep an eye on it.

[v-gchenna-msft]

Hi @sathishsam ,

We haven’t heard from you on the last response and was just checking back to see if you got a chance to create a support ticket. If yes, please provide the ticket number here as we can keep an eye on it.