Dataflow timeout using ODBC Athena

[Pandadev]

Hi I have recently been provided with access to an EC2 environment , and I am able to connect to data when in service creating a dataflow using the odbc connector , but there is an issue with timeout after a duration of 3 minutes 23 seconds , here are some errors I keep receiving , I am running out of ideas , the sql takes 11 minutes to run in Athena online. And I have tried using incremental refresh , which it allows . but I still get these errors.

Error: Encountered user gateway exception: '<ccon>ODBC: ERROR [HY000] : curlCode: 28 Timeout was reached

;ERROR [HY000] : curlCode: 28 Timeout was reached</ccon> [DM_ErrorDetailNameCode_UnderlyingErrorCode]=-2147467259 [DM_ErrorDetailNameCode_UnderlyingErrorMessage]=<ccon>ODBC: ERROR [HY000] : curlCode: 28 Timeout was reached

ERROR [HY000] : curlCode: 28  Timeout was reached</ccon> [GatewayPipelineErrorCode]=DM_GWPipeline_Gateway_MashupDataAccessError [ErrorShortName]=GatewayClientErrorResponseException[GatewayId=327596]/MashupDataAccessValueException[ErrorCode=-2147467259 HResult=-2147467259]/Wrapped(MashupValueException)[ErrorCode=-2147467259 HResult=-2147467259] [ExceptionErrorShortName]=GatewayClientErrorResponseException[GatewayId=327596] Request ID: 9010-aa27-d4fa-1beacf2a96a5.

[Pandadev]

Hi Yulia Yan

I managed to get dataflows created by breaking them down to yearly reports , with each report having approx 30 million rows , and dataflows are 2019,2020,2021,2022,2023,and 2024 , 2019,2020,2021,2022,2023 dataflows are static data so am only refreshing 2024 data . I then craeted a dataflow on linked dataflows , and appended the dataflows in a datflow named full. Now when I refersh this it takes about 1 hour , but is always doing a full refresh , I am working in a premium workspace and changed the compute engine setting to on . I have one date time column , so I wanted to try and use the incremental refresh , but it fails when I attempt to do that. It is not the end of the worl If I have to do a full refersh every morning , but I am struggling to work out why incremental refresh is failing , I have it set to 7 years of data , 3 days append , and refresh full days only. The final issue I have and the biggest issue , is taht I need to add this dataframe into my powerbi report , but it is not giving me any option to set it as direct query .

[v-weiyan1-msft]

Hi @Pandadev ,

You may need to increase the timeout value for the ODBC connection. This can often be done either in the connection string or through the ODBC data source administrator tool.
If you are using a shared gateway, please consider using a dedicated gateway with more resources allocated to it, which might help with timeout issues.
Please ensure that the network connection between your Power BI service and the EC2 environment is stable and has sufficient bandwidth to handle the data transfer without timing out.

In addition, the following link had the similar problem as yours. Please refer to it and see if it helps you.

Solved: ODBC: ERROR [HY000] [Microsoft][BigQuery] (115) Op... - Microsoft Fabric Community

Best Regards,
Yulia Yan

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.