Scheduled dataset refresh fails randomly

[413x]

Hi - could do with some help:

Our dataset connects to a API which then connects to our database.   On our scheduled refreshes it normally refreshes fine, however at random points some of the refreshes will just fail with one of 3 different processing error messages :

1. We reached the end of the buffer.
2. (504) Gateway Time-out
3. (502) Bad Gateway

There is not order to any of these errors they just happen randomly. We could go weeks without any problems then out of the blue one of the refreshes fails with one of these errors.

We have tried multiple different things to try and combat this :

1. Limited the amount of results returned by the API and then just called the query recursively to get all the results
2. Doubled the servers CPU & RAM 
3. Extended the Timeout for each query in Power BI

We are just wondering is anyone has had any sort of problem like this and any soultions to try would be greatly appreciated.

[v-zhangti · Accepted solution]

Hi, @413x 

Regarding the first error, you can refer to the following link to troubleshoot it.

Solved: Refresh fails with error We reached the end of the... - Microsoft Power BI Community

Regarding the error reported by the gateway, please make sure the gateway version is the latest version first. Please refer to the following link for troubleshooting.

https://community.powerbi.com/t5/Service/Gateway-timeout-error/m-p/1146938 

https://powerusers.microsoft.com/t5/Building-Flows/How-to-fix-gateway-timeout-error-504/td-p/968039 

https://community.powerbi.com/t5/Power-Query/Connecting-to-Application-Insights-Bad-Gateway-Error-fr... 

Best Regards,

Community Support Team _Charlotte

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[Hariharan_R]

Hi @413x 

The below links might help since all are referrring the same issues.

https://community.powerbi.com/t5/Power-Query/We-reached-end-of-buffer-Error/td-p/989369

https://community.powerbi.com/t5/Service/Gateway-timeout-error/m-p/1146938

Thanks

Hari

Did I answer your question? Then please mark my post as the solution.
If I helped you, click on the Thumbs Up to give Kudos.

My Blog :: YouTube Channel :: My Linkedin