Reply
Frequent Visitor
Posts: 5
Registered: ‎09-07-2018

Spark Connector Scheduled Refresh Failing

I've been POC'ing loading data from Spark into power BI.

 

All was going fine until I started load testing the scheduled refresh in the power service and I'm hitting lots of different intermittant issues. I've had various errors and can't seem to establish 1) a pattern 2) a cause and 3) a solution.

 

I've tried upping the cluster size and memory to no avail. Anyone got any ideas?

 

Errors that have occured intermintantly i.e. sometimes it succeeds:

  • ODBC: ERROR [HY000] [Microsoft][Hardy] (34) Error from server: Could not refill buffer. Table: ******* 
  • ODBC: ERROR [HY000] [Microsoft][Hardy] (34) Error from server: Invalid method name: 'GetPrimaryKeys'. Table: *********
  • The table has no visible columns and cannot be queried. Table: *****

I get this sometimes also...

  • Last refresh succeeded: null

The connection driver seems really flakey!

Community Support Team
Posts: 7,711
Registered: ‎08-14-2016

Re: Spark Connector Scheduled Refresh Failing

HI @chiburi,

 

Can you please share more detailed error message and some trace log to help us clarify your issue?

Fiddler - Free Web Debugging Proxy - Telerik

 

Regards,

Xiaoxin Sheng

Community Support Team _ Xiaoxin Sheng
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.



For learning resources/Release notes, please visit: | |
Frequent Visitor
Posts: 5
Registered: ‎09-07-2018

Re: Spark Connector Scheduled Refresh Failing

Hi.

 

I know what fiddler is but how are you expecting me to get further detail using it?

This is all Cloud Azure - powerbi app service is reshreshing against databricks spark cluster PaaS service.

 

Thanks

Community Support Team
Posts: 7,711
Registered: ‎08-14-2016

Re: Spark Connector Scheduled Refresh Failing

[ Edited ]

HI @chiburi,

 

>>I know what fiddler is but how are you expecting me to get further detail using it?

Please trace some logs for refresh operation.

 

Actually, I want to confirm if your issue is same as one support ticket which other user submitted.

 

BTW, azure has canceled spark subscription, I guess your issue may related to it.

 

Regards,

Xiaoxin Sheng

Community Support Team _ Xiaoxin Sheng
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.



For learning resources/Release notes, please visit: | |
Frequent Visitor
Posts: 5
Registered: ‎09-07-2018

Re: Spark Connector Scheduled Refresh Failing

Hi.

 

Which other support ticket?

 

"BTW, azure has canceled spark subscription, I guess your issue may related to it."

 

Sorry this makes no sense to me. Spark is available in Azure with Data Bricks or HDInsight. What specifically do you mean?

 

Thanks

Community Support Team
Posts: 7,711
Registered: ‎08-14-2016

Re: Spark Connector Scheduled Refresh Failing

Hi @chiburi,

 

The ticket is opened on CRI 81831666 about refresh issue with spark and odbc driver. I'm not sure if your issue is same as his.

 

Regards,

Xiaoxin Sheng

Community Support Team _ Xiaoxin Sheng
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.



For learning resources/Release notes, please visit: | |
Frequent Visitor
Posts: 5
Registered: ‎09-07-2018

Re: Spark Connector Scheduled Refresh Failing

[ Edited ]

Hi.

 

I'm not sure either. I googled the reference... How can I see the details of the issue?

Do you have a link?

 

Thanks

Frequent Visitor
Posts: 2
Registered: ‎10-23-2018

Re: Spark Connector Scheduled Refresh Failing

I am also facing the same issue when i am importing data using spark connector.

     Error: ODBC: ERROR [HY0000] [Microsoft][Hardy] (34) Error from server: Invalid method name: 'GetPrimaryKeys'

 

Please help me with this if you have any resolution of this error

Frequent Visitor
Posts: 5
Registered: ‎09-07-2018

Re: Spark Connector Scheduled Refresh Failing

Hi.

 

We ended raising a support call. It's an issue with the simba driver that they're using. They're working on a fix... I don't know if it will be rolled out across all tenacies in the same time frame but fix would be rolled out to everyone at some point I imagine.