cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
chiburi
Regular Visitor

Spark Connector Scheduled Refresh Failing

I've been POC'ing loading data from Spark into power BI.

 

All was going fine until I started load testing the scheduled refresh in the power service and I'm hitting lots of different intermittant issues. I've had various errors and can't seem to establish 1) a pattern 2) a cause and 3) a solution.

 

I've tried upping the cluster size and memory to no avail. Anyone got any ideas?

 

Errors that have occured intermintantly i.e. sometimes it succeeds:

  • ODBC: ERROR [HY000] [Microsoft][Hardy] (34) Error from server: Could not refill buffer. Table: ******* 
  • ODBC: ERROR [HY000] [Microsoft][Hardy] (34) Error from server: Invalid method name: 'GetPrimaryKeys'. Table: *********
  • The table has no visible columns and cannot be queried. Table: *****

I get this sometimes also...

  • Last refresh succeeded: null

The connection driver seems really flakey!

8 REPLIES 8
Community Support
Community Support

Re: Spark Connector Scheduled Refresh Failing

HI @chiburi,

 

Can you please share more detailed error message and some trace log to help us clarify your issue?

Fiddler - Free Web Debugging Proxy - Telerik

 

Regards,

Xiaoxin Sheng

Community Support Team _ Xiaoxin Sheng
If this post helps, please consider Accept it as the solution to help the other members find it more quickly
Learning resources: Power BI
chiburi
Regular Visitor

Re: Spark Connector Scheduled Refresh Failing

Hi.

 

I know what fiddler is but how are you expecting me to get further detail using it?

This is all Cloud Azure - powerbi app service is reshreshing against databricks spark cluster PaaS service.

 

Thanks

Community Support
Community Support

Re: Spark Connector Scheduled Refresh Failing

HI @chiburi,

 

>>I know what fiddler is but how are you expecting me to get further detail using it?

Please trace some logs for refresh operation.

 

Actually, I want to confirm if your issue is same as one support ticket which other user submitted.

 

BTW, azure has canceled spark subscription, I guess your issue may related to it.

 

Regards,

Xiaoxin Sheng

Community Support Team _ Xiaoxin Sheng
If this post helps, please consider Accept it as the solution to help the other members find it more quickly
Learning resources: Power BI
chiburi
Regular Visitor

Re: Spark Connector Scheduled Refresh Failing

Hi.

 

Which other support ticket?

 

"BTW, azure has canceled spark subscription, I guess your issue may related to it."

 

Sorry this makes no sense to me. Spark is available in Azure with Data Bricks or HDInsight. What specifically do you mean?

 

Thanks

Community Support
Community Support

Re: Spark Connector Scheduled Refresh Failing

Hi @chiburi,

 

The ticket is opened on CRI 81831666 about refresh issue with spark and odbc driver. I'm not sure if your issue is same as his.

 

Regards,

Xiaoxin Sheng

Community Support Team _ Xiaoxin Sheng
If this post helps, please consider Accept it as the solution to help the other members find it more quickly
Learning resources: Power BI
chiburi
Regular Visitor

Re: Spark Connector Scheduled Refresh Failing

Hi.

 

I'm not sure either. I googled the reference... How can I see the details of the issue?

Do you have a link?

 

Thanks

sumitg
Regular Visitor

Re: Spark Connector Scheduled Refresh Failing

I am also facing the same issue when i am importing data using spark connector.

     Error: ODBC: ERROR [HY0000] [Microsoft][Hardy] (34) Error from server: Invalid method name: 'GetPrimaryKeys'

 

Please help me with this if you have any resolution of this error

chiburi
Regular Visitor

Re: Spark Connector Scheduled Refresh Failing

Hi.

 

We ended raising a support call. It's an issue with the simba driver that they're using. They're working on a fix... I don't know if it will be rolled out across all tenacies in the same time frame but fix would be rolled out to everyone at some point I imagine.

Helpful resources

Announcements
Announcing the New Spanish Forum

Announcing the New Spanish Forum

Do you need help in Spanish? Check out our new Spanish community section.

April 2020 Community Highlights

April 2020 Community Highlights

Info on our Super Users, MBAS content and badges, and updates to our support articles. - Read the full Community Highlights.

MBAS Gallery 2020

MBAS Gallery 2020

Watch Microsoft Business Applications Summit sessions on-demand.