Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
chiburi
Regular Visitor

Spark Connector Scheduled Refresh Failing

I've been POC'ing loading data from Spark into power BI.

 

All was going fine until I started load testing the scheduled refresh in the power service and I'm hitting lots of different intermittant issues. I've had various errors and can't seem to establish 1) a pattern 2) a cause and 3) a solution.

 

I've tried upping the cluster size and memory to no avail. Anyone got any ideas?

 

Errors that have occured intermintantly i.e. sometimes it succeeds:

  • ODBC: ERROR [HY000] [Microsoft][Hardy] (34) Error from server: Could not refill buffer. Table: ******* 
  • ODBC: ERROR [HY000] [Microsoft][Hardy] (34) Error from server: Invalid method name: 'GetPrimaryKeys'. Table: *********
  • The table has no visible columns and cannot be queried. Table: *****

I get this sometimes also...

  • Last refresh succeeded: null

The connection driver seems really flakey!

8 REPLIES 8
v-shex-msft
Community Support
Community Support

HI @chiburi,

 

Can you please share more detailed error message and some trace log to help us clarify your issue?

Fiddler - Free Web Debugging Proxy - Telerik

 

Regards,

Xiaoxin Sheng

Community Support Team _ Xiaoxin
If this post helps, please consider accept as solution to help other members find it more quickly.

Hi.

 

I know what fiddler is but how are you expecting me to get further detail using it?

This is all Cloud Azure - powerbi app service is reshreshing against databricks spark cluster PaaS service.

 

Thanks

HI @chiburi,

 

>>I know what fiddler is but how are you expecting me to get further detail using it?

Please trace some logs for refresh operation.

 

Actually, I want to confirm if your issue is same as one support ticket which other user submitted.

 

BTW, azure has canceled spark subscription, I guess your issue may related to it.

 

Regards,

Xiaoxin Sheng

Community Support Team _ Xiaoxin
If this post helps, please consider accept as solution to help other members find it more quickly.

Hi.

 

Which other support ticket?

 

"BTW, azure has canceled spark subscription, I guess your issue may related to it."

 

Sorry this makes no sense to me. Spark is available in Azure with Data Bricks or HDInsight. What specifically do you mean?

 

Thanks

Hi @chiburi,

 

The ticket is opened on CRI 81831666 about refresh issue with spark and odbc driver. I'm not sure if your issue is same as his.

 

Regards,

Xiaoxin Sheng

Community Support Team _ Xiaoxin
If this post helps, please consider accept as solution to help other members find it more quickly.

Hi.

 

I'm not sure either. I googled the reference... How can I see the details of the issue?

Do you have a link?

 

Thanks

I am also facing the same issue when i am importing data using spark connector.

     Error: ODBC: ERROR [HY0000] [Microsoft][Hardy] (34) Error from server: Invalid method name: 'GetPrimaryKeys'

 

Please help me with this if you have any resolution of this error

Hi.

 

We ended raising a support call. It's an issue with the simba driver that they're using. They're working on a fix... I don't know if it will be rolled out across all tenacies in the same time frame but fix would be rolled out to everyone at some point I imagine.

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors