cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
joshua1990
Post Prodigy
Post Prodigy

Databricks very slow

Hello all,

I recently got access to a Databricks cluster. In PBI Desktop I use the Azure Databricks connection type to access this data.
Is it possible that Databricks is not primarily intended for importing large tables? But rather intended to work through Direct Query? Compared to Oracle, ODBC or Impala, Databricks is really very slow here.

Does anyone know what could be the reason for this?

Would it be helpful to access the Databricks cluster via ODBC?

Using an enterprise gateway and a corresponding dataflow, the refresh is faster than in PBI Desktop.

2 REPLIES 2
v-xulin-mstf
Community Support
Community Support

Hi @joshua1990

 

Maybe you can access the Databricks cluster via ODBC.

Please refer:

https://mauridb.medium.com/powerbi-and-azure-databricks-193e3dc567a

If you still have some question, please don't hesitate to let me known.‌‌

 

Best Regards,

Link

 

Is that the answer you're looking for? If this post helps, then please consider Accept it as the solution. Really appreciate!

daxer-almighty
Solution Sage
Solution Sage

The answer is as usual: it depends. It's no secret that the best and fastest data source for PBI is a relational database. But the speed depends on many factors. One of them being the throughput of the network and the quality of the driver. I have no experience with Databricks as the source for PBI, so can't really comment on this any more than I have. I can just add that getting data from a plain csv file is much faster than from other data sources (minus the relational db mentioned). It might also be faster to get data from a parquet file if the data is BIG.

Helpful resources

Announcements
Microsoft Build 768x460.png

Microsoft Build is May 24-26. Have you registered yet?

Come together to explore latest innovations in code and application development—and gain insights from experts from around the world.

charticulator_carousel_with_text (1).png

Charticulator Design Challenge

Put your data visualization and design skills to the test! This exciting challenge is happening now through May 31st!

May UG Leader Call Carousel 768x460.png

What difference can a User Group make for you?

At the monthly call, connect with other leaders and find out how community makes your experience even better.

Power BI Dev Camp Session 22 768x460.jpg

Check it out!

Mark your calendars and join us on Thursday, May 26 at 11a PDT for a great session with Ted Pattison!