Can I connect Azure Databricks from Power BI Desktop by DirectQuery?

[yoshihirok]

I want to know, Can I connect Azure Databricks from Power BI Desktop by DirectQuery?

Currently, the data sources for DirectQuery support Apache Spark on Azure HDInsight and Spark.

https://docs.microsoft.com/en-us/power-bi/desktop-directquery-data-sources

and, Power BI Desktop will support to analyze 1 trillion rows on Apache Spark.

https://www.microsoft.com/en-us/businessapplicationssummit/video/BAS2018-2172

and, Azure Databricks is Apache Spark-based analytics platform

https://azure.microsoft.com/en-us/services/databricks/

Therefore, How about Azure Databricks by DirectQuery?

What do you think?

Regards,

Yoshihiro Kawabata

[Anonymous]

Hi,

I followed the steps but no luck in connecting databricks from power BI.

getting below error.

Details: "ODBC: ERROR [HY000] [Microsoft][DriverSupport] (1170) Unexpected response received from server. Please ensure the server host and port specified for the connection are correct."

any help will be much appreciated.

Thanks

Jith

[susriram]

Yes, You can connect Azure Databricks from Power BI Desktop by DirectQuery. You need to select Spark connector. I tried in PowerBI November 2018 Version, it's working.

FYI, BI Version that I used.

Version: 2.64.5285.741 64-bit (November, 2018)

[v-frfei-msft · Accepted solution]

Hi @yoshihirok,

Power BI Desktop can be connected directly to an Azure Databricks cluster using the built-in Spark connector (Currently in preview). The connector enables the use of DirectQuery to offload processing to Databricks. For more details, please check the online document.

Regards,

Frank

[Anonymous]

The Spark connector is very slow....

Queries in Azure Databricks are very fast less than 200ms, but data takes more than 10s to arrive to PowerBI Desktop...

Can I do anything to optimize performance?

Thank you,

Miguel

[Anonymous]

This an example of the query on Dax Studio:

This is the timings of the Azure Databricks server for the same querie:

Can someone explain me why is so slow.....

And its not a network problem, because if you run it from PowerBi Online it will take the same time...

Thank you,

Miguel

[djd]

I'm witnessing the same behaviour; < 2 seconds in Databricks and 10 - 20 seconds in Power BI.  Have spent many hours trying to troublehsoot unsuccessfully.

[yoshihirok]

Thank you @v-frfei-msft for your nice information

Awesome, Power BI Desktop can connect the data on Azure Databricks as DirectQuery

Just, I'm looking the information for share with partners, friends.

Yes, I will check the online document about Power BI Desktop and Azure Databriks.

and, I hope to add the Azure Databricks to data source supported by DirectQuery.

Regards,

Yoshihiro Kawabata