Delta Tables with Liquid Clustering Applied

[dozer_1985]

Hi all,

Looking for some general advice if anyone out there can help. I'm trying to access data in a Blob Storage Container in Azure Data Lake Storage Gen 2. Liquid Clustering has been applied to the tables and it seems Power BI (Power Query) can't currently connect to such tables. 

Does anyone know if this feature is going to be available any time soon or if there is a workaround for it? 

Many thanks 

[lbendlin]

Read data from a clustered table

You can read data in a clustered table using any Delta Lake client that supports reading deletion vectors. For best query results, include clustering keys in your query filters, as in the following example:

SQLCopy

 

SELECT * FROM table_name WHERE cluster_key_column_name = "some_value";

How are you connecting, via DataBricks?

[dozer_1985]

Hi, 

Thanks for the response. Not using Databricks, we only use Databricks to create views, Power BI uses Power Query functions to access directly to Storage Accounts. 

We opted for that approach as it takes too long to spin up a cluster when people want to access, alternative is to run a cluster all day, but we didn't want to do that due to cost. 

[lbendlin]

I assume you have seen this? Best practices for Power BI on Databricks (crossjoin.co.uk)