cannot fetch data from populated hive table on hdinsight spark cluster

[drzinks]

Hi,

I am processing data in hdinsight spark cluser and storing it into .parquet files.

I made external hive table basing on it and it is populated.

I want to visualise the data in power bi desktop.

It connects to spark hdinsight cluster, sees the tables, even sees schema (shows proper column names).

However it says table is empty. I cannot import data in any way (direct query/import).

I tried also to connect directly to azure storage, but it says access is denied.

Tried also 'hdinsight interactive query' - the same but the preview of data was available.

Most meaningful thing according to me in stack trace:

Resolution of the dependency failed, type = "Microsoft.PowerBI.Client.Windows.AnalysisServices.Relationships.IRelationshipManager", name = "(none)".
Exception occurred while: Calling constructor Microsoft.PowerBI.Client.Windows.AnalysisServices.Relationships.RelationshipManager(Microsoft.PowerBI.Client.Shared.IPowerBISettings powerBISettings, Microsoft.PowerBI.Client.Windows.Services.IReportManager reportManager).

Can anyoone help me with this ? I run out of ideas.

Cheers!

[v-qiuyu-msft]

Hi @drzinks, 

Was the .parquet file stored in Azure Data Lake Storage?

I would suggest you try the Azure Data Lake Storage Gen1 or Azure Data Lake Storage Gen2 connector in Power BI desktop to check if you are able to retrieve data. 

Best Regards,
Qiuyun Yu

[drzinks]

Hi @v-qiuyu-msft,

yes it is stored in blob storage, I think it is gen2: blob.core.windows.net.

I tried it but both with gen1, gen2 I cannot even connect : access is denied.

Kind regards.

[drzinks]

I cannot use the cluster credentials admin/**** in this case - it says admin account is blocked because tried many times bad password. Whilst when using other connectors like hdinsight spark it connects without issue.

[v-qiuyu-msft]

Hi @drzinks, 

When using Azure Data Lake Storage Gen2 connector in Power BI desktop, please ensure your account meets below requirements: 

• A storage account that has a hierarchical namespace. Follow these instructions to create one. This article assumes that you've created an account named myadlsg2.
• You are granted one of the following roles for the storage account: Blob Data Reader, Blob Data Contributor, or Blob Data Owner.

Best Regards,
Qiuyun Yu