cancel
Showing results for 
Search instead for 
Did you mean: 

cannot fetch data from populated hive table on hdinsight spark cluster

Hi,

I am processing data in hdinsight spark cluser and storing it into .parquet files.

I made external hive table basing on it and it is populated.

I want to visualise the data in power bi desktop.

It connects to spark hdinsight cluster, sees the tables, even sees schema (shows proper column names).

However it says table is empty. I cannot import data in any way (direct query/import).

I tried also to connect directly to azure storage, but it says access is denied.

Tried also 'hdinsight interactive query' - the same but the preview of data was available.

Most meaningful thing according to me in stack trace:

Resolution of the dependency failed, type = "Microsoft.PowerBI.Client.Windows.AnalysisServices.Relationships.IRelationshipManager", name = "(none)".
Exception occurred while: Calling constructor Microsoft.PowerBI.Client.Windows.AnalysisServices.Relationships.RelationshipManager(Microsoft.PowerBI.Client.Shared.IPowerBISettings powerBISettings, Microsoft.PowerBI.Client.Windows.Services.IReportManager reportManager).

Can anyoone help me with this ? I run out of ideas.

Cheers!

Status: New
Comments
Moderator

Hi @drzinks

 

Was the .parquet file stored in Azure Data Lake Storage?

I would suggest you try the Azure Data Lake Storage Gen1 or Azure Data Lake Storage Gen2 connector in Power BI desktop to check if you are able to retrieve data. 

 

Best Regards,
Qiuyun Yu

 

New Member

Hi @v-qiuyu-msft,

yes it is stored in blob storage, I think it is gen2: blob.core.windows.net.

I tried it but both with gen1, gen2 I cannot even connect : access is denied.

 

Kind regards.

New Member
I cannot use the cluster credentials admin/**** in this case - it says admin account is blocked because tried many times bad password. Whilst when using other connectors like hdinsight spark it connects without issue.
Moderator

Hi @drzinks

 

When using Azure Data Lake Storage Gen2 connector in Power BI desktop, please ensure your account meets below requirements: 

 

  • A storage account that has a hierarchical namespace. Follow these instructions to create one. This article assumes that you've created an account named myadlsg2.
  • You are granted one of the following roles for the storage account: Blob Data ReaderBlob Data Contributor, or Blob Data Owner.

 

Best Regards,
Qiuyun Yu