Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
hubix2000
New Member

PowerBI with Azure Data Lake Store

Hi,

 

we already run all our operations on Azure SQL databases. Now we are going to invest into Azure Data Lake Store and like to integrate it with PowerBI. Currently there is no external data source to ADLS available.

 

Can you provide us an estimate roughly when a beta phase might start and if the external data source will support direct query?

 

Best Regards

Jörg

12 REPLIES 12
desertislesql
Helper III
Helper III

I would like to use an Azure Data Lake for a Power BI source as well. Any information on this would be appreciated.

I found out that ir you are looking to use an Azure Data Lake as your data store, you cannot directly connect to it online. If you want to use an Azure Data Lake as a data source, you will need to use an ODBC data source in Power BI Desktop.

If I have to use an ODBC data source as I want to publish, is there any reference as to how to set this up?

Not sure why we talk about ODBC here, since PowerBI has out of the box support for Azure Data Lake Store.

It was a follow up to desertislesql post above in this thread...

I found out that ir you are looking to use an Azure Data Lake as your data store, you cannot directly connect to it online. If you want to use an Azure Data Lake as a data source, you will need to use an ODBC data source in Power BI Desktop.

 

I am testing out PowerBI with our AppInsights data. One of the options is to move the data into DataLake and use that as a source. I can access everything fine from PowerBI, but when I publish, the connection doesnt work. I keep getting prompted to login.

 

Doing a search, I found this reference which aligns with my experiance. Having gotten the data into DataLake, I am trying any option I can find to get the data into PowerBI.

 

I would have thought using ODBC was a bit of hack, and if there is a way to get this working "natively" then I would like to use it.

I managed to get data from a Data Late Store into PowerBI desktop, but not the way I would like it:

  • PowerBI Desktop, Version: 2.32.4307.641 64-bit (February 2016)
  • Get Data / Azure / Microsoft Azure Data Lake Store (Beta)
  • Provide the adl:// or swebhdfs:// path
  • It will show you a preview with columns like "content, name, extension, .."
  • Click on "Edit"
  • Click on the link "Binary" in the first column ("Content")
  • Now you can see the the data as it appears in the file itself.

 

how can you we load all the files for all the files.. I am not sure clicking Binary would work if I want to work with all of the data? 

Would it 

What happens when we have files stored in multiple sub folders by timestamp?

 

When I browse binary data and check the scripts inside Power BI advance editor I get follow:

 

let
Source = DataLake.Contents("https://xxxxxxxx.azuredatalakestore.net"),
Content = Source[Content],
Content1 = Content{0},
#"2017" = Content1{[Name="2017"]}[Content],
#"04" = #"2017"{[Name="04"]}[Content],
#"20" = #"04"{[Name="20"]}[Content],
#"14_0_f44e6b5c95fa49dfa3b78c89878777d1 json" = #"20"{[Name="14_0_f44e6b5c95fa49dfa3b78c89878777d1.json"]}[Content],
#"Imported CSV" = Csv.Document(#"14_0_f44e6b5c95fa49dfa3b78c89878777d1 json",[Delimiter=",", Columns=17, Encoding=1252, QuoteStyle=QuoteStyle.None]),

 

which tells I went all the way to 2017-04-20 and picked that day file. 

 

Can I setup this to load all files as they come?

 

Shabbir

 

That's a very interesting solution. Thanks for sharing. Here's a question, are you able to refresh the data extracted from the datastore in the personal gateway?

 

I published the report to PowerBI.com and I could refresh the data there. Don't need the gateway, there's no local to cloud data movement involved here. (Direct connection). I just needed to re-enter my credentials for the DataLake reference.

I am getting the followin error, while trying to re-enter the credentials

Failed to update data source credentials: The credentials you provided for the data source are invalid. Please check the credentials for all the data sources.Hide details

Activity ID:    f3753c07-f1f2-4d8e-b55e-af0a9b0ec4e8
Request ID:    323a3391-1a49-6c15-3ed8-c0792e4c2610
Status code:    400
Time:    Wed Nov 16 2016 13:26:42 GMT+0100 (W. Europe Standard Time)
Version:    13.0.1700.581
Cluster URI:    https://wabi-north-europe-redirect.analysis.windows.net
Details:    The credentials provided for the DataLake source are invalid. (Source at https://xxx.azuredatalakestore.net/yyy.)

hubix2000
New Member

Hi,

 

any answer is welcome... 

 

Jörg

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors