cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
oKo Frequent Visitor
Frequent Visitor

How to use data stored in ADLS Gen 2

Hi Everyone,

 

After successfully connect power bi with Azure Data Lake Storage (ADLS Gens 2) and create dataflow to ETL data to the storage. I need to load that data into Power BI and other Applications.  However, Power BI Dataflow connection does not support direct query to apply greate functions such as Aggregations.

 

Is it possible to use ADLS Gen 2 as Azure SQL DataWarehouse External Data Source? If yes, please help me with set up. If no, Are there any better way to use that data?

 

Thanks all

 

 

 

 

1 ACCEPTED SOLUTION

Accepted Solutions
Microsoft v-yulgu-msft
Microsoft

Re: How to use data stored in ADLS Gen 2

Hi @oKo ,

 

Based on research, below link talks about how ADLS Gen 2 integrates with Azure SQL DataWarehouse. 

 

The prepared data is also loaded into staging tables in an Azure SQL Data Warehouse, where it is transformed into a dimensional model.  Reference: New tutorial explores data sharing between Power BI and Azure

 

Besides, please have a look at this blog which shows Access Data from Azure Data Lake Store using Polybase with Azure Data Warehouse.

 

Best regards,

Yuliana Gu

Community Support Team _ Yuliana Gu
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

3 REPLIES 3
Microsoft v-yulgu-msft
Microsoft

Re: How to use data stored in ADLS Gen 2

Hi @oKo ,

 

Based on research, below link talks about how ADLS Gen 2 integrates with Azure SQL DataWarehouse. 

 

The prepared data is also loaded into staging tables in an Azure SQL Data Warehouse, where it is transformed into a dimensional model.  Reference: New tutorial explores data sharing between Power BI and Azure

 

Besides, please have a look at this blog which shows Access Data from Azure Data Lake Store using Polybase with Azure Data Warehouse.

 

Best regards,

Yuliana Gu

Community Support Team _ Yuliana Gu
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

oKo Frequent Visitor
Frequent Visitor

Re: How to use data stored in ADLS Gen 2

Thank @v-yulgu-msft  for your support,

 

I am successfully connected ADLS with Data WareHouse via Polybase External Data Souce.

 

However, every time dataflow refreshed, a new snapshot created without deleting the old one.

 

When query Azure SQL DataWarehouse gets all data in the folder => data is duplicated.

 

Is it possible to delete old snapshots after data refreshed or how to query only the newest snapshot.

 

Thanks & Best Regards

gacecilidw Occasional Visitor
Occasional Visitor

Re: How to use data stored in ADLS Gen 2

Hi @oKo - what format is your data stored in in adls? I expected it to look like described here: https://docs.microsoft.com/en-us/common-data-model/data-lake (CDM entity folders with csv files inside) - but all my files and folders except model.json are stored with the extension .snapshot, and not able to be downloaded... 

Helpful resources

Announcements
New Topics Started Badges Coming

New Topics Started Badges Coming

We're releasing new versions of the badge that everyone's talking about. ;) Check your inbox for notifications.

MBAS 2020

Save the new date (and location)!

Our business applications community is growing—so we needed a different venue, resulting in a new date and location. See you there!

Difinity Conference

Difinity Conference

The largest Power BI, Power Platform, and Data conference in New Zealand

Power Platform 2019 release wave 2 plan

Power Platform 2019 release wave 2 plan

Features releasing from October 2019 through March 2020

Top Solution Authors
Top Kudoed Authors (Last 30 Days)