cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
giampitz
Helper I
Helper I

Power BI Dataflows or Azure Data Factory?

Every day, you need to load 10GB of data both from on-prem instances of SAP ECC, BW and HANA to Azure DL Store Gen2. This is only the first step of a job that will continue to transform that data using Azure Databricks, Data Lake Analytics and Data Factory.
What would you use for that load, Power BI (Premium) Dataflows or Azure Data Factory?
How do you think to orchestrate the steps considering the use of Power BI Dataflows together with Azure Data Factory? Maybe using Azure Automation? 
It is a real use case.

Thanks!

5 REPLIES 5
TomMartens
Super User II
Super User II

Hey,

 

this is just my first thought.

It seems, that this process is not owned by a business user. From y first understanding I would use ADF to load your data into ADLS Gen2.

 

Please be aware that currently Power BI expects a filesystem called powerbi inside the storage account: https://docs.microsoft.com/en-us/power-bi/service-dataflows-connect-azure-data-lake-storage-gen2

This might become useful in the future as you progress in your architecture.

 

Regards,
Tom



Did I answer your question? Mark my post as a solution, this will help others!

Proud to be a Super User!
I accept Kudos 😉
Hamburg, Germany

Hi @TomMartens,

 

It means Dataflows are thought only for business users? Instead, I think that IT departments could use PBI workspaces only for certain mash-up processes that store the data in Azure Datalake. Extremely more rapid to develop than ADF. Any official thoughts about it? ...  

It would be perfect to have power query source in adf too. This would close the circle!

 

Thanks!

 

 

 

 

@giampitz How did you eventually solve your challenge?

I am facing a similar issue: adf vs. dataflows?

Any conclusions?

Interested myself in some comparisions between the two.


@giampitz wrote:

It would be perfect to have power query source in adf too.

 


It would appear that this is coming: https://azure.microsoft.com/en-us/blog/analytics-in-azure-remains-unmatched-with-new-innovations/

 

"A new capability called Wrangling Data Flows, available in preview, gives users the ability to explore and wrangle data at scale. Wrangling Data Flows empowers users to visually discover and explore their data without writing a single line of code."

Helpful resources

Announcements
MBAS on Demand

2021 Release Wave 2 Plan

Power Platform release plan for the 2021 release wave 2 describes all new features releasing from October 2021 through March 2022.

July 2021 Update 768x460.png

Check it out!

Click here to read more about the July 2021 Updates

Power Query PA Forum 768x460.png

Check it out!

Did you know that you can visit the Power Query Forum in Power BI and now Power Apps

Top Solution Authors
Top Kudoed Authors