Power BI Dataflows or Azure Data Factory?

[giampitz]

Every day, you need to load 10GB of data both from on-prem instances of SAP ECC, BW and HANA to Azure DL Store Gen2. This is only the first step of a job that will continue to transform that data using Azure Databricks, Data Lake Analytics and Data Factory.
What would you use for that load, Power BI (Premium) Dataflows or Azure Data Factory?
How do you think to orchestrate the steps considering the use of Power BI Dataflows together with Azure Data Factory? Maybe using Azure Automation? 
It is a real use case.

Thanks!

[TomMartens]

Hey,

this is just my first thought.

It seems, that this process is not owned by a business user. From y first understanding I would use ADF to load your data into ADLS Gen2.

Please be aware that currently Power BI expects a filesystem called powerbi inside the storage account: https://docs.microsoft.com/en-us/power-bi/service-dataflows-connect-azure-data-lake-storage-gen2

This might become useful in the future as you progress in your architecture.

Regards,
Tom

[giampitz]

Hi @TomMartens,

It means Dataflows are thought only for business users? Instead, I think that IT departments could use PBI workspaces only for certain mash-up processes that store the data in Azure Datalake. Extremely more rapid to develop than ADF. Any official thoughts about it? ...  

It would be perfect to have power query source in adf too. This would close the circle!

Thanks!

[michaelsh]

@giampitz How did you eventually solve your challenge?

I am facing a similar issue: adf vs. dataflows?

Any conclusions?

[jcarville]

Interested myself in some comparisions between the two.

[zegor]

---

@giampitz wrote:

It would be perfect to have power query source in adf too.

---

It would appear that this is coming: https://azure.microsoft.com/en-us/blog/analytics-in-azure-remains-unmatched-with-new-innovations/

"A new capability called Wrangling Data Flows, available in preview, gives users the ability to explore and wrangle data at scale. Wrangling Data Flows empowers users to visually discover and explore their data without writing a single line of code."