cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
felipejeunon
Regular Visitor

(Help) Modelling Using 2 Dataflows

Hello everybody.

 

I read some documentations about modeling using two dataflows and one dataset. The first dataflow would bring the data from the source without any transformation being applied (raw data). The second dataflow would be the layer where we would apply the necessary transformations. When we use a dataflow as a source to create a second dataflow, it comes in the model as an entity, and when we try to do some transformation we get the error message bellow. I am also leaving the image with the modeling I want to implement.

 

My question is: how do I apply transformations to the second Dataflow since I can't because the first one exists as an entity? What is the correct way to do this modeling?

 

Thank you very much.

 

https://dataandai.files.wordpress.com/2020/06/organizedataflows_linkedentitiesnamingconvention.png 

https://i.imgur.com/acaycqg.png 

1 ACCEPTED SOLUTION

A reference is just a pointer, nothing is duplicated. When a dataflow refreshes, data is read from the source and stored in data lake storage behind the scenes. A second dataflow will read from that store.

If you plan on doing large transformations (aggregations, merges, sorts) or have a large volume of data, seperate ingest and transform dataflows can be helpful, or if you want to leverage the ingest dataflow in another way.

View solution in original post

5 REPLIES 5
felipejeunon
Regular Visitor

Hello everybody.

 

I read some documentations about modeling using two dataflows and one dataset. The first dataflow would bring the data from the source without any transformation being applied (raw data). The second dataflow would be the layer where we would apply the necessary transformations. When we use a dataflow as a source to create a second dataflow, it comes in the model as an entity, and when we try to do some transformation we get the error message bellow. I am also leaving the image with the modeling I want to implement.

 

My question is: how do I apply transformations to the second Dataflow since I can't because the first one exists as an entity? What is the correct way to do this modeling?

 

Thank you very much.

 

https://dataandai.files.wordpress.com/2020/06/organizedataflows_linkedentitiesnamingconvention.png 

https://i.imgur.com/acaycqg.png 

jeffshieldsdev
Impactful Individual
Impactful Individual

Right click on it and create a Reference query to the Linked Table, then make the transformations you need. If the Workspace is not in Premium capacity, you'll have to uncheck "Enable Load" on the Linked Table.

Hi Jeff.

 

Thank you for the answer.

 

When I create a reference, I'm doubling the base, in terms of data, right?
Given this, is it more interesting that I have a single dataflow bringing the data from the base already with the transformation or modeling as I initially proposed?

 

Thanks.

A reference is just a pointer, nothing is duplicated. When a dataflow refreshes, data is read from the source and stored in data lake storage behind the scenes. A second dataflow will read from that store.

If you plan on doing large transformations (aggregations, merges, sorts) or have a large volume of data, seperate ingest and transform dataflows can be helpful, or if you want to leverage the ingest dataflow in another way.

View solution in original post

Understood. Perfect Jeff.

 

Thank you so much. Have a good Day!

Helpful resources

Announcements
PBI_User Group Leader_768x460.jpg

Manage your user group events

Check out the News & Announcements to learn more.

Welcome Super Users.jpg

Super User Season 2

Congratulations, the new Super User Season 2 for 2021 has started!

Community Connections 768x460.jpg

Community & How To Videos

Check out the new Power Platform Community Connections gallery!