cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Highlighted
Advocate II
Advocate II

Dataflows - how to use staging tables?

Hi there,

 

I'm moving some queries to Dataflows but I've come across an issue I can't solve.

 

In PBI desktop, I've created a few tables I use to import, clean and transform the raw data before splitting it in different tables. Once I've created the tables that reference these, I uncheck the "Enable load" so they are not loaded to the report.

 

I tried to do the same in Dataflows, but when querying the dataflow all tables are empty because the staging tables are not loaded.

 

Of course I can't just "enable load" for the staging tables, because they contain columns of the type "table" or "record", which won't load as an entity (well, they will load but with an extra few steps that remove any column of these types).

 

Any suggestions on how to fix this? I don't want to rebuild the queries to add the transformations made on the staging tables to every table that's built from them...

 

NAOS

 

 

1 REPLY 1
Highlighted
Resolver I
Resolver I

Re: Dataflows - how to use staging tables?

Data doesn't get loaded into just by creating an entity in the Dataflow. You can only see the preview.

You will have to load the data atleast once.

When you create a new entity in dataflow, save and close. Changes will be saved and a popup appears to Refresh Now. Only then data from the source will be imported into the underlying Datalake storage folders (CDM Folders).

 

If you want to create staging tables and tabes on top of it, you have to first create the data flows for Staging tables, load the data and create linked entities on top of it.

 

You can load the raw entities into Staging dataflows , and apply transformations on your Transformation dataflows.

 

Hope i answered your Query. If yes, please accept this as a solution.

 

Thanks,

Nishanth.

Helpful resources

Announcements
Community Blog

Community Blog

Visit our Community Blog for articles, guides, and information created by fellow community members.

Using the Community

Using the Community

Need help with the Power BI Community? Our 'Using the Community' support articles are a great place to start.

Community Summit North America

Community Summit North America

Innovate, Collaborate, Grow. The top training and networking event across the globe for Microsoft Business Applications

Power Platform 2020 release wave 2 plan

Power Platform 2020 release wave 2 plan

Features releasing from October 2020 through March 2021

Top Solution Authors
Top Kudoed Authors