Dataflows - how to use staging tables?

[NAOS]

Hi there,

I'm moving some queries to Dataflows but I've come across an issue I can't solve.

In PBI desktop, I've created a few tables I use to import, clean and transform the raw data before splitting it in different tables. Once I've created the tables that reference these, I uncheck the "Enable load" so they are not loaded to the report.

I tried to do the same in Dataflows, but when querying the dataflow all tables are empty because the staging tables are not loaded.

Of course I can't just "enable load" for the staging tables, because they contain columns of the type "table" or "record", which won't load as an entity (well, they will load but with an extra few steps that remove any column of these types).

Any suggestions on how to fix this? I don't want to rebuild the queries to add the transformations made on the staging tables to every table that's built from them...

NAOS

[Anonymous]

Data doesn't get loaded into just by creating an entity in the Dataflow. You can only see the preview.

You will have to load the data atleast once.

When you create a new entity in dataflow, save and close. Changes will be saved and a popup appears to Refresh Now. Only then data from the source will be imported into the underlying Datalake storage folders (CDM Folders).

If you want to create staging tables and tabes on top of it, you have to first create the data flows for Staging tables, load the data and create linked entities on top of it.

You can load the raw entities into Staging dataflows , and apply transformations on your Transformation dataflows.

Hope i answered your Query. If yes, please accept this as a solution.

Thanks,

Nishanth.