How use Deployment Pipelines with Dataflows in Power BI?

[Anonymous]

Hello to everyone, i have 3 workspaces here with dataflows and datasets.

My workspaces are "Dev", "Test" and "Prod".

Lets supposed that my dataflows are : "A" , "B", "C" ....

I would like to know how to deal with datasets that have to connect in multiple dataflows, (e.g. Dataset 1 : "A","D" and "F") .

I read in microsoft best pratices for Deployment Pipelines (https://docs.microsoft.com/en-us/power-bi/create-reports/deployment-pipelines-best-practices) that i need to use parameters, but i dont know how the parameters will change dynamically when the datasets go for another workspace.

Thank you for any sugestions

[Nimrod_Shalit]

For dataflows and datasets that are managed in the same pipeline, the pipeline automatically changes the connections so that Dev dataset will connect to dev dataflow, and Test dataset will connect to Test dataflow. You can also control that yourself through parameters and parameter rules, which requires the user to configure it once.
But if you just want it to work that dev points to dev, and test points to test, it will just work.

[ptacquet]

Hi, 

Thank you for your quick response.

We did a lot of tests. Datasets are always linked to dev dataflows after deploy. Because, in the m query there's the ids of workspaces and dataflows...

[Nimrod_Shalit]

If datasets and dataflows are in the same pipeline and workspace/ stage,

when they are deployed they will reconnect in Test/ Prod stages.

If they are not in the same pipeline, then today we don't have a good solution for that yet, but we are working on it- Deployment pipelines - multiple pipelines working together | Microsoft Docs

[Al345]

I have had the same experience as @ptacquet. I have one deployment pipeline with a dev, test, and production stage. When I create something in dev and deploy it to test, the dataflow still references the dev DSN (we use an ODBC connection with a DSN to connect to an AWS instance). I can go in and manually update the DSN in the test workspace but if I deploy from dev again it will be overwritten with the dev DSN. 

The real issue here is that when I deploy a report that pulls from the dev dataflow, it doesn't seem to keep a connection to any dataflow. When I download the report and go to the advanced editor, there are no queries in the queries pane.  

[Nimrod_Shalit]

It's two separate issues:

for dataflow-->datasource connection, you should configure rules on the dataflow to make sure you are pointing to you Test/Prod data source respectively. it can be done directly on the data source, or by using parameters.

As for the dataset-->dataflow connection, it sounds like a bug. this shuold be automatically changed by the pipeline. please open a support ticket.

[ptacquet]

We have the same problem. 

With the pipeline we can deploy dataflows between workspaces.

But we can't deploy datasets based on thoses datflows. They are always based on dev worspace.

Wich parameters to add ? the id of the dataflow ? it's too complicated and can generate errors...

What is the best practice ?

[lbendlin]

Deployment pipelines allow you to use different data sources for the DEV/ITG/PRO stages. Doesn't really matter if these data sources are dataflows or other, native sources.

There is no problem to point all three workspaces to the same dataflows sources, but you could also have a scenario where DEV and ITG point to a smaller version of the dataflow than PRO.

The data source switch is done via the parameters.

[Anonymous]

lbendlin, thanks for your help.

So are you telling me that I should use the parameters there in the power query, and configure each workspace according to each data source? This seems a little too manual. I thought about doing this dynamically, for example: IF workspace "Dev" Then Dataflow "A". But this is just an idea that I had, I would really like to know what is the correct recommendation for this case.

Another problem I am having is that I chose a list of values ​​for the parameter, but in the dataset settings I need to enter the ID, the list does not appear for me to choose. The solution I found for this is to enter the values ​​in the parameter description.

(I dont know how to insert images, i will insert just links to images)

https://ibb.co/HXSBR8B
https://ibb.co/C1sGBhk
https://ibb.co/8DqrKxQ

Thank you again for your help.