cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
harold
Regular Visitor

Request: Deployments pipelines should support centralized datasets

In this link it is recommended to use a centralized workspace with datasets.

https://docs.microsoft.com/en-us/power-bi/create-reports/deployment-pipelines-best-practices


However in Deployment Pipelines -> Deployment Settings, when creating a "Dataset Rule", the only option to use another dataset is to add new connection string.  This creates a new dataset in the Workspace.

 

This seems to contradict the suggestion to use Workspace with centralized datasets.

"Deployment pipelines best practices

If you're using centralized datasets that are used across the organization, we recommend that you create two types of workspaces:

  • Modeling and data workspaces - These workspaces will contain all the centralized datasets"

 

From my understanding, it should be possible to link a Dataset Rule to a Dataset in another Workspace.

This would allow to have one dataset per environment (e.g. DEV) for all pipelines.

 

 

 

 

1 ACCEPTED SOLUTION
Nimrod_Shalit
Power BI Team
Power BI Team

Hi @harold ,

Reports can be managed in pipelines even is they are connected to external datasets.

whenever you deploy report update across the stages, it will keep pointing to the same shared dataset.

You can also manage 'central datasets' WS in pipelines.

 

However, managing both datasets and connected reports in separate pipelines is still not working in an optimal way, as there's no built-in way to connect each dataset in each stage with the correlated reports in each stage.

 

We recommend that for now, you connect all reports in all stages to the Prod dataset. in the 'central datasets' pipeline, keep few reports for testing in each stage, so that any change in datasets will be verified on reports before hitting Prod.

 

If you still encounter problems in some reports, you can use the dev/ test stages to fix it in each pipeline, and deploy to Prod.

 

Hope it was clear.

View solution in original post

5 REPLIES 5
Nimrod_Shalit
Power BI Team
Power BI Team

Hi @harold ,

Reports can be managed in pipelines even is they are connected to external datasets.

whenever you deploy report update across the stages, it will keep pointing to the same shared dataset.

You can also manage 'central datasets' WS in pipelines.

 

However, managing both datasets and connected reports in separate pipelines is still not working in an optimal way, as there's no built-in way to connect each dataset in each stage with the correlated reports in each stage.

 

We recommend that for now, you connect all reports in all stages to the Prod dataset. in the 'central datasets' pipeline, keep few reports for testing in each stage, so that any change in datasets will be verified on reports before hitting Prod.

 

If you still encounter problems in some reports, you can use the dev/ test stages to fix it in each pipeline, and deploy to Prod.

 

Hope it was clear.

View solution in original post

This is not an acceptable solution, in my honest opinion. Here is an example of where the Deployment Pipeline functionality breaks down.

Let's say I have my "Finance Dataset" in the Production centralized workspace "Certified Datasets", and a "Finance" app with an associated deployment pipeline. Within that pipeline, there is a P&L report. I want to test an update we just made one of our database views, to fix an issue in the existing production P&L report. I obviously don't do this immediately in the production Finance Dataset, I want to do it in a test dataset. So I deploy the test dataset with my dataset updates to our "Test Datasets" workspace. 

What I would like to have, is the ability to set a "Dataset rule" for my P&L front-end .pbix file so that when I deploy it from Test workspace to the Prod workspace, it can switch the connection from the Finance Dataset in the Test Datasets workspace, to the Finance Dataset in the Certified Datasets workspace.


Right now, that isn't possible. I only have the option of retaining the connection that was established in the Test version of the P&L report throughout the deployment pipeline process.

What is even more frustrating is the lack of clarity on Microsoft's part for how to handle this situation (which many customers will run into, if they are following Microsoft's documented guidance on an Enterprise Power BI deployment).

If this functionality is not planned on coming to Deployment Pipelines, what is the recommended solution?

@daxman thanks for the thorough explanation. We understand the difficulties in implementing the desired use case. I can share that we are planning to address this issue and have multiple pipelines be 'aware' of each other when different items are connected across pipelines. 

We will start working on it this year, but i don't have a timeline to share yet.

Thank you for keeping everyone in the loop! Your quick response is much appreciated. Cheers

lbendlin
Super User III
Super User III

This may be a mixing of two concepts - or me misunderstanding the issue.

 

concept 1:  Ability to use shared datasets across multiple workspaces: allows disparate audiences to be served with the same set of master data, but filtered/restricted according to the audience.  Reduces development effort, avoids duplication of data, gets you closer to the "one source of truth"  fallacy. Sorry, ideal. I meant to say ideal.

 

concept 2: Ability to speed up development and testing proess without impacting actual production systems.  DEV and ITG workspaces can be pointed to datasets that have a reduced but still representative subset of the data, and can also emulate flush&fill events that may not be desirable in production

 

Neither concept needs deployment pipelines, but the second concept sure benefits from them.

Helpful resources

Announcements
PBI User Groups

Welcome to the User Group Public Preview

Check out new user group experience and if you are a leader please create your group

April Update

Check it Out!

Click here to read more about the April 2021 Updates!

MBAS Carousel

Sign up for our May 4th event!

May the fourth be with you, join us online!

secondImage

The largest Power BI virtual conference

100+ sessions, 100+ speakers, Product managers, MVPs, and experts. All about Power BI. Attend online or watch the recordings.

Top Solution Authors
Top Kudoed Authors