Automatic deployment (CICD) for a Dataset

[Christophe93]

Hi  everyone

Allow me please to expose a CICD question in a particular context/use case  :

-The update of a gold DATASET, quite large and therefore incremental
& let's say the update is only for DAX metrics formulas & so no need to recalculate/roll back old rows 

-We think that once deployed in Service (WS of DEV) a pipeline call will do the trick to push the DS into TEST & PROD
(Via API / Azure DEv ops extension / powershell)  

-Because from my memory ,  PBI pipelines only updates the metadata (without obligation of full refresh) & therefore we don't waste time recalculating all the cube

My questioning (and request for help...) will be more on the 1st KM  &  updating Dataset in DEV WS

Reminder: We are on a DS in incremental
-We want to avoid the recalculation of the whole dataset after deployment
-We don't want to use AML toolkit (unofficial MS tool ) to update metadata -  yet a good method
- We are afraid that the API --> Imports - Post Import In Group - REST API (Power BI Power BI REST APIs)Imports - Post Import In Grou... force a refresh of  the entire Dataset at the end

Please, is there a powerbi API that only updates the DAX formulas of a Dataset (metadata)? Like running a TMSL update script (<-- not sure already that this possibility exists at TMSL via SSMS)

Thank you in advance for your opinions please

Christophe

[Christophe93]

OK thanks a lot  Ibendlin

We have only ALM toolkit for  the moment  regarding these operations <--  

Metadata only deployment

" ...In some cases, you may not want to replace the dataset, especially with incremental refresh.

The dataset in Power BI Desktop could be much smaller than the one in the service.

If the dataset in the service has an incremental refresh policy applied, it may have several years of historical data that will be lost if the dataset is replaced.

Refreshing all the historical data could take hours and result in system downtime for users.

Instead, it's better to perform a metadata only deployment, which allows deployment of new objects without losing the historical data.

For example, if you've added a few measures you can deploy only the new measures without needing to refresh the data, saving a lot of time.

For workspaces assigned to a Premium capacity configured for XMLA endpoint read-write, compatible tools enable metadata only deployment.

For example, the ALM Toolkit is a schema diff tool for Power BI datasets and can be used to perform deployment of metadata only."

[Christophe93]

Thank you very much Ibendlin

In fact, the subject of this article is not really how to manage a gold data set.

But rather: Is there a Power BI Rest API to modify a DAX formula for a model (especially on incremental <-- super useful) ? 

Best regards

[lbendlin]

You may want to read this article

Advanced incremental refresh and real-time data with the XMLA endpoint in Power BI - Power BI | Micr...

[Christophe93]

I am the IT who made this gold / unique source of thruth dataset
And now (still IT) I would like to modify via API (ci/cd) the DAX formula of a measure without causing the refresh of all my dataset partitions  (DS on incremental)
An API  to update the DEV  Workspace <-- the "chain head"   on service , after pipeline will do the deployment job 

I don't think this PBI Rest  API exists, well for now I'm using AML toolkit to do the JOB
But it's a manual manipulation & the tool is not recognized by MS

Luckily the Premium deployment pipeline seems to only  deploy my new DAX metric formula

Why are you talking please ,  about loss of trust of my users here? I do not understand Ibendlin  ?

[lbendlin]

A golden dataset is supposed to be authoritative. If you make any changes to its logic you need to inform your users beforehand.  I am sure you are familiar with the setup for a certified dataset.  As soon as you make meta data changes to a certified dataset it will become decertified.

[lbendlin · Accepted solution]

So you want to update the DAX formulas of your golden dataset ?!? That kinda defeats its purpose.  You will lose user trust if you do that.  You would need to design a MOC process for these scenarios.