I am trying to optimize my company's shared dataset code promotion procedure. I suspect that we're doing something wrong.
We use PBI in the cloud (not Report Server), shared capacity (not Premium). Our deployment "pipeline" goes Dev-Test-UAT-Prod. We have a separate app workspace to represent each of these stages. We publish our pbix files by placing them into a OneDrive folder that is synced with a SharePoint document library (actually, 1 library for each pipeline stage), and SharePoint automatically persists these files into the PBI service (i.e., 1:1 correspondence between SharePoint document library and PBI service app workspace). We have a handful of "model" pbix files that generate datasets (no visuals, just model). We have lots of "report" pbix files that consume from these datasets. Sometimes, we need to update both the model pbix file and the affected report pbix files. With our pipeline, we're finding this to be a very cumbersome process. As an example of the code promotion from UAT to Prod for this situation, we have to:
1) on our local machines, place the UAT model file into the Prod OneDrive folder (almost no wait time to sync with Prod SharePoint doc library)
2) wait for the new Prod model file (current UAT model file) to persist from the Prod SharePoint doc library to the Prod PBI service app workspace (usually have a wait time of 15 minutes)
3) run a manual refresh of the model file's dataset in the Prod PBI service app workspace (since the model data will inherit the old data from our local machines) (refresh can take about an hour)
4) on our local machines, update the UAT report file's dataset to be Prod (Transform data \ Data source settings \ select the model dataset in the Prod PBI app workspace)
5) on our local machines, place the UAT report file into the Prod OneDrive folder
6) wait for the new Prod report file to persist from the Prod SharePoint doc library to the Prod PBI service app workspace (again, about 15 minutes)
7) update the Prod PBI app
Seven steps isn't too bad for just 1 stage. But, when scaled by our number of target stages (i.e., 3), this becomes a very slow process. Hence, this doesn't seem right.
How should this be done? Or, how can this be optimized? (To be sure, we will not be upgrading to Premium, and we will not grant testers / consumers access to the app workspaces -- only to the apps.)
