Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
Willgart
Helper II
Helper II

automated testing of a powerbi desktop report?

Hi,

 

how I can implement some automated testing around PowerBI reports?

We have multiple Tabular models and SSAS cubes in place, and PowerBI desktop report connected to them.

I want to make sure that the reports still working when we release a new version of the source cube or model.

 

how can I do this?

(I will certainly ask the same things for Excel and SSRS reports, but for now we focus on powerbi)

 

 

6 REPLIES 6
Brendon
Frequent Visitor

I'm at a client with a similar setup - shared models backing multiple reports. Their objective was to make sure that changes to the model didn't break any of the reports as well as that visuals on reports were configured to display the correct information, i.e., had the correct filters and DAX calculations. I couldn't find any tools at the time so we implemented a Selenium framework that extracted data from each visual in a report and compared it to a SQL query against the data warehouse.

 

This works but is a bit of a bottleneck due to the amount of development effort required to build and maintain the test framework assets and running ~2000 Selenium tests also takes a few hours. I've since built PowerTester as a dedicated Power BI test automation solution to get around some of the challenges we faced when using a Selenium framework. 

Anonymous
Not applicable

Hello  @Brendon ,

I have a simialr situation. One central dataset and multiple reports built around the central dataset. 

 

I am constantly worried that an inadvertent change to the central dataset might break the dozens of satellite reports.

 

I am considering Selenium.  Could you please explain,  what did you mean by "Selenium tests also takes a few hours".

 

Thank you,

Sau

 

v-qiuyu-msft
Community Support
Community Support

Hi @Willgart,

 

After you update data and process the model on SSAS side, you can refresh the dataset in Power BI manually or use schedule refresh to update the data.

 

Best Regards,
Qiuyun Yu

Community Support Team _ Qiuyun Yu
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

well... first  I'm using SSAS and Tabular to have shared models instead of multiple standalone powerbi files. (this also allows me to have model of any size)

so my powerbi are in directquery mode connected to these cube and models.

 

second I want to do automatic testing, not manual testing.

I want to make sure that all the existing reports did not suffer any issue due to a renaming or something like this.

and i want this done automatically to be done in minutes instead of hours of manual processes.

(the data side will be validated directly in the cube and tabular model, this is easy, and the main reason why I want to use direct query)

 

and if we have powerbi report with extracted data (which shoukd be a rare case) I'll want to test the refresh process.

 

if 1 of the report failed, we have to correct it before we releases our new cube or tabular model in prod.

We dont want to wait to be in prod and hearing a user crying...

 

Hi Willgart,

 

I am currently in the same situation.

Have you figured any solution, appreciate if you share your findings.

 

I have 2 technologies in mind, CodedUI or Selenium+C#.

 

 

Thanks

 

-Kishore

i am not sure if you can create a trace on the power bi service/ gateway, if there is a log that is generated with the values you require (maybe look in the audit log settings?), if you could access that data and do a power bi report and set an alert on it with a threshhold of errors > 0, that would be one way of checking for errors. 





If I took the time to answer your question and I came up with a solution, please mark my post as a solution and /or give kudos freely for the effort 🙂 Thank you!

Proud to be a Super User!




Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.