cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Helper III
Helper III

Web Scraping a table, Identify Changes

Hello everyone!

 

Im scraping a web table to see when new products are released from a supplier.  As they dont have a "new product" list, Im using their "full Product List" table, and want to compare old and new lists to see changes...

 

I have the PBI pulling the updated table data (latest refresh), but how do you recommend creating the old comparrion table in preparatrion to compare with the new data pull?  I tried adding a System Date to the table but I cant make it static and it keeps incrementing with each refresh.

 

Any ideas?

1 ACCEPTED SOLUTION
Super User III
Super User III

Power BI is not built to handle prior data and compare with new data. It is an all or nothing refresh.

 

There are some hacks that can simulate partitioning off data in an ad-hoc incremental refresh, which you can read about here, but I'm not sure I'd put that in production. Also, everytime you republish your PBIX file, it wipes out the history unless you got really fancy and use the ALM Toolkit to preserve partions.

It is a very cool hack. It is a bit too Rube Goldberg for me to rely on for production reporting.



Did I answer your question? Mark my post as a solution!
Did my answers help arrive at a solution? Give it a kudos by clicking the Thumbs Up!

DAX is for Analysis. Power Query is for Data Modeling


Proud to be a Super User!

MCSA: BI Reporting

View solution in original post

2 REPLIES 2
Super User III
Super User III

Power BI is not built to handle prior data and compare with new data. It is an all or nothing refresh.

 

There are some hacks that can simulate partitioning off data in an ad-hoc incremental refresh, which you can read about here, but I'm not sure I'd put that in production. Also, everytime you republish your PBIX file, it wipes out the history unless you got really fancy and use the ALM Toolkit to preserve partions.

It is a very cool hack. It is a bit too Rube Goldberg for me to rely on for production reporting.



Did I answer your question? Mark my post as a solution!
Did my answers help arrive at a solution? Give it a kudos by clicking the Thumbs Up!

DAX is for Analysis. Power Query is for Data Modeling


Proud to be a Super User!

MCSA: BI Reporting

View solution in original post

Thank you!!!  Sad to see, but it makes sense... thanks for the swift reply.

Helpful resources

Announcements
secondImage

Happy New Year from Power BI

This is a must watch for a message from Power BI!

December Update

Check it Out!

Click here to read more about the December 2020 Updates!

Community Blog

Check it Out!

Click here to read the latest blog and learn more about contributing to the Power BI blog!

Get Ready for Power BI Dev Camp

Get Ready for Power BI Dev Camp

Mark your calendars and join us for our next Power BI Dev Camp!.

Top Solution Authors
Top Kudoed Authors