Extract Web Data and Create New Row Everyday with Scheduled Refresh

[itsme]

I am working on a personal PBI project that contains home values from Redfin and Zillow. In the past, I would go to those sites each day and record the estimated home values of my house. This data is saved in Excel on OneDrive. With a scheduled refresh, the Power BI report would refresh each day grabbing the updated data from that Excel spreadsheet.

As you see, this is not 100% automated as I would need to update the spreadsheet each day. So I created new queries that connect to Redfin and Zillow and add the current date, then appended it to the original dataset that has all the historic prices I manually inputed. The only problem is that the data set will not keep all historic values since this new method was implemented - it will only keep the most recent day and historic values from the old method.

Is there a way to add each day to the dataset so that when it refreshes tomorrow the dataset will keep today's value and so forth?

Thank you very much in advance!!

[v-frfei-msft · Accepted solution]

Hi @itsme,

Currecntly it is not possbile to stroe historic data when refresh data in power bi. There is an idea about that, you can vote it up to make the feature come sooner.

Also there is a way to work around. Please check this case.

Regards,

Frank

[itsme]

Thanks, definitely voted up that idea. I also saw the workaround - looks a bit complicated but I think I'll try it out when I get some more time. I always enjoy a good challenge! Thanks again @v-frfei-msft !

[v-frfei-msft]

Hi @itsme ,

Has your issue been solved? If so, kindly mark my answer as a solution to close the case. If any other question, feel free to let me know please.

Regards,

Frank