Data Collection / Incremental Refresh from Website - Is it Possible
I have been trying to establish a method for collecting data from a website. Below is a link to the website. Contained within this are a number of tables; "table 1" contains information on its course & speed, as well as when this information was last refreshed / updated. For clarity when this is downloaded initially to Power Bi, it only creates a single row of data, which represents the vessels "live position"
ideally, I would like to connect to this website, and re-download the latest course/speed information every 15 minutes, and save this information in a new row.
I have been able to Transform this data so far, as well as utilise "Use the first row as Headers" function. However I'm currently unclear how to refresh it every 15 minutes (or another time interval) and continually add data to a new row. I did try publishing the data and using a scheduled refresh, however this did not seem to work with "website" data, and also the scheduled refresh function appears to be better suited to scheduling in "days" rather than "minutes".
Any help which can be provided will be much appreciated. I've just started using Power Bi today and am hoping to find a use for it. But struggling at the moment. So prescriptive guidance would be much appreciated.
"ideally, I would like to connect to this website, and re-download the latest course/speed information every 15 minutes, and save this information in a new row."
Power BI has no memory. You need to do your data collection and archiving further upstream, for example via a curl script that appends the position to a text file on your PC. Then you can point Power BI to that text file.