cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Tim_Control
Regular Visitor

Data Collection / Incremental Refresh from Website - Is it Possible

Hi All,

 

I have been trying to establish a method for collecting data from a website.  Below is a link to the website.  Contained within this are a number of tables; "table 1" contains information on its course & speed, as well as when this information was last refreshed / updated.  For clarity when this is downloaded initially to Power Bi, it only creates a single row of data, which represents the vessels "live position"

https://www.vesselfinder.com/vessels/TANNER-IMO-0-MMSI-232038663

 

ideally, I would like to connect to this website, and re-download the latest course/speed information every 15 minutes, and save this information in a new row.

 

I have been able to Transform this data so far, as well as utilise "Use the first row as Headers" function.  However I'm currently unclear how to refresh it every 15 minutes (or another time interval) and continually add data to a new row.  I did try publishing the data and using a scheduled refresh, however this did not seem to work with "website" data, and also the scheduled refresh function appears to be better suited to scheduling in "days" rather than "minutes".

 

Any help which can be provided will be much appreciated.  I've just started using Power Bi today and am hoping to find a use for it.  But struggling at the moment.  So prescriptive guidance would be much appreciated.

 

Thanks,

Tim

1 REPLY 1
lbendlin
Super User
Super User

"ideally, I would like to connect to this website, and re-download the latest course/speed information every 15 minutes, and save this information in a new row."

 

Power BI has no memory. You need to do your data collection and archiving further upstream, for example via a curl script that appends the position to a text file on your PC. Then you can point Power BI to that text file.

Helpful resources

Announcements
November 2022 Update

Check it Out!

Click here to read more about the November 2022 updates!

Microsoft 365 Conference â__ December 6-8, 2022

Microsoft 365 Conference - 06-08 December

Join us in Las Vegas to experience community, incredible learning opportunities, and connections that will help grow skills, know-how, and more.