Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.

Reply
Tim_Control
Regular Visitor

Data Collection / Incremental Refresh from Website - Is it Possible

Hi All,

 

I have been trying to establish a method for collecting data from a website.  Below is a link to the website.  Contained within this are a number of tables; "table 1" contains information on its course & speed, as well as when this information was last refreshed / updated.  For clarity when this is downloaded initially to Power Bi, it only creates a single row of data, which represents the vessels "live position"

https://www.vesselfinder.com/vessels/TANNER-IMO-0-MMSI-232038663

 

ideally, I would like to connect to this website, and re-download the latest course/speed information every 15 minutes, and save this information in a new row.

 

I have been able to Transform this data so far, as well as utilise "Use the first row as Headers" function.  However I'm currently unclear how to refresh it every 15 minutes (or another time interval) and continually add data to a new row.  I did try publishing the data and using a scheduled refresh, however this did not seem to work with "website" data, and also the scheduled refresh function appears to be better suited to scheduling in "days" rather than "minutes".

 

Any help which can be provided will be much appreciated.  I've just started using Power Bi today and am hoping to find a use for it.  But struggling at the moment.  So prescriptive guidance would be much appreciated.

 

Thanks,

Tim

1 REPLY 1
lbendlin
Super User
Super User

"ideally, I would like to connect to this website, and re-download the latest course/speed information every 15 minutes, and save this information in a new row."

 

Power BI has no memory. You need to do your data collection and archiving further upstream, for example via a curl script that appends the position to a text file on your PC. Then you can point Power BI to that text file.

Helpful resources

Announcements
Microsoft Fabric Learn Together

Microsoft Fabric Learn Together

Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City

PBI_APRIL_CAROUSEL1

Power BI Monthly Update - April 2024

Check out the April 2024 Power BI update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.