Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
paulroses
New Member

URL Scraping function to be limited to certain cells

Hello everyone,

 

i have a table with 3 columns looking like this:

trackingurl | Order Number | Shipment Status

This table gathers the above columns from another "master" table that contains many more columns and several rows for each order number (one row per item in each order).

 

I created the function below so that it goes through every row looking for a URL, scrapes it and delivers the result to column "Shipment Status".

 

let GetResults=(trackingurl) =>

let
Origine = Web.Page(Web.Contents(trackingurl)),
Data0 = Origine{0}[Data],
Children = Data0{0}[Children],
Children1 = Children{1}[Children],
Children2 = Children1{1}[Children],
Children3 = Children2{0}[Children],
Children4 = Children3{0}[Children],
Children5 = Children4{2}[Children],
Children6 = Children5{0}[Children],
Children7 = Children6{0}[Children],
Children8 = Children7{0}[Children],
Children9 = Children8{0}[Children],
#"Rinominate colonne" = Table.RenameColumns(Children9,{{"Text", "Shipment Status"}})
in
#"Rinominate colonne"

in GetResults

 

What i wish to achieve, is for the function to only scrape the URLs that correspond to any "Shipment Status" different from "Delivered" in order to avoid going through every single row on every update.

 

Can anyone help with this?

Thank so much in advance

Paolo

1 ACCEPTED SOLUTION
lbendlin
Super User
Super User

Power BI has no memory.  It will always start from scratch.  If you need to keep track of prior runs then you need to do that before the data hits Power BI.

View solution in original post

4 REPLIES 4
lbendlin
Super User
Super User

Power BI has no memory.  It will always start from scratch.  If you need to keep track of prior runs then you need to do that before the data hits Power BI.

Yes, i know that it wouldn't be possible to modify PowerBI's logic, but i hoped to find a "thinking out of the box" solution. 

Maybe an extra table where "delivered" rows are moved to hence removed from the table where the function scrapes URLs?

I would think that "you need to do that before the data hits Power BI"  is pretty much the same as  "thinking out of the box" ?

I guess 🙂

Thanks for your help!

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.