cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
paulroses
New Member

URL Scraping function to be limited to certain cells

Hello everyone,

 

i have a table with 3 columns looking like this:

trackingurl | Order Number | Shipment Status

This table gathers the above columns from another "master" table that contains many more columns and several rows for each order number (one row per item in each order).

 

I created the function below so that it goes through every row looking for a URL, scrapes it and delivers the result to column "Shipment Status".

 

let GetResults=(trackingurl) =>

let
Origine = Web.Page(Web.Contents(trackingurl)),
Data0 = Origine{0}[Data],
Children = Data0{0}[Children],
Children1 = Children{1}[Children],
Children2 = Children1{1}[Children],
Children3 = Children2{0}[Children],
Children4 = Children3{0}[Children],
Children5 = Children4{2}[Children],
Children6 = Children5{0}[Children],
Children7 = Children6{0}[Children],
Children8 = Children7{0}[Children],
Children9 = Children8{0}[Children],
#"Rinominate colonne" = Table.RenameColumns(Children9,{{"Text", "Shipment Status"}})
in
#"Rinominate colonne"

in GetResults

 

What i wish to achieve, is for the function to only scrape the URLs that correspond to any "Shipment Status" different from "Delivered" in order to avoid going through every single row on every update.

 

Can anyone help with this?

Thank so much in advance

Paolo

1 ACCEPTED SOLUTION
lbendlin
Super User
Super User

Power BI has no memory.  It will always start from scratch.  If you need to keep track of prior runs then you need to do that before the data hits Power BI.

View solution in original post

4 REPLIES 4
lbendlin
Super User
Super User

Power BI has no memory.  It will always start from scratch.  If you need to keep track of prior runs then you need to do that before the data hits Power BI.

Yes, i know that it wouldn't be possible to modify PowerBI's logic, but i hoped to find a "thinking out of the box" solution. 

Maybe an extra table where "delivered" rows are moved to hence removed from the table where the function scrapes URLs?

I would think that "you need to do that before the data hits Power BI"  is pretty much the same as  "thinking out of the box" ?

I guess 🙂

Thanks for your help!

Helpful resources

Announcements
May 23 2022 epsiode 5 without aka link.jpg

The Power BI Community Show

Welcome to the Power BI Community Show! Jeroen ter Heerdt talks about the importance of Data Modeling.

charticulator_carousel_with_text (1).png

Charticulator Design Challenge

Put your data visualization and design skills to the test! This exciting challenge is happening now through May 31st!

Power BI Dev Camp Session 22 with aka link 768x460.jpg

Check it out!

Mark your calendars and join us on Thursday, May 26 at 11a PDT for a great session with Ted Pattison!

Power BI Release May 2022 768x460.png

Check it out!

Click here to read more about the May 2022 updates!