Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.

Reply
SLR71
Frequent Visitor

Get data from multiple pages on a website - no page # in URL

Hi,

I am trying to extract all the pages of data from this URL https://www.cahss.ca/surveillanceDiseaseAlert/equine/ . I am able to bring in the first 10 in a nice table with web connector and extract HTML but I need all the pages. The URL does not change as you move through the pages on the site.

 

I have looked at a number of the suggestions in the forum

https://www.mattmasson.com/2014/11/iterating-over-multiple-pages-of-web-data-using-power-query/

https://exceleratorbi.com.au/using-power-query-extracting-non-tabular-data-web-pages/

https://www.sumproduct.com/thought/referencing-multi-page-data-from-a-web-site-with-a-single-url

 

but all require the URL to have a page number reference which mine does not have.

 

Any help would be appreciated.

Thanks

1 ACCEPTED SOLUTION
SLR71
Frequent Visitor

Hello

Using just the first line of code from https://www.mattmasson.com/2014/11/iterating-over-multiple-pages-of-web-data-using-power-query/ as an example, I tried: 

 

= Web.Page(Web.Contents("https://www.cahss.ca/surveillanceDiseaseAlert/equine/"))

This resulted in a table that I could expand with all the pages required.

 

As soon as I ask for help . . . the brain kicks in.

View solution in original post

3 REPLIES 3
SLR71
Frequent Visitor

Hello

Using just the first line of code from https://www.mattmasson.com/2014/11/iterating-over-multiple-pages-of-web-data-using-power-query/ as an example, I tried: 

 

= Web.Page(Web.Contents("https://www.cahss.ca/surveillanceDiseaseAlert/equine/"))

This resulted in a table that I could expand with all the pages required.

 

As soon as I ask for help . . . the brain kicks in.

Hi @SLR71 , the link to Matt Masson's page leads to a Wordpress Error. By chance, do you have this code you're referring to?  I am trying to scrape rows from the site: https://www.coronavirus.vic.gov.au/exposure-sites but it ceases at 30 records only.

Thanks heaps in advance!

If I have posted a response that resolves your question, please accept it as a solution to formally close the post.

Also, if you are as passionate about Power BI, DAX and data as I am, please feel free to reach out if you have any questions, queries, or if you simply want to connect and talk to another data geek!

Want to connect?www.linkedin.com/in/theoconias

This is great! So how do you get the data into excel?

Helpful resources

Announcements
Microsoft Fabric Learn Together

Microsoft Fabric Learn Together

Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City

PBI_APRIL_CAROUSEL1

Power BI Monthly Update - April 2024

Check out the April 2024 Power BI update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors