cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Highlighted
Regular Visitor

Extract Data from Web for Multiple Pages with Non-Changing URLs

Hello! As the title says, I would just like to ask if it is possible to use the extract data from web feature of PBI to get data from a table in a webpage that has multiple pages, but non-changing URLs when you click on "Next Page." I see many examples explaining how it is possible to extract data for changing URLs e.g. ?page=1 for page 1 and ?page=2 for page 2, but I cannot seem to find a solution for tables that have non-changing URLs 😞

 

I cannot exactly disclose the website to extract data from but just to illustrate:

 

The URL I need to use is www.website.com/current-files

This URL has a table showing the result, with multiple pages. But when I click on "Next Page," the URL does not change, that's why I cannot use the 'Invoke Function' solution that is outlined in many blog posts.

 

I'd appreciate whatever ideas you may have. Thank you!

2 REPLIES 2
Highlighted
Super User IV
Super User IV

Re: Extract Data from Web for Multiple Pages with Non-Changing URLs

Perhaps @ImkeF knows some magic for this. I suppose it might be possible if the entire table is loaded and it is just javascript magic that displays the right rows. If it is some kind of AJAX javascript call back, no idea.


---------------------------------------

Putting square pegs in round holes since 1972.

I have a NEW book! 
DAX Cookbook from Packt
Over 120 DAX Recipes!




Did I answer your question? Mark my post as a solution!

Proud to be a Super User!




Super User III
Super User III

Re: Extract Data from Web for Multiple Pages with Non-Changing URLs

Sorry guys, no help with M on this one.

But you can check out UI Flows.

They use the SELENIUM enginge in the background for webscraping. (which you can use directly with Python, if you're proficient with it)

 

Imke Feldmann (The BIccountant)

If you liked my solution, please give it a thumbs up. And if I did answer your question, please mark this post as a solution. Thanks!

How to integrate M-code into your solution -- How to get your questions answered quickly -- How to provide sample data -- Check out more PBI- learning resources here -- Performance Tipps for M-queries

Helpful resources

Announcements
Super Users of the Quarter - Q2 2020

Super Users of the Quarter - Q2 2020

Who are our Super User Superstars? Who made it to the top of the leaderboards? Get the answers!

June 2020 Community Highlights

June 2020 Community Highlights

Featured community members, changes to the Community, and more! Read up on recent Power BI community news.

Community Summit North America

Community Summit North America

Innovate, Collaborate, Grow. The top training and networking event across the globe for Microsoft Business Applications

Power Platform 2020 release wave 2 plan

Power Platform 2020 release wave 2 plan

Features releasing from October 2020 through March 2021