Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
queryfan
New Member

How to load Data from a webpage having multiple pages

Hello,

 

I am trying to import data using the below query , But I am able to import only the first 30rows in table0..How do I loop to the next table and import the remaing?

 

Please suggest necessary changes to the 'M' code

 

URL: http://www.moneycontrol.com/markets/earnings/latest-results/latest/yoy/standalone/

 

let
    Source = Web.Page(Web.Contents("http://www.moneycontrol.com/markets/earnings/latest-results/latest/yoy/standalone/")),
    Data0 = Source{0}[Data],
    #"Changed Type" = Table.TransformColumnTypes(Data0,{{"Company Name", type text}, {"Sales (Rs Crore) Dec 2016", type number}, {"Sales (Rs Crore) Dec 2015", type number}, {"Sales (Rs Crore) % Chg", type text}, {"Gross Profit (Rs Crore) Dec 2016", type number}, {"Gross Profit (Rs Crore) Dec 2015", type number}, {"Gross Profit (Rs Crore) % Chg", type text}, {"Net Profit (Rs Crore) Dec 2016", type number}, {"Net Profit (Rs Crore) Dec 2015", type number}, {"Net Profit (Rs Crore) % Chg", type text}})
in
    #"Changed Type"

Thank you.

9 REPLIES 9
Ahammedjaleel
Frequent Visitor

Please check on below article  for loading Data from a webpage having multiple pages

https://www.myonlinetraininghub.com/scrape-data-multiple-web-pages-power-query

Anonymous
Not applicable

Hi! I have the same situation as you.  Were you able to find a way to get all the pages? Parameterizing the query won't work for our scenario because the URL doesn't have the page number.  In fact, it doesn't change when we switch page by clicking the other page numbers at the bottom.

 

 

Regards,

 

Ferdinand

Anonymous
Not applicable

Hello @Anonymous,

 

I am running into the same dilemma as you are.

 

Have you had any luck finding help with this issue?

 

Please let me know. Thanks.

 

-K

Anonymous
Not applicable

Hello @Anonymous,

 

Sorry I wasn't able to find a solution.  I ended up just settling for getting just the data on the default page (first page).  Just to clarify, the issue is not being able to get additional pages from a URL if the webpage uses javascript for pagination because the URL remains the same even if you navigate the table to next pages.

 

 

Regards,

 

Ferdinand

Anonymous
Not applicable

Gotcha. Yes, that was the issue I had in mind as well.

 

Thanks for getting back.

 

-K

Phil_Seamark
Employee
Employee

This is also worth reading.


To learn more about DAX visit : aka.ms/practicalDAX

Proud to be a Datanaut!

Thank you

 

I have no knowledge about the parameters in the URL.

 

Could you please let me know how to identify the variable so that I could loop through all the sheets and get the data using power query.

Hi @queryfan,

 

You can't just use the site URL to get table data in desktop simply. You need to view corresponding API and find the actual URL to get all table data. So the first thing you need to do is contacting Money Control to find the rest API to get data. Then you can follow this simple to do it in desktop: How to call REST APIs and parse JSON with Power BI

 

Best Regards,
Qiuyun Yu

Community Support Team _ Qiuyun Yu
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Phil_Seamark
Employee
Employee

This looks like it might be useful

 

http://blogs.adatis.co.uk/callumgreen/post/Loop-through-Multiple-Web-Pages-using-Power-Query


To learn more about DAX visit : aka.ms/practicalDAX

Proud to be a Datanaut!

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.