cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Di_P Frequent Visitor
Frequent Visitor

Dynamic web content scraping where URL doesn't change

Hello,

 

I can find lots of cases where you can get the data from web pages dynamically as the parameters change in the URL, however I was interested in such a scenario where the web content is changed say by picking an item from the drop down list, and this change doesn't get reflected in the URL. The example is below. I was wondering and hoping that you could help. Thanks in advance - Di

 

E.g.  the UR is https://azure.microsoft.com/en-us/pricing/details/virtual-machines/linux/

 

In the web page when I iterate through the OS options the URL gets updated accordingly and therefore it's easy to get the pricing tables, but when I go through the Region drop down the URL doesn't get changed, but the pricing in the tables does. So my question is how can you loop through the regions' list and get the updated pricing tables?

3 REPLIES 3
Community Support Team
Community Support Team

Re: Dynamic web content scraping where URL doesn't change

Hi @Di_P ,

By my tests and research, I'm afraid that we cannot get the data from web pages when the change doesn't get reflected in the URL in Power BI currently. We only could get data based on the URL.

Actually, when you get data from the web url, you get all of the data. I think you could create the Region slicer to filter the price when you create the report in power bi.

Best  Regards,

Cherry

 

Community Support Team _ Cherry Gao
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Di_P Frequent Visitor
Frequent Visitor

Re: Dynamic web content scraping where URL doesn't change

Thanks @v-piga-msft ,

 

I pulled the regions into a slicer but by switching between the regions the pricing in the query table doesn't get updated dynamically. I guess because the region list is actually not a table in the webpage? Could you give more hints on how I can get the pricing updated after picking different regions?

Community Support Team
Community Support Team

Re: Dynamic web content scraping where URL doesn't change

Hi @Di_P ,

Yes, you're right. The region list is actually not a table in the webpage, so we cannot extract the data via web connector in power bi currenly.

Sorry, I cannot help further on this case. Hope others have better ideas.

Best  Regards,

Cherry

 

Community Support Team _ Cherry Gao
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Helpful resources

Announcements
Ask Amir Anything

Exclusive LIVE Community Event No. 2 – Ask Amir Anything

Next in our Triple A series: Ask Amir Netz questions about the latest updates, features and future.

October 2019 Community Highlights

October 2019 Community Highlights

October was a busy month in the community. Read the recap article to learn about some of the events and content.

New Solution Badges

New Solution Badges

Two waves of brand new solution badges are coming! Read the article for more information on our new community badges.

Analytics in Azure virtual event

Analytics in Azure virtual event

Experience a limitless analytics service built to ingest, prep, manage, and serve data for immediate use in Power BI.

Top Kudoed Authors
Users Online
Currently online: 50 members 1,146 guests
Please welcome our newest community members: