Reply
Regular Visitor
Posts: 50
Registered: ‎03-09-2016

Web scraping a site which requires form data entry before producing a data table

Hi all,

 

I'm after some guidance on the best way of obtaining all of the available data from the following web site.

 

https://www.rec-registry.gov.au/rec-registry/app/public/lgc-register

 

The site requires a form to be completed before producing a data table. (or several tables over many pages)

 

Registry form - "select all" needed for each drop downRegistry form - "select all" needed for each drop down

 

Is there a way of parameterising a query so that ALL of the available data would be returned?

 

Thanks in advance.

Moderator
Posts: 3,641
Registered: ‎03-06-2016

Re: Web scraping a site which requires form data entry before producing a data table

Hi @pat_energetics,

 

If the REC Registry provide APIs for query this kind of data, then it's able to use Web data source to call API in Power BI desktop. You can see this sample of calling REST API: How to call REST APIs and parse JSON with Power BI. If not, I'm afraid it's not possible to get data in Power BI.

 

Best Regards,
Qiuyun Yu

Regular Visitor
Posts: 50
Registered: ‎03-09-2016

Re: Web scraping a site which requires form data entry before producing a data table

Thanks Qiuyun,

It appears the site does have an API - https://www.rec-registry.gov.au/rec-registry/app/api/public-register/certificate-actions?

but the documentation on it is limited.

I will try to make contact with the site admin to see what we can call.