Web scraping a site which requires form data entry before producing a data table

[pat_energetics]

Hi all,

I'm after some guidance on the best way of obtaining all of the available data from the following web site.

https://www.rec-registry.gov.au/rec-registry/app/public/lgc-register

The site requires a form to be completed before producing a data table. (or several tables over many pages)

Registry form - "select all" needed for each drop down

Is there a way of parameterising a query so that ALL of the available data would be returned?

Thanks in advance.

[v-qiuyu-msft]

Hi @pat_energetics,

If the REC Registry provide APIs for query this kind of data, then it's able to use Web data source to call API in Power BI desktop. You can see this sample of calling REST API: How to call REST APIs and parse JSON with Power BI. If not, I'm afraid it's not possible to get data in Power BI.

Best Regards,
Qiuyun Yu

[pat_energetics]

Thanks Qiuyun,

It appears the site does have an API - https://www.rec-registry.gov.au/rec-registry/app/api/public-register/certificate-actions?

but the documentation on it is limited.

I will try to make contact with the site admin to see what we can call.