Insights needed - Approved web scrappers and distance map API -

[Sreesudha_OC]

Hi All,

Can someone please share some insights about the IT approved web scrappers which we can use to scrape data from various links to pbi. Which one is more widely used.  Something to scrape the underlying data.

Has anyone in this group tried distance map API ? Requesting to kindly share some insights regarding the same

Do we need to have lisence for both the request that we asked above.

Thanks

Sudha

[v_mark]

@Sreesudha_OC  - You might want to check Overview  |  Distance Matrix API  |  Google Developers

[collinq]

HI @Sreesudha_OC ,

I am not positive what you mean because there are a couple of layers here.  When you say "IT approved" then that refers to your organization's approval process.  They may or may not allow you to reach certain data sources. 

As for the part about "scrape data to PBI" I think you are referring to getting data to report on it.  To that end, Power BI has around 140 built-in connectors available to you.  But, (and this is my own statement, not Microsoft's) "if you can see it you can report on it".  There are API connectors, there are "Web" connectors, there are odata connectors.  So, that is why I say you can see it you can report on it.

Also, I am not sure what you mean by "distance map".  In Power Bi there are a number of map visuals available to you for free and if you want to pay a reasonable fee you can use the ArcGSI map that gives all sorts of distance and time.

[Sreesudha_OC]

Thanks Collinq

My idea is little dim in this. You are correct from the perspective of approval part of it.  I can check that.

To the second point, i would like to understand more about 3rd party commenly  or widely used API and Web connectors that are available.  If you can share few names, that would be helpful

For the third,  my requirement was to identify pincodes for India with hospitals in that area. Heard in common forum that Distance map API would be a better option.  Thought of understanding the same better. If ArcGSI has capability, then this is something that can be looked at. 

[collinq]

Hi @Sreesudha_OC ,

To try and clarify, each datasource has it's own potential API or web connector - meaning that I am now aware of any sort of generic API that can just be used.  The way it normally works is that if you want data from a certain data source, you use the API that was made by that company and/or by an external company specifically to access that data source.  For example, the organization cdata builds api's and other connectors for use with certain data sources.

As for the third point, I would look at the ArcGIS map for your needs.

[Sreesudha_OC]

Hello team,

Has anyone seen any API to develop a covid dashboard(internally).

I have seen a json in Coronavirus Outbreak in India (covid19tracker.in)Coronavirus Outbreak in India (covid19tracker.in)

However extraction is not happening with this. Can someone help me with the steps.

Current steps - Get file->Json>paste the API link taken from website >try to connect..

Thanks

Sudha

[Sreesudha_OC]

Thankyou so much.

Just one more query. Can you name some 3rd party web scrappers. I can check back in my org if something similar in those lines has been approved.

Thanks

Sudha

[collinq · Accepted solution]

HI @Sreesudha_OC ,

Please mark this as a solution if this meets your needs.  Two companies I have personally dealt with and have had good experiences with are Cdata and Panoply.