Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
cnschulz
Helper IV
Helper IV

Gateway needed for web data ?

Hi, 

 

I have created a simple test report that gets data from a publicly available website. It simply gets the current weather from a provider and shows a graph over time. 

 

When the report is loaded to app.pwerbi.com, the dataset created cannot be refreshed as it stated I need a data gateway for this. I was of the belief that for direct connect to the web I did not need a gateway.

 

Could someone confirm this? Is there a way for me to source data from the web without a gateway?

 

Cheers

 

9 REPLIES 9
GilbertQ
Super User
Super User

Hi there

It depends on how the underlying M Code is being used.

I cannot recall off the top of my head, but I am certain there are different ways.




Did I answer your question? Mark my post as a solution!

Proud to be a Super User!







Power BI Blog

Hi there

I got this from a good friend of mine

Web.page function needs gateway. If you use web.contents you won't need it.

So just check the underlying M Code.




Did I answer your question? Mark my post as a solution!

Proud to be a Super User!







Power BI Blog

Thanks for your quick reply.

I am using the "out of the box" web data source. When I add my URL the following code is added as a source.

= Web.Page(Web.Contents("http://www.bom.gov.au/tas/observations/tasall.shtml"))

 

If I remove the "Web.Page()" and leave the Web.Contents, I am shown an icon of the website and my only option is to click to expands the data, which regenerates the Web.Page() wrapper.

 

When I attempt to set up refresh on the generated dataset, i am prompted to use a gateway. 😞

 

Any ideas?

 

Hi, 

 

The very first source line of that linked page is identical to mine 

Source = Web.Page(Web.Contents("http://..." 

 Therefore, it should require a web gateway as well. 

 

Has anyone processed web (http) data in a report on the powerbi service without a gateway? Is so please detail the steps.

Hi there

As explained above it can only be done with a Web.Contents, if there is a Web.Page it will require a gateway.




Did I answer your question? Mark my post as a solution!

Proud to be a Super User!







Power BI Blog

Hi,

 

Don't know if this is still actual, but I ran into the same issues.

 

I found a workaround, which I describe here: https://community.powerbi.com/t5/Power-Query/WORKAROUND-How-to-use-scheduled-refresh-of-a-web-page-w....

 

Hope that helps!

 

Kirvis

luisrh
Responsive Resident
Responsive Resident

You are screen scraping.  Is there a web service end point that returns JSON or CSV for this site?   You can hand code M to do the expansion.   Here is a post that can help you.      https://community.powerbi.com/t5/Desktop/Web-scraping-advice/td-p/330414

 

 

Not off the top of my head, but I am sure that there is a way to get this done.




Did I answer your question? Mark my post as a solution!

Proud to be a Super User!







Power BI Blog

Anyone? I must be missing something basic here.


@GilbertQ wrote:
Not off the top of my head, but I am sure that there is a way to get this done.

 

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors