No content with Web.Contents vs. Web.BrowserContents

[Anonymous]

Dear all,

I have an - at least for me - strange behaviour here. A simplified example:

I want to scrape some Reviews & Ratings values from a Web-Shop

It works perfectly fine in PBI Desktop and I receive the intended values in my table.

But I want to run it as a PBI service - so not possible with Web.BrowserContents as far as I know - thus I switch to Web.Contents (with Web.Contents I run some other projects successfully in PBI Service with RelativePath, etc).

But this time & in this specific Web-Shop I receive with the same code "w/o the Browser" only an empty table!?

Looking one step before I see in this case that I can find "manually" the respective product information in the HTML results of Web.BrowserContents while I can't see the product information in the HTML results of Web.Contents manually either, but rather a very short HTML result (in other Web-Shops I see the same HTML results).

Have you seen different responds from Web.BrowserContents vs. Web.Contents?
Do Servers accept We.BrowserContents rather as a human browser than Web.Contents?

Thanks a lot for any inspiration how to receive the same results with Web.Contents!

BR
Florigi

[wobi]

Well for this Gateway you need a "special" MS AD known user -special in that way that none of our existing pbi users is accepted.. so where to get this mysterious user?
Message:  AADSTS90020: The SAML 1.1 Assertion is missing ImmutableID of the user

- Wobi

[wobi]

Well for this Gateway you need a "special" MS AD known user -special in that way that none of our existing pbi users is accepted.. so where to get this mysterouse user?
Message:  AADSTS90020: The SAML 1.1 Assertion is missing ImmutableID of the user

- Wobi

[v-yingjl]

Hi @Anonymous ,

Sometimes, when you migrate your queries from Power Query in desktop tools to the dataflow, you get an error saying that a module is disabled in this context. One example of this situation is when your query uses functions such as Web.Page or Web.BrowserContents.

Reason:

Disabled modules are related to functions that require an on-premises data gateway connection to work. Even if the function is getting data from a webpage, because of some security compliance requirements, it needs to go through a gateway connection.

Resolution:

In power bi service, you need to configure gateway for your data source first. If you do not have a gateway, you can download from here.

Then refresh power query editor in the dataflow, you will get the same result which has been shown in power bi desktop.

Reference:

Expression.Error: The module named 'xyz' has been disabled in this context 

Best Regards,
Community Support Team _ Yingjie Li
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[Anonymous]

Hi @v-yingjl ,

thanks a lot for your reply.

But maybe I have not been very clear in my post: I am still in both scenarios within Power BI Desktop.

Only to avoid later-on issues when publishing to service I want now to switch from Web.BrowserContents to Web.Contents - why (small excurseion:

When using Web.BrowserContent - I receive in the PBI Service the following error message:

" This dataset includes a dynamic data source." with a link to https://aka.ms/dynamic-data-sources

There is said:
In most cases, Power BI datasets that use dynamic data sources cannot be refreshed in the Power BI service. There are a few exceptions in which dynamic data sources can be refreshed in the Power BI service, such as when using the RelativePath and Query options with the Web.Contents M function.

As I understand it I have to use Web.Contents with "RelativePath" instead of Web.BrowserContents. And this way it works also for other projects scraping other web-shops perfectly fine for me later-on in PBI Service (without On Premis Gateway).

But for the specific URL mentioned above I receive different results from Web.Contents compared to Web.BrowserContents.

So my main question is: How to workaround to receive the same HTML result from Web.Contents within PBI Desktop (in order to be able to publish it later-on to PBI service)?

[Anonymous]

Hi, 

It seems that the accepted solution doesn't really help to find a workaround. 

I also need to download HTM table (so I use the Web.BrowserContents function) but I can not refresh it on Power BI Service. 

Did you find a workaround to be able to use the web.Contents function with an HTML table ? 

[KNP]

@Anonymous - I'm also trying to figure out this issue. Usually replacing Web.BrowserContents with Web.Contents would work but it seems in some cases (maybe website design/formatting) it just results in no values. 

I also suspect that the "answer" here was not marked as the solution by the OP. 

@Anonymous- did you find a workaround or cause for this? I'm guessing you didn't mark the accepted answer as the solution?

[wb-power]

Changing "Web.BrowserContents" to "Web.Contents" in the source was the solution for me. No need for a local gateway. Thank you very much. 

[Anonymous]

Dear @KNP & @Anonymous,

if I remember correctly the issue was that the respective HTML-Product-Detail-Page (PDP) was built "on the fly" by a JAVA function getting the data from a data-base... while Web.BrowserContent obviously got the already "produced" PDP, Web.Content somehow failed.
Thus the workaround was not to scrape the PDP, but instead the URL called by the JAVA function to get the JSON data from the master-data server and afterwards converting this JSON into a table. An even significantly faster and more reliable way 😊
However, if you find some news on this topic above I would be glad to see the solution here - thanks a lot!
Best
FloriGi

[v-yingjl]

Hi @Anonymous ,

Based on my test, if you want to get the HTML table from web, you need to use Web.BroswerContents() which returns html, Web.Contents() will return a binary result.

If you nest Html.table() outside Web.Contents(), you will always get an empty table whatever your web address.

When you use the web connector to extract data, you would find that it also use Web.BroswerContents() instead of Web.Contents().

In addition, about Dynamic data source using Web.contents, you can refer: Dynamic Web.Contents() and Power BI Refresh Errors 

Best Regards,
Community Support Team _ Yingjie Li
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[vincent_nox]

Then, what if I have a pair of Web.BroswerContents() sourced tables? 

How can I have them properly refreshed, within the Service (without a gateway)? 

Sorry for the question, I'm new here (and I'm not sure to understand the whole flow) 🙂