WebScraping OLE DB or ODBC Error: [DataSource.Error] We are unable to retrieve the contents of ...

[raymond]

Hi Community,

I am attempting of scraping a lot of data from the web. I have a list of URLs from which I automated the scraping process with PowerBI. However I will get errors on a seemingly random basis. The error always stresses the following message: OLE DB or ODBC Error: [DataSource.Error] We are unable to retrieve the contents of 'https://www.domain.com/sample'

However, if you check the single domain that caused the error. No error will appear. But if I try to retrieve the data by using the list, the error is most likely to appear.

The error doesnt appears twice on the same URL. It is merely random.

Is there a way to fix that? Is there a way to skip errors like that with Power Query? Maybe it has something to do with loading the website; might there be a code that ensures the website is loaded?

appreciate any help 🙂

[tmschweiz]

Very frustrating error indeed, i came across the same.
Found a solution at last in this thread, which addressed ODBC errors more generally: https://www.cloudfronts.com/blog/power-bi/how-to-solve-ole-db-or-odbc-errors-in-powerbi/

Solution is:

Go to Data Source Settings > Global Permissions,

then select the website you were trying to access

> Clear Permissions

This solved my issue. I have to do it again from time to time, when i change the query in Power Query.

[iNick]

hi there, Did you ever manage to solve this issue? I've run into the same problem and can't seem to solve it. 

OLE DB or ODBC error: [DataSource.Error] We were unable to retrieve the contents of the web page. 

Like you it loads fine in the query editor but when actually loading the data into power query I get this error. 

Weirdly the error pops up randomly so it's not necessarily an issue with the URL. I tried to invoke the function with a delay but this didn't help much either. 

[V-pazhen-msft]

@raymond 

This error usually is the URL problem, sometime you cannot get the content of the url if the url is refreshed with new data layout, format, etc.

Regards
Paul

[raymond]

There @V-pazhen-msft

Thank you for responding.

However this is not the solution I think. The page stays the same and if I do the scraping all over the error will not occur on that website. If I scrape ONLY the website where an error occured, it will work.

It does however not work when I try to scrape in large batches of a 100, 1.000, 10.000 etc.

So I was thinking, is it maybe possible to check whether the page is fully loaded?