Cannot get data from website that requires login credentials / Credentials are not valid

[Anonymous]

Hi,

After an intense research on the subject, I could not find any solution to my problem. Apologize if there is any solution which was previosly posted but I have missed. 

Following Get Data -> Web steps, I am trying to retrieve data from a website named "capitaliq.com".

As the website requires credentials to interact further, from Options and settings -> Data source settings -> Edit permissions, I set the credential type as Basic and privacy level as organization and enter my correct username and password. After giving Power BI required credential information, I attempt to get data from the website.

Despite I confirm that the username and the password are both correct, I get the error message stating that "The credentials provided are invalid. (See below image)

I have scanned the internet and found no solution to this problem. I am non-IT guy but I am wondering if the cause of the problem is rather related to the properties set by the website (capitaliq.com) I try to connect? I am supposing that this may not related to Power BI services indeed.

In other words, is there any possibility that I am having this issue as the website does not give access to Power BI, despite having the correct credentials entered. 

Any help would be greatly appreciated

Thanks in advance!

[V-lianl-msft]

Hi @Anonymous ,

I think you need to use power query to customize the web connector, and you need to bring the cookies to access to the website.

If this website is a get request to get data:

let
    url="",             //The previous section of "?" in the requset URL
    headers=[Cookie=""],             //Cookies for web pages after login
    query=[],               //Query String Parameters
    web=Text.FromBinary(Web.Contents(url,[Headers=headers,Query=query]))
in
    web

If this website is a post request to get data:

let
    url="",
    headers=[#"Content-Type"="",Cookie=""],          
    query=[],
    content="",
    web=Text.FromBinary(Web.Contents(url,[Headers=headers,Query=query,Content=Text.ToBinary(content)]))
in
    web

Best Regards,
Liang
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[Anonymous]

Hi @V-lianl-msft 

Thank you so much for your response. The way you proposed helped me to go dive into the details of power query that I have never experienced before.

However, I would like to note that I am currently at the very early stages of my learning curve and I have just utilized the built-in concept that the power tool offers. Frankly, I am having a bit of difficulties when trying to figure out how to implement your potential solution as this is my first practice customizing the query. I have few regarding questions as below: 

1.  How can I differ whether it is GET request / POST request? After some research, I had an idea that websites requiring login information working with POST request as the client gives information to the website to retrieve the data. I am not sure if this is correct statement, I would appreciate if you could verify. 

2.  I understand the syntax of M. Although I couldn't find any tip on specifying (a) headers and (b) query variables.

     I learned that cookie is a standard HTTP request field and is something sent by the server to keep me logged in but how I am going to define it?

3. If it is POST request type, does the variable [content] has to cover my login credentials? How do I have to contain that information in that variable?

Regards,

[Anonymous]

Hi all,

I could not figured out a way. If this is something simple to be directed with instructions given in the forum, I would be very grateful for your assistance. 

[amitchandak]

@Anonymous , Try to check if this can help

https://community.powerbi.com/t5/Power-Query/Data-source-accessing-a-web-URL-which-requires-Authentication/m-p/347342

https://blog.kloud.com.au/2018/07/23/use-anonymous-authentication-on-web-data-sources-with-a-api-key-in-url/

https://kohera.be/blog/power-bi/the-hidden-gems-of-the-function-web-contents/