cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Highlighted
Regular Visitor

Getting data from API with JSON body

I'm trying to get data out of my ElasticSearch and into PowerBI.
I can do it with Postman Fine but am having issues with PowerBI.

 

If i remove the body I do not get an error.

 

My Query for PowerQuery Editor is

 

let
    url = "http://MYSITE/search",
    body = "
            {
            ""query"": {
                ""bool"": {
                    ""must"": [
                        {
                            ""range"": {
                                ""createdDate"": {
                                    ""gt"": ""2020-01-01T00:00:00"",
                                    ""lt"": ""2099-12-31T23:59:59""
                                }
                            }
                        },
                        {
                            ""terms"": {
                                ""interestFlags.systemName"": [""SalesApplicant"",""LettingsApplicant"",""Vendor"",""Company"",""Landlord""]
                            }
                        }
                    ],
                    ""must_not"": [],
                    ""should"": []
                }
            },
            ""from"": 0,
            ""size"": 0,
            ""sort"": [],
            ""aggs"": {
                ""className"": {
                    ""terms"": {
                        ""field"": ""interestFlags.systemName"",
                        ""size"": 200
                    }
                }
            }
        }
    ",
    Source = Json.Document(Web.Contents(url),[
        Headers = [#"Content-Type"="application/json"],
        Content=Text.ToBinary(body)
    ])
in
    Source

 

 

I get the following error when I pass in the JSON body

Expression.Error: We cannot convert a value of type Record to type Number.
Details:
Value=[Record]
Type=[Type]

 

Screenshot 2020-04-16 17.48.37.png

Hope someone can point me in the right direction.

Thanks,

Keri.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

The problem is the use of Json.FromValue. body is a text value, so it is giving you a json encoded string rather than the structured object the API is expecting.

 

If you want to keep body as text, then you'd use Text.ToBinary to convert it to POST content. 

 

let
    url = "xxxxxxx",
    body = "{""query"": {""bool"": {""must"": [{""range"": {""createdDate"": {""gt"": ""2020-01-01T00:00:00"", ""lt"": ""2099-12-31T23:59:59""}}},{""terms"": {""interestFlags.systemName"": [""SalesApplicant"",""LettingsApplicant"",""Vendor"",""Company"",""Landlord""]}}],""must_not"": [],""should"": []}},""from"": 0,""size"": 0,""sort"": [],""aggs"": {""className"": {""terms"": {""field"": ""interestFlags.systemName"",""size"": 200}}}}",
    header = [  #"Authorization"="xxxxxx",
                #"Content-Type"= "application/json"],  
    response = Web.Contents(url,[Content=Text.ToBinary(body),Headers=header]),
    Source = Json.Document(response,1252
in
    Source
 
Alternatively, you could build up body as a record, and then use Json.FromValue. The main thing to keep in mind here is that the {} and [] are reversed between M and JavaScript. This would look something like:
 
let
    url = "xxxxxx",
    body = [
        query = [
                bool = [
                    must = {[ 
                        range = [ 
                            createdDate = [ gt = "2020-01-01T00:00:00", lt = "2099-12-31T23:59:59" ]
                        ]
                    ]},
                    must_not = {},
                    should = {}
                ]
            ],
        from = 0,
        size = 0,
        sort = {},
        aggs = [
            className = [
                terms = [
                    field = "interestFlags.systemName",
                    size = 200
                ]
            ]
        ]
    ],
    header = [  #"Authorization"="xxxxxxxx",
                #"Content-Type"= "application/json"],  
    response = Web.Contents(url,[Content=Json.FromValue(body),Headers=header]),
    Source = Json.Document(response,1252
in
    Source
 

View solution in original post

6 REPLIES 6
Highlighted
Community Support
Community Support

Hi @keribeal , 

I am not sure whether this is caused by the code, you could try below to see whether it work or not

Source = Json.Document(Web.Contents(url,[
        Headers = [#"Content-Type"="application/json"],
        Content=Text.ToBinary(body)
    ]))

You could refer to this post   for details. By the way, you also could @ super users in this forum for more suggestions.

 

Best Regards,
Zoe Zhi

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Highlighted
Regular Visitor

@dax Thanks for the reply, you are right I had a ) after url which shouldn't have been there.

 

That gives a new error for me

 

DataSource.Error: Web.Contents with the Content option is only supported when connecting anonymously.
Details:
DataSourceKind=Web
DataSourcePath=http://XXXXX

 

So that's a new error to get searching for as the BASIC AUTH credentials were entered at the start when prompted.

 

Sorry i'm new to these forums, with your comment about @ super users, do you mean i just look for the names of super users and just add them in here?

 

Thanks,

Keri.

Highlighted

Having read what feels like half the internet i've now learnt that you can pass the BASIC AUTH U&P as an Authorization header, thus allowing the save it in PowerBI as anonymous and therefore allowing to post the body.

 

I've tested this in postman and it works fine, however in powerBI i get the following error

 

DataSource.Error: Web.Contents failed to get contents from 'http://mysite/_search' (400): Bad Request
Details:
DataSourceKind=Web
DataSourcePath=http://mysite/_search
Url=http://mysite/_search

 

let
    url = "http://mysite/_search",
    body = "
            {
            ""query"": {
                ""bool"": {
                    ""must"": [
                        {
                            ""range"": {
                                ""createdDate"": {
                                    ""gt"": ""2020-01-01T00:00:00"",
                                    ""lt"": ""2099-12-31T23:59:59""
                                }
                            }
                        },
                        {
                            ""terms"": {
                                ""interestFlags.systemName"": [""SalesApplicant"",""LettingsApplicant"",""Vendor"",""Company"",""Landlord""]
                            }
                        }
                    ],
                    ""must_not"": [],
                    ""should"": []
                }
            },
            ""from"": 0,
            ""size"": 0,
            ""sort"": [],
            ""aggs"": {
                ""className"": {
                    ""terms"": {
                        ""field"": ""interestFlags.systemName"",
                        ""size"": 200
                    }
                }
            }
        }
    ",
    //body="",
    header = [  #"Authorization"="Basic BASE64-USER:PWD",
                #"Content-Type"= "application/json"],


    response = Web.Contents(url,[Content=Json.FromValue(body),Headers=header]),
    Source = Json.Document(response,1252) 
in
    Source

 

 Is there anyone that can help shine some light on this issue for me, help point me in the right direction?

 

Thanks,

Keri

Highlighted

Good thing is I now know that the auth is definitely working, if I make the call without a body i get data back on one of my endpoints, i get 5m records instead of the 4 totals i was hoping for, but it's a start.

 

Looking at the fiddler data for both Postman and PowerBI i think i can see what is probably the issue.

 

For the Postman call, i can see the JSON all neatly formatted in the request

Screenshot_2020-04-21_12_55_31-1.png

The original PowerBI with neatly formatted JSON came out like this

Screenshot_2020-04-21_12_55_43-2.png

So i put it all on a single line and had thisScreenshot_2020-04-21_12_55_53-3.png

"{\"query\": {\"bool\": {\"must\": [{\"range\": {\"createdDate\": {\"gt\": \"2020-01-01T00:00:00\", \"lt\": \"2099-12-31T23:59:59\"}}},{\"terms\": {\"interestFlags.systemName\": [\"SalesApplicant\",\"LettingsApplicant\",\"Vendor\",\"Company\",\"Landlord\"]}}],\"must_not\": [],\"should\": []}},\"from\": 0,\"size\": 0,\"sort\": [],\"aggs\": {\"className\": {\"terms\": {\"field\": \"interestFlags.systemName\",\"size\": 200}}}}"

I'm guessing the quotes either end aren't helping nor is the escaped characters throughout, anyone got any tips how to get this working?

 

Thanks,

Keri

Highlighted

The problem is the use of Json.FromValue. body is a text value, so it is giving you a json encoded string rather than the structured object the API is expecting.

 

If you want to keep body as text, then you'd use Text.ToBinary to convert it to POST content. 

 

let
    url = "xxxxxxx",
    body = "{""query"": {""bool"": {""must"": [{""range"": {""createdDate"": {""gt"": ""2020-01-01T00:00:00"", ""lt"": ""2099-12-31T23:59:59""}}},{""terms"": {""interestFlags.systemName"": [""SalesApplicant"",""LettingsApplicant"",""Vendor"",""Company"",""Landlord""]}}],""must_not"": [],""should"": []}},""from"": 0,""size"": 0,""sort"": [],""aggs"": {""className"": {""terms"": {""field"": ""interestFlags.systemName"",""size"": 200}}}}",
    header = [  #"Authorization"="xxxxxx",
                #"Content-Type"= "application/json"],  
    response = Web.Contents(url,[Content=Text.ToBinary(body),Headers=header]),
    Source = Json.Document(response,1252
in
    Source
 
Alternatively, you could build up body as a record, and then use Json.FromValue. The main thing to keep in mind here is that the {} and [] are reversed between M and JavaScript. This would look something like:
 
let
    url = "xxxxxx",
    body = [
        query = [
                bool = [
                    must = {[ 
                        range = [ 
                            createdDate = [ gt = "2020-01-01T00:00:00", lt = "2099-12-31T23:59:59" ]
                        ]
                    ]},
                    must_not = {},
                    should = {}
                ]
            ],
        from = 0,
        size = 0,
        sort = {},
        aggs = [
            className = [
                terms = [
                    field = "interestFlags.systemName",
                    size = 200
                ]
            ]
        ]
    ],
    header = [  #"Authorization"="xxxxxxxx",
                #"Content-Type"= "application/json"],  
    response = Web.Contents(url,[Content=Json.FromValue(body),Headers=header]),
    Source = Json.Document(response,1252
in
    Source
 

View solution in original post

Highlighted

Fantastic, many thanks to PowerBI support for this help.

Helpful resources

Announcements
Community Conference

Power Platform Community Conference

Check out the on demand sessions that are available now!

Community Conference

Microsoft Power Platform Communities

Check out the Winners!

secondImage

Create an end-to-end data and analytics solution

Learn how Power BI works with the latest Azure data and analytics innovations at the digital event with Microsoft CEO Satya Nadella.

Top Solution Authors
Top Kudoed Authors