How to disable REST API call caching

[Anonymous]

HI All,

we are getting data from a REST API with PowerBuilder Web.Contents. Due to the amount of data we need to query them in more chunks. This means, that we have to send the exactly the same call each time, till we do not have all the data. The issue is that we get back only the first page of the data every time we send a call. The same call from Postman works as expected, so we do get all the (different data), but when we send the same request from PowerBI we only get the first page. We thought this is a caching issue, and tried to set the Headers Cache-Control and Pragma as well without any effect. We also observed the generated request made by Postman and PowerBI and regarding the caching they are the same. So right now we are out of ideas...

In Postman I am calling REST API with POST method using chunked, ChunkQId , and ChunkSize, and working with the following request header settings : 

Content-Type: application/json
User-Agent: PostmanRuntime/7.13.0
Accept: */*
Postman-Token: 921bb20e-9c69-41c7-88bb-7f28ddcc14bb
Host: xxxx.xxxxxx.com
cookie: ASP.NET_SessionId=l2hdbs5oansx5fidvfovryq5
accept-encoding: gzip, deflate
content-length: 370
Connection: close

the response header settings :

HTTP/1.1 200 OK
Cache-Control: private
Content-Type: application/json; charset=utf-8
Content-Encoding: gzip
Vary: Accept-Encoding
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Powered-By: ASP.NET
Date: Mon, 20 May 2019 07:41:32 GMT
Connection: close
Content-Length: 263

I got back in case of every calling the next 100 records. It is working well!

In Fiddler I was analysing the result if I called the REST API via PowerBI :

the request header settings:

Content-Type: application/json
Cache-Control: no-cache
Accept: */*
User-Agent: Microsoft.Data.Mashup (https://go.microsoft.com/fwlink/?LinkID=304225)
Accept-Encoding: gzip, deflate
Host: xxxx.xxxxxxx.com
Content-Length: 130
Expect: 100-continue
Connection: Keep-Alive

the response header settings:

HTTP/1.1 200 OK
Cache-Control: private
Content-Type: application/json; charset=utf-8
Content-Encoding: gzip
Vary: Accept-Encoding
Strict-Transport-Security: max-age=31536000; includeSubDomains
Set-Cookie: ASP.NET_SessionId=vvysfq1vu54ljqcxpvlujuom; path=/; secure; HttpOnly
X-Powered-By: ASP.NET
Date: Mon, 20 May 2019 07:47:43 GMT
Content-Length: 23973

If I called the REST API more times via PowerBI I got back just the first 100 record everytime. I thought it could be a caching issue but it did not help..

Thanks for any idea!!!

[v-yuta-msft]

@Anonymous ,

So your requirement is to achieve the real time data, right? I would suggest you to refer to doc below:

https://docs.microsoft.com/en-us/power-bi/service-real-time-streaming

Community Support Team _ Jimmy Tao

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[Anonymous]

Hi @v-yuta-msft ,

thanks 4 the answer. Perhaps my description was not presize enough. My goal is not to get realtime data. The situation is that i have to get a lot of data from that webservice. If i try to get it in one request i got timeout issue. There is an option for this (Aurea) Webservice to get only chunks of a data. Ex.: there are 500 data, i can get them in chunks with size 100 with 5 request. The issue is that when i execute the same webservice call from PowerBi (web.content) i get the first 100 record 5 times, because of caching (i think). I tried to disable the caching with Cache-control and/or Pragma but had no effect...

[Anonymous · Accepted solution]

Hi All,

meanwhile i've found the solution. I had to introduce a custom header value which is different for every request. For example a counter in the header like in this article:

https://blog.crossjoin.co.uk/2017/01/06/web-contents-caching-and-the-excludedfromcachekey-option-in-...

the "MyHeader" value.

Cheers,

Gábor