historical data analysis in streaming API

[amitabhk1971]

Hi,

My power bi report is a snapshot of a moment. The dataset (queried from Azure SQL database) does not keep historical data. In that case is it possible to apply historical data analysis in streaming api in direct query mode? Will I be able to keep a datetime field in the api which is not there in the dataset and then use that to get historical data? Does Power BI Service keep historical data when the dataset itself is a snapshot of a moment

Thanks,

Amitabh

[Seth_C_Bauer]

@amitabhk1971 The historical data toggle on a streaming dataset only lives for a certain amount of time, so depending on how far back you need the data, this probably isn't what you want. Is there some reason why you don't just snapshot the data in your database?

[amitabhk1971]

Will it keep the data for say last 1 week?

[v-huizhn-msft]

Hi @amitabhk1971,

With a streaming dataset, data is also pushed into the Power BI service, with an important difference: Power BI only stores the data into a temporary cache, which quickly expires. And Data is temporarily stored for an hour to render visuals. Please see the capabilities and limitations of streaming dataset here.



Best Regards,
Angelia

[amitabhk1971]

Hi Angelia,

The article that you referred to says that if you use Push to make the report real time then the historical data is stored permanently.

 

And it also says that you can push data by applying three methods:

There are three primary ways you can push data into a dataset:

• Using the Power BI REST APIs
• Using the Streaming Dataset UI
• Using Azure Stream Analytics

When Historic data analysis is disabled (it is disabled by default), you create a streaming dataset as described earlier in this article. When Historic data analysis is enabled, the dataset created becomes both a streaming dataset and a push dataset. This is equivalent to using the Power BI REST APIs to create a dataset with its defaultMode set to pushStreaming, as described earlier in this article.

Doesn't this mean that if I use the Streaming API and turn the Historical data analysis on then data will be stored permanently?

[v-huizhn-msft]

Hi @amitabhk1971,

Yes, when you create a dataset with its defaultMode, the prerequiite is that the defaultMode value is set to pushStreaming, the dataset is both a push and streaming dataset. The data will be updated and stored in the push dataset. However, any streaming tiles will temporarily fail.

Best Regards,
Angelia

[amitabhk1971]

Hi Angelia,

Can you please let me know how to set the default mode to PushStreaming?

Thanks,

Amitabh

[v-huizhn-msft]

Hi @amitabhk1971,

I have consulted with others and post the update if there is any response. Thanks for undestanding.

Best Regards,
Angelia