Questions about real time datasets that I couldnt find on the docs

[ruthpozuelo]

Here are the questions I have for the dataset that is created when enabling historical data storage for realtime streaming:

1. Where is the dataset stored?
2. How can I access it so I can, for example, delete rows?
3. Can I change the retention policy from 200k rows to say....1k rows?

Thanks in advance,

/Ruth

[rainer1 · Accepted solution]

Hi ruthpozuelo,

All of the Data is stored in a storage i.e. Azure Data Lake by Power BI Service which is not directly accesible.

If you create a streaming dataset it is only accessibile via api. You can not directly delete a row from the dataset, the only way will be delete all rows via the api. A solution for your purpose can be, delete the rows in Power BI Desktop in the Editor after you connect to your dataset. The default rention is 200k and is handeld Fifo see the Picture Below from Microsoft.

for more Information

https://docs.microsoft.com/en-us/power-bi/developer/api-automatic-retention-policy-for-real-time-dat...

If you Post the 200001 Row to your Dataset the first posted row will be dropped.

You can Modifiy the Retention policy with

POST https://api.powerbi.com/v1.0/myorg/datasets?defaultRetentionPolicy={None | basicFIFO}

If you set your streaming Dataset to to none retention you can store up to 5 Million rows but you some have Limits. ( See below).

To POST Rows

• 75 max columns
• 75 max tables
• 10,000 max rows per single POST rows request
• 1,000,000 rows added per hour per dataset
• 5 max pending POST rows requests per dataset
• 120 POST rows requests per minute per dataset
• If table has 250,000 or more rows, 120 POST rows requests per hour per dataset
• 200,000 max rows stored per table in FIFO dataset
• 5,000,000 max rows stored per table in ‘none retention policy’ dataset
• 4,000 characters per value for string column in POST rows operation

https://docs.microsoft.com/en-us/power-bi/developer/api-rest-api-limitations

[ruthpozuelo]

Wondeful!! May I ask some more?

1. Can I share publicly a realtime report/tile? 

2. If I didnt turn on the historic data, no data was streamed to the dataset (this is the example I was trying: https://studio.youtube.com/video/O6aEzXAFEqw/edit). Is that an issue with flow or do I need to turn on historic data all the time? 

Thanks in advance.

/Ruth

[rainer1]

Hi Ruth,

to 1.) Yes you can share your Reports as any other Reports since a few weeks there is also a solution to automatic refresh the report for example every 5 minutes so new data will show quickly without pressing the refresh button.

2. unfortunatly i cant watch the video

-------------------------------------------------------------------
Did I answer your question? Mark my post as a solution!
It was useful? Press Thumbs Up!

[Anonymous]

Hi @rainer1 

How is it possible to have an automatic refresh on a report every 5 minuts? even with Premium the refresh is every 30minuts. 

On the other hand, if you don't have the historic data activated, you cannot connect to data from the report because there is no dataset. And the most important thing for me: real time data is only alive in dashboards, not reports. 

Why would I want to refresh the report?

Thanks for all your explanations, you really back up my last's months work.

[rainer1]

Hi @Anonymous,

we talk about different things 🙂 . You talk about the dataset refresh and you are right about the 30 min refresh time.

I talk about a preview feature wich you can activate in the options pane. 

See here:

 

If you have a real time dataset you push the data in the dataset therefor you don't have to refresh the dataset.

The problem is the Report it self, if you keep it open. You will see always the sate of the data the time you open it until you do a refresh or reopen it.

For Example:

You push data every 2 minutes in your dataset.

Someone open your Report at 2 pm and keep it open. At 2.15 PM the same person look at the report but the values didn't changed because no refresh of the report is done. So the person have to manualy refresh the report.

If i can help you fürthermore please let me.

-------------------------------------------------------------------
Did I answer your question? Mark my post as a solution!
It was useful? Press Thumbs Up!

[Anonymous]

Hi @rainer1 

Thank you so much for your reply! 

I did not know about that feature! It is so awesome!!

Even though the combination between page refresh and direct query simulates real time data streaming, it also depends on Power BI license:

Shared workspaces - For regular workspaces (workspaces that are not part of a Premium capacity), automatic page refresh has a minimum interval of 30 minutes (the lowest interval allowed).

Premium workspaces - Automatic page refresh availability in Premium workspaces will depend on the workload settings your premium administrator has set up for the Power BI Premium capacity. 

https://docs.microsoft.com/en-us/power-bi/desktop-automatic-page-refresh

Still awesome feature!

My customer doesn't like the Power BI dashboards(I must use them because of real time streaming) so I have built up an APP in PowerAPPs in order to give it more design and more userfriendly look.

BUT! the main problem is with the scrolling pages APPs, are quite slow! Is there anything I can do about it?

Another problem is the white background of the tiles, I couldn't get them changed.

Thanks again!