cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
flennon
Frequent Visitor

What happens when you reach the size limit of a Push Dataset with Historical Analysis Enabled

I thought up until recently that push datasets followed the 200k first in first out (basciFIFO) retention policy. I realise now that this is just for streaming datasets as the 200k rows is stored in cache.

 

I am pushing into the dataset using the RESTful API but I have historical analysis enabled so that I could avail of all the different types of charts for my dashboards. 

 

I see here that datasets are limited to 1gb. What happens when that limit is reached. Is there any FIFO system here? I know you cant selectively delete from a dataset, its either all or nothing. Would that mean that when I hit the 1gb limit, I'd have to delete out all the data and start again?

1 ACCEPTED SOLUTION

Accepted Solutions
Microsoft
Microsoft

Re: What happens when you reach the size limit of a Push Dataset with Historical Analysis Enabled

Hi @flennon,

 

Yes, I'm quite sure. I have tested it. Please refer to the snapshot below.

1. If it's only a streaming dataset, I can't create a report.

2. If it's only a push data set, it won't have an API info.

3. I pushed 300K rows into it from ID 1 to ID 300000. (I intended to do so. It's obvious my code doesn't that precise. But the 300K data is pushed successfully)

There are only 226K rows exist. And the minimum ID is 80001.

What happens when you reach the size limit of a Push Dataset with Historical Analysis Enabled2.png

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Best Regards,

Dale

Community Support Team _ Dale
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

6 REPLIES 6
az2451 Resolver I
Resolver I

Re: What happens when you reach the size limit of a Push Dataset with Historical Analysis Enabled

The answer to your question is right below: https://docs.microsoft.com/en-us/power-bi/service-admin-manage-your-data-storage-in-power-bi#what-ha...

 

You simple have to reduce the size to create new dashboards etc.

 

There is not other way around, thats the rule.

 

You can try it out by your self..

 

 

Greetings

flennon
Frequent Visitor

Re: What happens when you reach the size limit of a Push Dataset with Historical Analysis Enabled

My apologies, I did not communicate my point across well enough.

 

I understand that I have to reduce the size of the dataset. However, when attempting to find a way to reduce the number of records in the dataset, I found that there is no method to delete specific rows (i.e. You cant delete rows older than a certain date), you can only delete all rows.

 

Therefore, I'm asking is there something I'm missing, or do I really have to delete all the rows in the dataset and start again when I hit the maximum capacity? Rather than being able to delete 50mb of the oldest records in the dataset. 

 

 

Microsoft
Microsoft

Re: What happens when you reach the size limit of a Push Dataset with Historical Analysis Enabled

Hi @flennon,

 

You can set a retention policy. Then it will run automatically. Please refer to api-automatic-retention-policy-for-real-time-data.

 

Best Regards,

Dale

Community Support Team _ Dale
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Highlighted
flennon
Frequent Visitor

Re: What happens when you reach the size limit of a Push Dataset with Historical Analysis Enabled

Hi Dale,

 

Are you certain you can set the retention policy for a dataset when 'Historic Data Analysis' is enabled? The solution to this forum post states that PowerBI will store everything without the basicFIFO retention policy if Historic Data Analysis is enabled. 

 

This would make sense as other streaming datasets arent stored, they're just held in memory. 

 

I've also been trying to see what other retention policies there are, but there only seems to be basicFIFO and no retention policy, are there any others?

Microsoft
Microsoft

Re: What happens when you reach the size limit of a Push Dataset with Historical Analysis Enabled

Hi @flennon,

 

Yes, I'm quite sure. I have tested it. Please refer to the snapshot below.

1. If it's only a streaming dataset, I can't create a report.

2. If it's only a push data set, it won't have an API info.

3. I pushed 300K rows into it from ID 1 to ID 300000. (I intended to do so. It's obvious my code doesn't that precise. But the 300K data is pushed successfully)

There are only 226K rows exist. And the minimum ID is 80001.

What happens when you reach the size limit of a Push Dataset with Historical Analysis Enabled2.png

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Best Regards,

Dale

Community Support Team _ Dale
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

flennon
Frequent Visitor

Re: What happens when you reach the size limit of a Push Dataset with Historical Analysis Enabled

Thank you very much Dale

Helpful resources

Announcements
Announcing the New Spanish Forum

Announcing the New Spanish Forum

Do you need help in Spanish? Check out our new Spanish community section.

MBAS Gallery 2020

MBAS Gallery 2020

Watch Microsoft Business Applications Summit sessions on-demand.

‘Better Together’ Integration Forum Launch

‘Better Together’ Integration Forum Launch

We've launched a how-to forum where you can learn about how Power BI integrates with other Power Platform products.