Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
flennon
Frequent Visitor

What happens when you reach the size limit of a Push Dataset with Historical Analysis Enabled

I thought up until recently that push datasets followed the 200k first in first out (basciFIFO) retention policy. I realise now that this is just for streaming datasets as the 200k rows is stored in cache.

 

I am pushing into the dataset using the RESTful API but I have historical analysis enabled so that I could avail of all the different types of charts for my dashboards. 

 

I see here that datasets are limited to 1gb. What happens when that limit is reached. Is there any FIFO system here? I know you cant selectively delete from a dataset, its either all or nothing. Would that mean that when I hit the 1gb limit, I'd have to delete out all the data and start again?

1 ACCEPTED SOLUTION

Hi @flennon,

 

Yes, I'm quite sure. I have tested it. Please refer to the snapshot below.

1. If it's only a streaming dataset, I can't create a report.

2. If it's only a push data set, it won't have an API info.

3. I pushed 300K rows into it from ID 1 to ID 300000. (I intended to do so. It's obvious my code doesn't that precise. But the 300K data is pushed successfully)

There are only 226K rows exist. And the minimum ID is 80001.

What happens when you reach the size limit of a Push Dataset with Historical Analysis Enabled2.png

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Best Regards,

Dale

Community Support Team _ Dale
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

6 REPLIES 6
az2451
Resolver I
Resolver I

The answer to your question is right below: https://docs.microsoft.com/en-us/power-bi/service-admin-manage-your-data-storage-in-power-bi#what-ha...

 

You simple have to reduce the size to create new dashboards etc.

 

There is not other way around, thats the rule.

 

You can try it out by your self..

 

 

Greetings

My apologies, I did not communicate my point across well enough.

 

I understand that I have to reduce the size of the dataset. However, when attempting to find a way to reduce the number of records in the dataset, I found that there is no method to delete specific rows (i.e. You cant delete rows older than a certain date), you can only delete all rows.

 

Therefore, I'm asking is there something I'm missing, or do I really have to delete all the rows in the dataset and start again when I hit the maximum capacity? Rather than being able to delete 50mb of the oldest records in the dataset. 

 

 

Hi @flennon,

 

You can set a retention policy. Then it will run automatically. Please refer to api-automatic-retention-policy-for-real-time-data.

 

Best Regards,

Dale

Community Support Team _ Dale
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Hi Dale,

 

Are you certain you can set the retention policy for a dataset when 'Historic Data Analysis' is enabled? The solution to this forum post states that PowerBI will store everything without the basicFIFO retention policy if Historic Data Analysis is enabled. 

 

This would make sense as other streaming datasets arent stored, they're just held in memory. 

 

I've also been trying to see what other retention policies there are, but there only seems to be basicFIFO and no retention policy, are there any others?

Hi @flennon,

 

Yes, I'm quite sure. I have tested it. Please refer to the snapshot below.

1. If it's only a streaming dataset, I can't create a report.

2. If it's only a push data set, it won't have an API info.

3. I pushed 300K rows into it from ID 1 to ID 300000. (I intended to do so. It's obvious my code doesn't that precise. But the 300K data is pushed successfully)

There are only 226K rows exist. And the minimum ID is 80001.

What happens when you reach the size limit of a Push Dataset with Historical Analysis Enabled2.png

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Best Regards,

Dale

Community Support Team _ Dale
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Thank you very much Dale

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.