cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Highlighted
Regular Visitor

Push Dataset is empty when reached the 200K limit

Hi,

 

I'm using a Push Dataset which received data from Stream Analytics.

The Dataset is created with de default values (basicFIFO). But from the weekend I have some problems, it works well before this weekend.

Now, when the Dataset reach the 200K records limit (a bit more of that 200K), the Dataset became empty. All content is deleted, not only the oldest as FIFO had to do.

Is this the normal functioning?

 

Thanks

Raúl

4 REPLIES 4
Highlighted
Super User V
Super User V

Re: Push Dataset is empty when reached the 200K limit

yes, there are a number of events that can cause the dataset to flush out. If you need the history then you should use a hybrid streaming dataset instead.

Highlighted
Regular Visitor

Re: Push Dataset is empty when reached the 200K limit

How can I do it from Stream Analytics?

I thought this is the type created by default from Stream Analytics, a mix between push and streaming.

Highlighted
Microsoft
Microsoft

Re: Push Dataset is empty when reached the 200K limit

Hi @RaulJaramillo ,

Please check if the solution in the following thread can help you.

What happens when you reach the size limit of a Push Dataset with Historical Analysis Enabled

Best Regards

Rena

Highlighted
Regular Visitor

Re: Push Dataset is empty when reached the 200K limit

Thanks @yingyinr,

 

I saw theis thread before ask this question. But I think is not my case.

The retention policy of my Dataset is basicFIFO, I have enabled the 'Historical data analysis', and each record has few information, something like name-value, then I never reach the size limit before reach the 200K records limit.

Last week the dataset is working as the documentation said, when the 200K records limit is reached, the oldest are deleted but keep information in the dataset.

But now, the same dataset, with the same configuracion, when reach the 200K limit, delete all data, let the dataset without any record. This is not what I read in the documentation for basicFIFO, and I cannot delete old records of the dataset from Stream Analytics because it is not allowed to write 'delete' in que query.

I think my dataset has the same configuration, but it seems that it does not work the same as the solution in the other thread.

 

My dataset receive information every minute from Stream Analytics, and I need to have the last hour of records (around 60K records and size less than 10MB). Deleting all records in the dataset, It's an hour with wrong informacion in the reports.

 

Regards

Raul

Helpful resources

Announcements
Ignite

Microsoft Ignite

This will be a conference that you do not want to miss!

Get Ready for Power BI Dev Camp

Get Ready for Power BI Dev Camp

Don't miss the Power BI Dev Camp this week!

August Community Highlights

Check out a full recap of the month!

August 2020 CYST Challenge

Check out the winners of the recent 'Can You Solve These?' community challenge!

Experience what’s next for Power BI

Join us for an in-depth look at the new Power BI features and capabilities at the free Microsoft Business Applications Launch Event.

Top Solution Authors
Top Kudoed Authors