incorrect rowcounts when using flow and streaming data

[wilson_smyth]

Im using Flow to capture survey data from a microft O365 form and save it to a sql database table.

Im then using streaming data in PowerBI to display this data in Powerbi.

The problem is my rowcounts in PowerBI are completely wrong.

I need help figuring how to make this work.

Each time a form is submitted the flow kicks off.

1. it saves the data to a database table.

2. it gets all records from that table.

3. it puts these rows into the RealtimeData table of my powerBi Streaming dataset.

For the first entry in the table, everything looks ok.

On submitting a second form, my rowcount in RealtimeData is now 3.

On submitting a third form, my rowcount in realtimeData table in PowerBi is now 9.

Its clear that powerBi is appending the same records to end of the RealtimeData table each time the flow runs.

Problem is im unsure how to ensure the RealtimeData table is emptied on each run of the Flow.

Appreciate any advice on how to fix this

[v-shex-msft]

Hi @wilson_smyth,

I'd like to suggest you pause your flow and try again, it is hard to ensure the calculation of history data of live streaming dataset.

BTW, it also appears on other calculation(first,last) which other users mentioned but I can't reproduce.

Regards,

Xiaoxin Sheng

[wilson_smyth]

Hi.

thanks, but that doesnt solve the issue.
Its down to the fact that teh "RealTimeData" table needs the historic flag set to true in order to store data, and the flow is adding all records from the table each time. the "realTimeData" table is getting the same dataset over and over.

I have since learned that  i need to access the powerbi API and empty the dataset.

theres a demo of how to do this here, by Konstantinos Ioannou which explains it fantastically. Its a lot of effort however, when the flow shape could simply have a check box that says "empty dataset before adding rows" or something similar.