Reducing/grouping/binning time series data

[Anonymous]

Hi folks. I'm just starting out with Power BI, and have what I think should be a simple problem. I've tried some googling, and all the suggestions seem to me to be more complex than they should be. I'm wondering if I'm missing something obvious.

I have a bunch of time series data, with data points every 15 minutes. I'd like to reduce the memory load and speed things up by grouping the time series into hourly bins. This should make my dataset 25% the length that it was previously.

What is the best way to do this? Does it happen at the import stage, or later on? Any tips or links to relevant tutorial videos would be greatly appreciated.

Cheers, Eddie

[parry2k]

@Anonymous I would recommend to do it at import stage, if you data source is sql , you can group it at hourly basis before importing. 

[Anonymous]

@parry2k, and if it is a CSV? 🙂

[parry2k]

@Anonymous well in that case you have to upload full csv and then it does't matter bin at hours or minutes, you can use groupby in power query to reduce number of rows but you have to import all the rows at first stage. GroupBy is expensive query thou and not sure if you really want to use it. Also depends how many records you are importing 1 million, 20 million, 100K

[Anonymous]

Ok, interesting. While googling, I came across the topic of 'Aggregating'. Is that relevant here at all? It sounded sort of like what I was interested in, especially when they spoke about improving performance.

My CSV data has 1.5M rows. If I use GroupBy, will it speed up the refresh rate for viewers on the online dashboards?

[parry2k · Accepted solution]

@Anonymous aggregating is totally different concept. 1.2 mil rows is not much, and also it is import and your users will not feel any performance hit because it is not direct query. There is no need of groupby as it is not going to help in anything

[Anonymous]

Thanks for your help. 🙂