Incremental refresh – not working properly

[EmaVasileva]

Hi team,

I have a large dataset using SQL database as a source, which have 40+tables and it is updated daily. It has data from Jan 2020 until today. I would like to keep the data from 2020 until the most recent date, but to refresh only the last 3 days. Most of my tables have a date&time column, which I filtered by RangeStart and RangeEnd parameters.
  

On every table, which I filtered by RangeStart and RangeEnd parameters, I set the Incremental refresh this way:

* Name of the table is different based on the selected tables, but the refresh/archive periods which I apply to all tables with date&time column are the same.

After that, I uploaded the report to PBI Service and set an automatic refresh. But I noticed that the time for refresh is the same as before. I think that the time (to refresh only the past 3 days) should be shorter than the full dataset refresh (without the incremental refresh).

Please let me know if there is something I did wrong or what step should I make in order for my automatic refresh to refresh data only for the last 3 days? (period between Jan 2020 until date of refresh, should be archived and not refreshed)

*I tested to set the parameters for a shorter period, (ex. 2022 beginning-now), but after that I am not able to use data from 2020 in the reports, so it’s not working this way for me.

Thank you.

[selimovd]

Hey @EmaVasileva ,

is your dataset on a premium or premium per user capacity?

If that's the case you can connect with SQL Server Management Studio or Tabular Editor and see if the partitions were created.

Be aware the first load is a full load, so you should see the impact after the initial load.

Also may I ask how many rows you have? If you don't have a few million rows, then usually incremental refresh doesn't make sense as the engine needs more time to combine the partitions for the analysis.

If you need any help please let me know.
If I answered your question I would be happy if you could mark my post as a solution ✔️ and give it a thumbs up 👍

Best regards
Denis

Blog: WhatTheFact.bi
Follow me: twitter.com/DenSelimovic

[EmaVasileva]

Hi @selimovd,
My dataset is Pro, so I'm not sure that the Tabular/SSMS works. Almost all of my tables are betwen 250 k and 500k, one or two are 1-2 M.

So, from what I understand, the way I set my incremental refresh is correct, but the number of rows are not so big, so the time will not be so different than the normal refresh(because of the engine needs for more time to combine the partitions for the analysis). Right?

Thank you.