I would like to set up incremental refresh based on "loaded date".
My data set does not have a date in it, so, I add a new column, and fill it using M in the query editor
Loaded = DateTime.LocalNow()
In a month, I will get a new file that I want to add to the dataset. (I do not want to overwrite the existing dataset)
This will happen every month - so the dataset will keep getting larger. I want to eventually store many months of data.
So, I have set up incremental refresh using "loaded date", but it does not seem to be working.
Does anyone have a solution that allows me to keep the original data or how to set up incremental refresh using my "loaded date"
Thanks in advance
For incremental refresh, datasets are filtered by using Power Query date/time parameters with the reserved, case-sensitive names RangeStart and RangeEnd.
Since your data set does not have a date in it, It difficult to filter your dataset.
and In a month, you will get a new file that I want to add to the dataset. So i recommend you use get data from the folder,
Then use file attributes(e.g. date Modified/ Date created ) as the date filter.
Here is a post for you as reference:
I Tried doing that, but the incremental refresh still did not work.
I was able to bring in the file attributes - date created, and then set up an incremental refresh on that column.
I loaded the data into my model, and published it.
To test it, I chnaged the data in an old file (that has a created date outside the incremental refresh settings), and then I refreshed the dataset.
The old data still refreshed.
Am I missing something or doing something wrong?
Do you have all the right Settings for it? you may refer to these articles