Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.
Good morning!
I'm beginning to dive into dataflows using a premium environment. If I understood incremental refreshes correctly, only new data should get refreshed, right? I think with my current setup, the entire entity gets refreshes on a scheduled refresh. I think. If not, then my question is: Is there a way to create a column maybe called "CreatedOn" and show when the data was added to the entity?
The setup:
1. Create custom entity for dataflows with initial one month of historical data from a folder on a server.
2. Single table is supplied / consumed with two datetime columns: Date and LastRefreshed:
Date = Table.AddColumn(#"Filtered Rows", "Date", each DateTime.FromText([Month] & "/1/" & [Year]), type datetime),
LastRefreshed = Table.AddColumn(Date, "LastRefreshed", each DateTime.From(DateTimeZone.ToText(DateTimeZone.FixedUtcNow(), "dd-MMM-yyyy hh:mm tt")), type datetime),
#"HFMData-44617465-autogenerated_for_incremental_refresh" = Table.SelectRows(LastRefreshed, each DateTime.From([Date]) >= RangeStart and DateTime.From([Date]) < RangeEnd)
in
#"HFMData-44617465-autogenerated_for_incremental_refresh"
3. Initial Refresh + add incremental refresh settings:
4. Connected to dataflow and check first dataset last refreshed:
5. Added new data to folder and waited for scheduled refresh. From what I can understand, the second refresh shouldn't change the values of the unchanged data but upon second refresh it seems to have:
Looking into the parameters supplied by the auto-generated RangeStart and RangeEnd I see "1/1/1901" and "12/31/9999" so I'm assuming it's picking up all the data within that range on refresh? Please correct me and guide me in the right direction if I'm wrong.
You may try using original column instead of a custom column to detect data changes.
Hi @v-chuncz-msft
When you say original column instead of custom column. What do you mean?
Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City
Check out the April 2024 Power BI update to learn about new features.
User | Count |
---|---|
109 | |
98 | |
80 | |
64 | |
57 |
User | Count |
---|---|
145 | |
111 | |
92 | |
84 | |
66 |