Is there a way to remove duplicates without breaking query folding? I need the folding (for incremental refresh), but I have a ton of duplicates (I go from 3 0378 271 rows to 10 472 after removing duplicates)...
Hi @grggmrtn ,
This can happen if the date ranges are set NOT up to as follows: [Date] >= RangeStart and [Date] < RangeEnd. Is your upper bound set to be equal to or less than or just less than?
Refer to this thread:
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Hi @V-lianl-msft thanks for the reply...
Your answer would make sense if I had actually set ranges, but I haven't gotten to setting the ranges yet, because query folding on my dataset breaks. The final step in my query (remove duplicates) breaks the folding.
So my question is, how can I remove the duplicates without breaking the folding?
Never mind. Table.Distinct() only breaks the fold if you use it to remove duplicates from a subset of the tables columns, instead of the whole table. Using Table.Distinct() on the entire table does not break the query folding.
Learn how to create your own user groups today!
Click here to read more about the November 2021 Updates!
Join us, in-person, December 7–9 in Las Vegas, for the largest gathering of the Microsoft community in the world.