Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more.
Get startedGrow your Fabric skills and prepare for the DP-600 certification exam by completing the latest Microsoft Fabric challenge.
Hi all, I would like to remove lines with duplicated "ID" and keep the lowest "price" line. May I know if there any quick way(not using groupby) I could do for this case?
Original Table
ID | Price |
1 | 4 |
1 | 346 |
3 | 27 |
4 | 458 |
5 | 734 |
5 | 6 |
7 | 45 |
7 | 85 |
7 | 7 |
Desired outcome:
ID | Price |
1 | 4 |
3 | 27 |
4 | 458 |
5 | 6 |
7 | 7 |
Appreciated if any help
Solved! Go to Solution.
@ngct1112 try this
CT = Table.TransformColumnTypes(#"Promoted Headers", {{"ID", Int64.Type}, {"Price", Int64.Type}}),
#"Filtered Rows" = Table.SelectRows(
CT,
each ([Price] = List.Min(Table.SelectRows(CT, (q) => q[ID] = [ID])[Price]))
)
Easier way, add sorted rows, buffer and remove duplicate step
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WclTSUTI0UIrVgTDNwSwnIMsSzjI0UoqNBQA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t, Column2 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}, {"Column2", Int64.Type}}),
#"Sorted Rows" = Table.Sort(#"Changed Type",{{"Column2", Order.Ascending}}),
#"Buffered" = Table.Buffer(#"Sorted Rows"),
#"Removed Duplicates" = Table.Distinct(Buffered, {"Column1"})
in
#"Removed Duplicates"
Easier way, add sorted rows, buffer and remove duplicate step
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WclTSUTI0UIrVgTDNwSwnIMsSzjI0UoqNBQA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t, Column2 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}, {"Column2", Int64.Type}}),
#"Sorted Rows" = Table.Sort(#"Changed Type",{{"Column2", Order.Ascending}}),
#"Buffered" = Table.Buffer(#"Sorted Rows"),
#"Removed Duplicates" = Table.Distinct(Buffered, {"Column1"})
in
#"Removed Duplicates"
@ngct1112 try this
CT = Table.TransformColumnTypes(#"Promoted Headers", {{"ID", Int64.Type}, {"Price", Int64.Type}}),
#"Filtered Rows" = Table.SelectRows(
CT,
each ([Price] = List.Min(Table.SelectRows(CT, (q) => q[ID] = [ID])[Price]))
)
This feels like a DAX sort of approach. In M, you want to be very careful about doing table filtering like this since the O(n^2) complexity will make it impractical for larger datasets.
In this particular case, we can do this with a simple Group By:
= Table.Group(#"Changed Type", {"ID"}, {{"Price", each List.Min([Price]), type nullable number}})
@AlexisOlson This feels like a DAX sort of approach - yes it is 😀 I was simply testing out how can I avoid groupby (OP wants to avoid groupby)and still get there and translate DAX to M
Ah, this makes much more sense now. I must have skimmed over that particular requirement.
Group by is the perfect tool for this, so I was rather confused about why no one suggested it. Turns out I need to learn how to read.
Join the community in Stockholm for expert Microsoft Fabric learning including a very exciting keynote from Arun Ulag, Corporate Vice President, Azure Data.
Ask questions in Eventhouse and KQL, Eventstream, and Reflex.
User | Count |
---|---|
42 | |
35 | |
27 | |
18 | |
17 |