Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
Irwin
Helper IV
Helper IV

Filter column for unique values and another column in Power Query

Hi Everyone,

 

I have a very large table that I need to reduce in size. I need to filter column A to only have unique values and display a specific value Q, X or Z from column B. From column B I need to display Q>X>Z. That means if column B contains Q, then display Q, else X and else Z.

 

All help appreciated. 🙂


Example on data.

AB
1Z
1Z
1X
1Z
1Z
1Z
1Z
1Z
2Z
2Z
2Q
2Z
3Z
3Z
3Z
3Z
3Z
4Z
4Z
4Z
4Q
4Z

 

Desired output:

AB
1X
2Q
3Z
4Q
2 ACCEPTED SOLUTIONS
JorgePinho
Solution Sage
Solution Sage

Hello @Irwin !

You can sort in ascending order column B and then remove duplicates on column A.

View solution in original post

Hi,
It did not work. It seems this solution works in excel. However, PBI remembers the original sort and therefore cannot keep the value I want i column B once I remove duplicates.


I found a solution that work when applied to your suggestion. You need to use the table function called "Table.Buffer".

https://learn.microsoft.com/en-us/powerquery-m/table-buffer

A video can be seen here.
https://www.youtube.com/watch?v=rqDdnNxSgHQ&list=PLDz00l_jz6zzttb28XH8GHZNL6vvpBlkQ&index=23

View solution in original post

7 REPLIES 7
v-rongtiep-msft
Community Support
Community Support

Hi @Irwin ,

Plaese refer to my steps: 

let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlTSUYpSitVBZ0VgESOGZYSDFYgmZkwyy4QgKxBZLBYA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [A = _t, B = _t]),
    #"Changed Type" = Table.TransformColumnTypes(Source,{{"A", Int64.Type}}),
    #"Grouped Rows" = Table.Group(#"Changed Type", {"A", "B"}, {{"Count", each Table.RowCount(_), Int64.Type}}),
    #"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom", each List.Min(Table.SelectRows(#"Grouped Rows",(x)=>x[A]=[A])[Count])),
    #"Added Conditional Column" = Table.AddColumn(#"Added Custom", "Custom.1", each if [Count] = [Custom] then [B] else null),
    #"Filtered Rows" = Table.SelectRows(#"Added Conditional Column", each ([Custom.1] <> null)),
    #"Reordered Columns" = Table.ReorderColumns(#"Filtered Rows",{"A", "B", "Custom.1", "Count", "Custom"}),
    #"Removed Columns" = Table.RemoveColumns(#"Reordered Columns",{"Custom.1", "Count", "Custom"})
in
    #"Removed Columns"

vpollymsft_0-1668577401747.png

 

If it still does not help, please provide more details with your desired output.

 

Best Regards

Community Support Team _ Polly

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Hi,
Thank you for your kind help. This might be a solution, but I think it is not feasible to do on a data source with millions of rows.

JorgePinho
Solution Sage
Solution Sage

Hello @Irwin !

You can sort in ascending order column B and then remove duplicates on column A.

Wauv... this is a very simple solution and weirdly enough I think it might actually work. I will play around with it and see if there is no errors. 
Thank you.

I already used it sometimes and I think it doesn't make any errors. However, if it happens in your case please let me know 😁

Hi,
It did not work. It seems this solution works in excel. However, PBI remembers the original sort and therefore cannot keep the value I want i column B once I remove duplicates.


I found a solution that work when applied to your suggestion. You need to use the table function called "Table.Buffer".

https://learn.microsoft.com/en-us/powerquery-m/table-buffer

A video can be seen here.
https://www.youtube.com/watch?v=rqDdnNxSgHQ&list=PLDz00l_jz6zzttb28XH8GHZNL6vvpBlkQ&index=23

The first comment on that video link provided a very easy solution I think.

1) Sort as needed. (Sort Column B Ascending here)

2) Create Index column (*magic step*, will override original index during existence) 

3) Select Column A. Use Remove Duplicate rows. (will keep the first instance of each)

4) Remove index column. (if desired)

5) Sort by Column A or whatever you wish.

Phoen1x09_0-1679512406987.png

 

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.