Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.

Reply
LucTP
Frequent Visitor

Filldown Query when input incorrect Value

LucTP_0-1709990889725.png 

How to FIlldown query input incorrect value to correct with same value 
Thank you!

 

1 ACCEPTED SOLUTION

@LucTP,

 

  1. change address to your source excel file in Source step.
  2. if you want to add more CharsToRemove you can do it here
    dufoq3_0-1710077967690.png
  3. if you want to add more StringsToReplace add them here
    dufoq3_0-1710079377947.png

     

    There are still some differences like theese but it is hard to catch everything

    dufoq3_1-1710079422896.pngdufoq3_2-1710079433681.png
    dufoq3_3-1710079441054.png

     

    dufoq3_4-1710079456149.png

 

let
    Source = Excel.Workbook(File.Contents("Address\Data.xlsx"), null, true),
    Sheet1_Sheet = Source{[Item="Sheet1",Kind="Sheet"]}[Data],
    PromotedHeaders = Table.PromoteHeaders(Sheet1_Sheet, [PromoteAllScalars=true]),
    CharsToRemove = ",.""-<>()[]{}",
    CharsToReplace = List.Buffer({{"&", "AND"}, {"LIMITED", "LTD"}, {"LIMITTED", "LTD"}}),
    StepBack = PromotedHeaders,
    Ad_CompanyCleaned = Table.AddColumn(StepBack, "Company Cleaned", each 
      [ removeChars = List.Select(Text.SplitAny([Company], CharsToRemove & " "), (x)=> not List.Contains(Text.ToList(CharsToRemove) & {"", " "}, x)),
        replaceChars = List.ReplaceMatchingItems(removeChars, CharsToReplace),
        trimEndS = List.Transform(replaceChars, (x)=> Text.TrimEnd(x, {"S", "s"})),
        combine = Text.Combine(trimEndS, " ")
      ][combine], type text),
    GroupedRowsCheck = Table.Group(Ad_CompanyCleaned, {"Company Cleaned"}, {{"1", each 1}}),
    RemovedColumns1 = Table.RemoveColumns(GroupedRowsCheck,{"1"}),
    MergedQueries = Table.FuzzyNestedJoin(RemovedColumns1, {"Company Cleaned"}, Ad_CompanyCleaned, {"Company Cleaned"}, "GroupedRowsCheck", JoinKind.LeftOuter, [IgnoreCase=true, IgnoreSpace=true, Threshold=0.8]),
    Ad_CompanyCommonName = Table.AddColumn(MergedQueries, "Company Common Name", each [GroupedRowsCheck]{0}[Company Cleaned], type text),
    RemovedColumns2 = Table.RemoveColumns(Ad_CompanyCommonName,{"GroupedRowsCheck"}),
    MergedQueries2 = Table.NestedJoin(Ad_CompanyCleaned, {"Company Cleaned"}, RemovedColumns2, {"Company Cleaned"}, "RemovedColumns1", JoinKind.LeftOuter),
    ExpandedRemovedColumns1 = Table.ExpandTableColumn(MergedQueries2, "RemovedColumns1", {"Company Common Name"}, {"Company Common Name"}),
    RemovedColumns3 = Table.RemoveColumns(ExpandedRemovedColumns1,{"Company Cleaned"})
in
    RemovedColumns3

 


Note: Check this link to learn how to use my query.
Check this link if you don't know how to provide sample data.

View solution in original post

6 REPLIES 6
LucTP
Frequent Visitor

Hi everyone,

I have a question about power Bi clean data, the data is not correct as this sample

NameAddress
CENSEA INC 
CENSEA INC. 
CENTRAL COLDSTORAGE KUCHING SDN. BHD 
CENTRAL COLDSTORAGE KUCHING SDN. BHD., 
CENTRO CUESTA NACIONAL 
CENTRO CUESTA NACIONAL., 
CENTRO CUESTA NACIONAL., 

 

result correct data

 

NameAddress
CENSEA INC 
CENSEA INC 
CENTRAL COLDSTORAGE KUCHING SDN. BHD 
CENTRAL COLDSTORAGE KUCHING SDN. BHD 
CENTRO CUESTA NACIONAL 
CENTRO CUESTA NACIONAL 
CENTRO CUESTA NACIONAL 

 

 

 Could you please have anyone help me?

 

I've chcecked your data roughly. It seems you need remove extra spaces and some characters like , and .

I'll create it for you when I come home. Probably within next two hours.


Note: Check this link to learn how to use my query.
Check this link if you don't know how to provide sample data.

@LucTP,

 

  1. change address to your source excel file in Source step.
  2. if you want to add more CharsToRemove you can do it here
    dufoq3_0-1710077967690.png
  3. if you want to add more StringsToReplace add them here
    dufoq3_0-1710079377947.png

     

    There are still some differences like theese but it is hard to catch everything

    dufoq3_1-1710079422896.pngdufoq3_2-1710079433681.png
    dufoq3_3-1710079441054.png

     

    dufoq3_4-1710079456149.png

 

let
    Source = Excel.Workbook(File.Contents("Address\Data.xlsx"), null, true),
    Sheet1_Sheet = Source{[Item="Sheet1",Kind="Sheet"]}[Data],
    PromotedHeaders = Table.PromoteHeaders(Sheet1_Sheet, [PromoteAllScalars=true]),
    CharsToRemove = ",.""-<>()[]{}",
    CharsToReplace = List.Buffer({{"&", "AND"}, {"LIMITED", "LTD"}, {"LIMITTED", "LTD"}}),
    StepBack = PromotedHeaders,
    Ad_CompanyCleaned = Table.AddColumn(StepBack, "Company Cleaned", each 
      [ removeChars = List.Select(Text.SplitAny([Company], CharsToRemove & " "), (x)=> not List.Contains(Text.ToList(CharsToRemove) & {"", " "}, x)),
        replaceChars = List.ReplaceMatchingItems(removeChars, CharsToReplace),
        trimEndS = List.Transform(replaceChars, (x)=> Text.TrimEnd(x, {"S", "s"})),
        combine = Text.Combine(trimEndS, " ")
      ][combine], type text),
    GroupedRowsCheck = Table.Group(Ad_CompanyCleaned, {"Company Cleaned"}, {{"1", each 1}}),
    RemovedColumns1 = Table.RemoveColumns(GroupedRowsCheck,{"1"}),
    MergedQueries = Table.FuzzyNestedJoin(RemovedColumns1, {"Company Cleaned"}, Ad_CompanyCleaned, {"Company Cleaned"}, "GroupedRowsCheck", JoinKind.LeftOuter, [IgnoreCase=true, IgnoreSpace=true, Threshold=0.8]),
    Ad_CompanyCommonName = Table.AddColumn(MergedQueries, "Company Common Name", each [GroupedRowsCheck]{0}[Company Cleaned], type text),
    RemovedColumns2 = Table.RemoveColumns(Ad_CompanyCommonName,{"GroupedRowsCheck"}),
    MergedQueries2 = Table.NestedJoin(Ad_CompanyCleaned, {"Company Cleaned"}, RemovedColumns2, {"Company Cleaned"}, "RemovedColumns1", JoinKind.LeftOuter),
    ExpandedRemovedColumns1 = Table.ExpandTableColumn(MergedQueries2, "RemovedColumns1", {"Company Common Name"}, {"Company Common Name"}),
    RemovedColumns3 = Table.RemoveColumns(ExpandedRemovedColumns1,{"Company Cleaned"})
in
    RemovedColumns3

 


Note: Check this link to learn how to use my query.
Check this link if you don't know how to provide sample data.

dufoq3
Super User
Super User

You have to identify incorrect rows (add seperate column) and then replace incorrect rows with null. Then you can apply fill down. If you don't know how to do it. Provide sample data (as table so we can copy/paste) and also expected resu.t


Note: Check this link to learn how to use my query.
Check this link if you don't know how to provide sample data.

LucTP
Frequent Visitor

What about expected result based on sample data? How do I know which value is incorrect?


Note: Check this link to learn how to use my query.
Check this link if you don't know how to provide sample data.

Helpful resources

Announcements
Microsoft Fabric Learn Together

Microsoft Fabric Learn Together

Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City

PBI_APRIL_CAROUSEL1

Power BI Monthly Update - April 2024

Check out the April 2024 Power BI update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors