Cleaning up duplicates

[jguy]

I'm importing data from a SQL server for a look up table and am running into an issue with duplicate records. I have tried using the 'Remove Duplicates' option, but am having no luck. I have exported the table into excel to identify the duplicate records, and they are showing up like this:

HF-TRH2
HF-TRH2
182699
184529
186358
188188
2400-3-17
2405-3-21
2410-3-24
2415-3-28
005001
5001
3/17/2400
3/21/2405
3/24/2410
3/28/2415

These are supposed to be product numbers. Excel is identifying them as duplicate values via conditional formatting.

Is there a way I can remove just these records, or is this something I am going to have to get done on the SQL side of things before import?

Apologies if this is a bit of a newb question. I have only been working with Power BI for a few weeks, and this is my first real go at building a decent data model.

Thanks!

[v-shex-msft · Accepted solution]

Hi @jguy ,

You can turn on advanced option and use t-sql query to get distinct records from power bi side.

Visualize data using the Azure Data Explorer connector for Power BI

Regards,

Xiaoxin Sheng

[jguy]

Will this let me go in and remove those records and reload the table?

[v-shex-msft]

Hi @jguy ,

When you add t-sql query to connector, power bi will use this query on your database and receive the result table. 

Regards,

Xiaoxin Sheng