Power Query Append Issue

[AndrewKent]

Hi there,

I've noticed a strange issue with Power Query (and I assume the same functionality in Power BI) whereby I have 2 connections loaded in Power Query (both csv files with c15K lines of data) and a third query that ties the two together;

Source = csv_data_candidates_2016,

#"Promoted Headers" = Table.PromoteHeaders(Source),

#"Appended Query" = Table.Combine({#"Promoted Headers", csv_data_candidates_2017}),

The problem is that I have noticed that data is being dropped in the "Append" part for "csv_data_candidates_2017" although I can see the missing data in the original connection.

If I delete the "Append" query and add it in again, it works fine!?

The problem is I don't want to have to keep re-editing the query everytime this happens (as the "csv_data_candidates_2017" will continue to grow)

Does anyone know why this would be an issue?

Andy

[v-yuezhe-msft]

Hi @AndrewKent,

I am not able to reproduce your issue in Power BI Desktop, below is the code generated for appended query for the CSV files in my scenario. Could you please share me the CSV files so that I can reproduce?

let
    Source = Csv.Document(File.Contents("C:\Users\v-yuezhe\Desktop\testcsv\de_postal_codes.csv"),[Delimiter=",", Columns=7, Encoding=1252, QuoteStyle=QuoteStyle.None]),
    #"Promoted Headers" = Table.PromoteHeaders(Source, [PromoteAllScalars=true]),
    #"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"Postal Code", Int64.Type}, {"Place Name", type text}, {"State", type text}, {"State Abbreviation", type text}, {"City", type text}, {"Latitude", type number}, {"Longitude", type number}}),
    #"Appended Query" = Table.Combine({#"Changed Type", #"de_postal_codes - Copy"})
in
    #"Appended Query"

Besides, you can put the CSV files in a folder, then connect to  the folder using Get Data->File-> Folder entry in Power BI Desktop and check if there are any missing data.

Thanks,
Lydia Zhang

[AndrewKent · Accepted solution]

Hi,

haven't been able to recreate this issue either. I have been expereincing multiple issues like this (and connection issues to datasets that were working fine) however restarting Excel seems to clear them up.

Thanks for your help.

Andy

[v-yuezhe-msft]

Hi @AndrewKent,

You can mark helpful reply as answer to close this thread. If you get any new issues, feel free to post back.

Thanks,
Lydia Zhang

[subhotech]

Hi ,

I have two data set in powerBI query editor which are connected to MS Sql Server.  I am trying to union those data sets to create a new dataset having all the records using 'Append as New' functionality.  I can see missing records in the final dataset.  

Source A has 10 records ( example)

Source B has 20 records ( example) 

Final Dataset has < 30 records.

Both dataset has same number/order of columns which is necessary for doing union i.e Append in PowerBI. Please Advice!

Regards,

[subhotech]

Hi ,

 I am experiencin similar issues. I have two data sources in Query editor (c connected to SQL Server). All i trying to do is union of both the datasets using the 'Append as New' option in Query Editor.   Source A has 10 records and source B 20 records. Append query is exptected to produce a result set having 20 records including the duplicates. Some records are not present in the New Data Set.   Please Advice !

Regards,