Import Data from Folder with Files That Have Additional Headings

[LoremIpsum]

I need to import data from 2 groups of files that have the same structure EXCEPT 1 group of files has additional columns that I need.

Group 1 has 4 relevant data columns. Group 2 has the same 4 relevant data columns but it also 5 additional columns I need to import. Both groups have multiple files.

One a single day, where there is 1 file from each group, I can import each file and merge the contents into 1 grand table. I figure, but haven't tested, that I can import from two different folders and then merge aftewards; however, I prefer not asking the users to separate out the files into a group 1 folder and a group 2 folder. The ideal solution would be dumping all the files from both groups into 1 folder, importing, and letting the program merge the contents.

Thank you.

[v-qiuyu-msft]

Hi @LoremIpsum,

You can place those two kind of files in one folder, then in Power BI desktop get data use the Folder data source, then use Combine Files follow this article: Combine binaries in Power BI Desktop. 

Best Regards,
Qiuyun Yu

[LoremIpsum]

@v-qiuyu-msft I'm not sure that article helps. It specifically says at the end, "With the new behavior of combine binaries, you can easily combine all binaries within a given folder as long as they have the same file type and structure (as in, the same columns)."

Half of my files have less columns than the other have, although the half with less have the same columns as the other group. It's just that the other group has additional columns that I also want to keep.

[v-qiuyu-msft]

Hi @LoremIpsum,

Have tried that Combine Files in desktop? Based on my test, when you click the Combine Files in Query Editor, then select the Excel which has additional column as sample, it will combine additional column as well, if some Excel file doesn't have this additional column, it will display as null value. 

Best Regards,
Qiuyun Yu

[MarcelBeug]

If you combine files from a folder, you can choose which file should act as the example file.

You can try and choose one of the files that has all columns.

Possibly that will be enough to have all files imported correctly.

If not, you may need to specify that missing fields should be ignored.

Anyhow, you can give it a try. If you run into errors, please share these (well specified so we know what is going wrong in which step) so we can help finetuning your queries.

[LoremIpsum]

I had tried your tip before posting without success. Here is what I get back: [Expression.Error] The key didn't match any rows in the table.

Another problem I forsee is my first file is always the shorter version, and the filenames change every day so I can't select the longer version file.

[MarcelBeug]

If all column names are known, then you can adjust the Transform Sample query and use Table.SelectColumns(Name-of-previous-step,list-with-column-names,MissingField.UseNull).

If any of the columns is missing, then the column will be created with null values.

E.g., if your previous step is "Source"  and your required columns are Column1 and Column2:

SelectColumns = Table.SelectColumns(Source,{"Column1","Column2"),MissingField.UseNull)