Get data- Folder contains CSV files

[HadasSpier]

Hi everyone,

The folder contains CSV files with more or less similar columns but in a different order.
For example:
CSV1 contains columns: A, B, C, D
CSV2 contains columns: A, D, C, B
CSV3 contains columns: C, B, A
CSV4 contains columns: D, C, B, A
Does anyone have an idea of how I can best load my data?

Thank you!

[v-easonf-msft · Accepted solution]

Hi, @HadasSpier 

There is no restriction on how large a pbix file can be locally. However, when you publish a pbix file over 1GB to Power BI Service, you will get a file size limit message. 

I cannot reproduce the problem you encountered. Everything is working well on my side, and there will be no case of putting column A into column B. It is recommended to recheck the column names of your related tables.

If you still have this issue for Power BI, you'd better create a support ticket in Power BI Support to get further help.

Best Regards,
Community Support Team _ Eason

[HadasSpier]

Thank you!!!!

[StefanoGrimaldi]

Currently Power BI desktops files have a limit for 1GB of space, those 150k x 83 its around 12gb but Power BI greatly compress the info of the file, check the end size how much its within the file. 

[HadasSpier]

Good to know! 

how can I check it?

[collinq]

HI @HadasSpier ,

If the header name is the same and the data type is the same, then when you use Folder and "Combine and Transform Data" then it will put them together.  I tested this and I ended up with four columns, with the CSV3, as expected, having nulls for column D for those rows.

If you are NOT wanting the columns combined then you might be best of bringing them in and selecting transform data only and doing it manually.

Or, of course, if you want the most control, don't use folder at all and bring them in one at a time and append/merge as needed.

I would appreciate Kudos if my response was helpful. I would also appreciate it if you would Mark this As a Solution if it solved the problem. Thanks!

[HadasSpier]

Hi!

If the header name is the same and the data type is the same, but when I use Folder and "Combine and Transform Data" I have errors and it didn't put them together (for some of the CSVs)
I think maybe it is not working because my data is too heavy?

Thank you!!

[StefanoGrimaldi]

you should post the error over here with image for better asistence, this way we can look over the error. in the best cast scenario post a dummy data pbi with the error and dummy data files. 

[HadasSpier]

Hi,

The error is due to the fact that it switched between the fields.
I.e. put column A in B and so on. I'm wondering if this is because I have a lot of CSV files in the folder I loaded (83) each of them is 150,000KB.

Thank you!