Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
tgalla010
Regular Visitor

CSV import is splitting JSON data into their own columns

Hey everyone, I'm running into an issue...I have a .csv file that is a table dump from a provider. Hence, I have no real control over the formatting of the data...I have to play the hand I'm dealt. So my issue is that the .CSV has a JSON string that PowerBi is parsing on import since the JSON has commas in the text. I don't think I can merge the columns post import and then try to parse with JSON as PowerBI is pushing the JSON text into columns that may have data for other records. 

 

Options = JSON column

Location ID = ID field that JSON is "spilling" into

country_code = ID field that JSON is "spilling" into

 

So merging these columns then trying to do the JSON parse will end up with an error as some of the records have a Location ID. 

image.png

 

Any thoughts are appreciated. Thank you!

1 ACCEPTED SOLUTION

Hi @tgalla010 ,

 

Glad to hear that you have resolved your problem. Thank you for sharing this wonderful solution, would you please kindly mark your sharing solution as an answer so that it can benefit more users?


Best regards,

 

Community Support Team _ Dong Li
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

13 REPLIES 13
pbiembedded
Regular Visitor

I am facing same issue. 

Did anyone find the solution to this ?

tgalla010
Regular Visitor

I wanted to close this but ended up working around the issue using conditional columns and "contains" function to pull the data out of the field. Thank you.

Hi @tgalla010 ,

 

Glad to hear that you have resolved your problem. Thank you for sharing this wonderful solution, would you please kindly mark your sharing solution as an answer so that it can benefit more users?


Best regards,

 

Community Support Team _ Dong Li
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
v-lid-msft
Community Support
Community Support

Hi @tgalla010 ,

 

Is there a string as json format in each rows of this csv file? Sorry for that, We cannot understand your data model clearly, Could you please provide a sample mockup csv based on fake data? Please don't have any Confidential Information or Real data in it. Please upload your files to One Drive and share the link here.

 

Best regards,

Community Support Team _ Dong Li
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Community Support Team _ Dong Li
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Yes, there is a JSON string in every row. I'm stumped on this one. Thank you!

 

Hi @tgalla010 ,

 

Sorry for that, we cannot access your link. Could you please put several json strings based on fake data into the rows of table? Please don't have any Confidential Information or Real data in it. Please delete your shared link if it contain real data.

 

1.PNG

 

Best regards,

Community Support Team _ Dong Li
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Community Support Team _ Dong Li
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Let me see if I can simplify the example just a bit. 

 

  • Source file is a .csv
  • The "details" header or column contains a JSON string. Here is an example of text from the raw .csv
    •  {"shipstation-synced":true,"shipstation-orderId":5287458}
  • This is what PowerBI is doing to the JSON String. It's being "pushed" into the shopify_order_id column. 
 

PowerBi Example.jpg

Is it always splitting csv into the same set of undesired columns? If so, it should be possible to merge them back with ," as separator and parse the results normally

e.g.
#"Merged Columns" = Table.CombineColumns(#"Replaced Value1",{"details", "shopify_order_id"},Combiner.CombineTextByDelimiter(",""", QuoteStyle.None),"Merged"),
#"Parsed JSON" = Table.TransformColumns(#"Merged Columns",{},Json.Document)
in
#"Parsed JSON"

Unfortunately no it's not being consistent. That's because the JSON isn't consistent and in some instances the following columns have data. I wish I could just do a merge and roll...but that's not the case here. 

I see. Can you open it as a text file instead of .csv?

 

let
Source = Table.FromColumns({Lines.FromBinary(File.Contents("C:\sample.csv"), null, null, 1252)}),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}})
in
#"Changed Type"

I can give that a whirl but I'm confused, is this truely an issue that is seldom seen? I would suspect that anyone who dumps a DB table into a .csv could run into this. Hmmm....

import of .csv accepts different delimeters, but comma is a default. You can change delimeter to be somethign what's not in the file or nothing (as text). I would not expect .csv export to work in any other way. 

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.