Skip to main content
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Grow your Fabric skills and prepare for the DP-600 certification exam by completing the latest Microsoft Fabric challenge.

AmirKatz

Python pandas/R dataframes passed to Python/R script are corrupted if text columns contain newlines

Submitted by Advocate I on ‎01-12-2023 08:06 AM

When the original table has a cell that contains embedded newline characters, the resulting dataframe (available in Python and R scripts) is corrupted:

1. The original field now contains only the data before the first newline

2. New "dummy" rows are added after this "bad" row, each containing the part of the cell data from the previous newline to the next.

3. This part is placed in the first column

4. In these corrupted rows, all other columns, including the original cell, are empty

 

IMO, this is a bug in the Power BI code that converts the table data to a Pandas/R dataframe and it barfs on newline characters.

 

This happens in the desktop client.

Here is a link to my original post in the forums: question 

Status: Investigating

Hi @AmirKatz ,

 

I think I can understand your confusion, but as far as I know, it seems that this issue is not designed to be taken into account when P/Python scripts interact with PowerBI.

If you would like to suggest  feature improvements, you may  vote the idea and comment here  to improve this feature. It is a place for customers provide feedback about Microsoft Office products . What’s more, if a feedback is high voted there by other customers, it will be promising that Microsoft Product Team will take it into consideration when designing the next version in the future.

 

Best Regards,
Community Support Team _ Caitlyn

See more ideas labeled with:
2 Comments (2 New)
Comments
AmirKatz
Advocate I
Advocate I
‎01-12-2023 08:09 AM

I am fully aware that I can sanitize the data in the source (in my case, it's a Salesforce report), but this is not a scalable solution.

v-xiaoyan-msft
Community Support
Community Support
‎01-12-2023 10:17 PM
Status changed to: Investigating

Hi @AmirKatz ,

 

I think I can understand your confusion, but as far as I know, it seems that this issue is not designed to be taken into account when P/Python scripts interact with PowerBI.

If you would like to suggest  feature improvements, you may  vote the idea and comment here  to improve this feature. It is a place for customers provide feedback about Microsoft Office products . What’s more, if a feedback is high voted there by other customers, it will be promising that Microsoft Product Team will take it into consideration when designing the next version in the future.

 

Best Regards,
Community Support Team _ Caitlyn

Idea Statuses