Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
mjohannesson
Advocate I
Advocate I

Conversion from number to string failed due to undersized character buffer

Starting yesterday afternoon, several of our datasets have failed to refresh with the below error. What is the reason for this error? Our datasets imports data from Databricks Spark Delta tables using the Spark connector into our Premium P1 capacity. I have restarted the Databricks cluster once after this error started occurring, but it did not help.

 

There was an error when processing the data in the dataset.
Please try again later or contact support. If you contact support, please provide these details.
Data source error: ERROR [HY000] [Microsoft][Support] (50130) Conversion from number to string failed due to undersized character buffer.. The exception was raised by the IDataReader interface. Please review the error message and provider documentation for further information and corrective action. Table: MyTable.
Cluster URI: WABI-WEST-EUROPE-redirect.analysis.windows.net
1 ACCEPTED SOLUTION

Thanks @saambave! Now, it's working. Microsoft has now posted an awareness message to their support page about this:

https://powerbi.microsoft.com/en-us/support/

View solution in original post

27 REPLIES 27
JonJagd
Helper I
Helper I

I have a couple of Delta Tables in Azure Data Bricks that I access with Power BI via an imprt query. One of these tables, which is identical in schema as two other tables, continously throw the same error as described above in this thread:

 

Failed to save modifications to the server. Error returned: 'OLE DB or ODBC error: [DataSource.Error] ERROR [HY000] [Microsoft][Support] (50130) Conversion from number to string failed due to undersized character buffer.. '.

 

I am running the latest Power Desktop version and runtime version (11.1) on the data bricks cluster. 

@JonJagd we have solved this error by creating views over our Delta Lake tables:

  1. Truncate any Timestamp columns to seconds
  2. Cast numeric columns to Float

Thanks for your input. I have made table(views) in data bricks over tyh delta tables in Delta Lake, so I guess that is what you have also done. This is the schema and some sample data:

JonJagd_0-1661436828387.png

So it is ether time timestamp column or the bigint or floats thatcould be causing the issue?

Most likely the Double columns - I would cast those as Float in your view definition and retry the refresh.

Double-valued columns will have more precision (i.e. more significant digits) that overflow the character buffer when imported via the ODBC driver. Casting to Float reduces the number of digits and should prevent overflow.

Thanks a lot for the input @gah113 . I changed all Double columns to Float, and so far the load has succeeded, so this might very well be the thing that did the trick. My total data volumes are still a bit smaller than before, so hopefully it will continue working, but so far it is looking good.

Anonymous
Not applicable

Hi Guys

I have to load data from Databricks 8.0 (includes Apache Spark 3.1.1, Scala 2.12), I am importing the data using Import and I receive the same error. I specified in the settings that the data types should not be automatically assigned. 

After some records load.  Powerbi Prompts me with the same error. I have tried to find the solution for this but no luck, this is the only forum thread that I could find. is there some sort of solution posted regarding this? 

Anonymous
Not applicable

I'm not an admin on my Databricks implementation, so I don't want to mess around with configurations under advanced options for my cluster if I can avoid it. But this update was just provided on the support page (https://powerbi.microsoft.com/en-us/support/), so I think I'll just hold and wait for this:

 

"Engineers are working on a fix and expect it to be deployed by end-of-day 07/01/2020."

Anonymous
Not applicable

I just experienced this issue. Thanks for the discussion and possible workarounds as well as the message from the Microsoft support page:

 

"Power BI customers using Databricks as a Datasource may encounter the following message during refresh: "Conversion from number to string failed due to undersized character buffer". Engineers have identified the root cause as an incompatibility between Databricks version 6.4 (and above) and connector used by Power BI service. Customers can avoid this issue by using a Databricks version lower than 6.4 until the fix is deployed. ETA for a fix will be provided soon. "

saambave
Regular Visitor

Hi

 

Is there any update on this ?

 

 

Thanks!

I raised a ticket with Microsoft on June 18th, but they haven't yet found a solution.

Thanks @mjohannesson MS recommend to use Databricks Version lower than 6.4. But no luck!

Hi

  We tried the downgrade and it didnt work for us either.

After some investigation we found that refreshes in PBI online through the spark connector (to databricks) fails when the data contains datetime and/or decimal types. The same dataset would refresh against the same cluster in powerbi desktop however.

Within our databricks jobs we converted datetime to unix_time and decimal to float or double and the refresh works in both desktop and online PBI versions.

Hope this helps some of you out there.

Hi

 

It's working fine when we downgraded it to 5.5LTS

Hope this helps !

Thanks @saambave! Now, it's working. Microsoft has now posted an awareness message to their support page about this:

https://powerbi.microsoft.com/en-us/support/

Anonymous
Not applicable

We are currently running Databricks 6.3, which should be fine, and it fails with the same error message. We opened a ticket with Microsoft.

Hi @Anonymous 

 

we tried 6.3 but the issue didn't solve. So we further downgraded to 5.5LTS

 

Thanks!

Anonymous
Not applicable

Same here, tried 6.3 as stated by Microsoft on the portal, but only works on 5.5 LTS (also suggested by Microsoft in response to our case and some of you guys)

 

Thanks

 

 

Anonymous
Not applicable

We were facing same issue.

you can set configuration “spark.thriftserver.arrowBasedRowSet.enabled false” under Advanced options -> Spark tab in any Databricks and restart cluster.

This fixed the issue for us

Anonymous
Not applicable

Yes, unfortunetly same here, since 17th June 2020.

 

Power BI connecting to Databricks using Spark connector trying to reach tables using Delta format.

 

Severity A case opened, almost all of our datasets are not refreshing anymore

 

If anyone found a workarround... would really be apreciated

 

Thanks

Anonymous
Not applicable

Same error since 17th June 2020. In our case when copying a table from Databricks to SQL Database with Data Factory. The pipeline was working fine until the error occurred. I think it might be a Databricks error and not PowerBI.

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors