Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
alakiran
Regular Visitor

Importing 320 million

Hi,

 

I am trying to import a large data set of 320 million rows into powerbi desktop and it is failing with below error. Any suggestions/ help?

 

Apply query changes

Failed to save modifications to the server. Error returned. There's not enough memory to complete this operation. please try again later when there may be more memory available.

8 REPLIES 8
zoloturu
Memorable Member
Memorable Member

@alakiran,

 

320 mln rows is quite big amount. I have some suggestions but firstly please clarify below:

 

1) How often do you need to run import from Azure SQL? What is the frequency?

2) I guess you not plan to use this file in Power BI Service. Correct?

3) How much is your RAM?

 

Greg_Deckler
Super User
Super User

What is the data source? Sounds like you need more memory or should switch to Direct Query.


@ me in replies or I'll lose your thread!!!
Instead of a Kudo, please vote for this idea
Become an expert!: Enterprise DNA
External Tools: MSHGQM
YouTube Channel!: Microsoft Hates Greg
Latest book!:
Mastering Power BI 2nd Edition

DAX is easy, CALCULATE makes DAX hard...

Azure SQL Server is the data source, Direct Query has a limitation to retrieve only 1 million records.

you need more RAM apparently. Be prepared to the fact that your .pbix-file may exceed the 1 GB size limitation (PBI Premium needed?)

alakiran
Regular Visitor

Hi,

 

I am trying to import 320 million records to PowerBI desktop and after running for 5 hours, it is failing with the error. Any suggestions or help please?

 

Apply Query changes

 

Failed to save modifications to the server. Error returned: There's not enough memory to complete this operation. Please try again later when there may be more memory available

 

Hi @alakiran

 

Remove any columns you don't need.  Do you have any columns that have highly unique columns?

 

Do you have any DateTime data values?

 

What is your data source?


To learn more about DAX visit : aka.ms/practicalDAX

Proud to be a Datanaut!

Yes, I think i selected 6-8 columns out of 30 columns and it is still failing with the error. Yes, There is a datetime column out of those selected columns.

Maybe try to separate your data into buckets with rownum and append the queries in the end?

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.