Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
V55
Helper I
Helper I

Getting error for huge records - Evaluation resulted in a stack overflow and cannot continue

I have a table with lakhs of records. 'Time' is one of the fields. I have created a function fetchlatest to get the latest record based on 'time'. As you can see in the below code, the function basically groups the table based on unique key (GroupByIds) and then create column 'MaxDateTime'. Then filter out the records which has time = MaxDateTime.

 

I validated the function with a table having thousands of records and it worked fine. But when I did scale testing with lakhs of records, I am getting the error 'We encountered an error during evaluation. Errors- Evaluation resulted in a stack overflow and cannot continue'. When I manually tested each step, it happens at the step Table.Group.

 

Note - I used the below power query in Dataflows. I am assuming this error is thrown by power query and not by dataflows.

 

Could you please suggest me why this error is occuring with large records? How can I rectify this error or can you give me any other workaround to get latest using M code ?

 

 

 

let
  FetchLatest = (Table , AllProperties, GroupByIds) =>
let    
    Properties = List.Difference(AllProperties, GroupByIds),
    GroupRowsById = Table.Group( Table, GroupByIds, {{"MaxDateTime", each List.Max([time]), type datetime}, {"All rows", each _, type table}}),
    ExpandGroupedRows = Table.ExpandTableColumn(GroupRowsById , "All rows", Properties, Properties),
    FilterMaxTimeRow = Table.SelectRows(ExpandGroupedRows, each [time] = [MaxDateTime]),
    RemoveExtraColumns = Table.RemoveColumns(FilterMaxTimeRow ,{"MaxDateTime"}),

    Result = 
    if List.IsEmpty(GroupByIds) = true
    then Table
    else RemoveExtraColumns 
in 
    Result
in
  FetchLatest

 

4 REPLIES 4
Denis_Slav
Helper III
Helper III

Do you resolve this problem? I've got the same trouble with dataflow after adding sorting.

v-yuezhe-msft
Employee
Employee

@V55,

Could you please share the sample data of your table so that I can test?


Regards,
Lydia

Community Support Team _ Lydia Zhang
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

I have shared pbix and excel in this link: pbix and excel table

 

Please note that I used Dataflows and the code was written in powerquery online.

 

To get you the original table, this is what I have done:

Fetch data from blob in dataflow, parse and convert to table.

Import the entire table in powerBI desktop by connecting to dataflow.

 

In dataflows, I am getting this stack overflow error and that too only for huge records and not for small table.

 

 

Please let me know how can I get rid of this error in dataflow or is there any better way to fetch latest entries?

 

 

 

@V55,

Thanks for your sharing the files, I would recommend you open a support ticket in Power BI support site

https://powerbi.microsoft.com/en-us/support/ since I am unable to reproduce the issue on my side. Thanks for your understanding.
Support Ticket.gif



Regards,
Lydia

Community Support Team _ Lydia Zhang
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors