Dataflows - Evaluation was canceled

[Anonymous]

Hello all,

I've setup a Dataflow, but I keep hitting the same wall.  Here's a summary:

1) At first, I copied my existing queries from Power BI Desktop advanced editor to a blank query in my new dataflow. All of my simple tables with little ETL (spreadsheets) load fine.  However, one Azure SQL table (900k rows) has appreciable ETL in Power BI's query editor, including merging from the other spreadsheets, and it never fully loads in the Dataflow. I just get an error that says "Evaluation was cancelled". There's no other information at all, or any clues.

2) I decided to recreate all of the applied steps from scratch. I started a new connection, and it loaded the SQL table just fine. The next applied step was to sort the rows.  It took like 15 minutes, but it worked.  On to the next step - add index column.  After another 15 minutes, I get the error "Evaluation was cancelled".

What's going on here and how can I troubleshoot this? I'm going to waste an entire day on this and that's before I even get to the merges...

I get it's in Preview, but am I hitting some limit? I'd atleast like to know if what I'm trying to do is possible.

Thanks

[GilbertQ]

Hi there

Could I suggest limiting the rows to a smaller amount and see if that all works?

That will allow you to see if there is a size limit or a timeout limit some where.
It will also allow you to test and ensure that everything is working as expected.

[Anonymous]

I filtered the rows after import, and before all my ETL, and I didn't get the error this time.

I also asked our admin to increase the max memory for dataflows in our capacity.  He said it was set to 5% and he bumped it to 50% to see what would happen.

I took out the filter step and ran into the error again.  Evalution was cancelled, and refreshing fails.

I think MS needs to work on the error messaging for dataflows, because there's precious little to go on here. I think I'm just hitting a memory limit...  which is kind of crazy to pay so much for premium and my laptop can refresh the dataset better than the dataflow can.

Help me understand something - if I schedule a refresh of the dataflow for after hours, am I more likely to be succesful with less users taking up memory on our capacity?

[Anonymous]

Ok, so the admin jacked up the max memory alottment for Dataflows to 100% for our capacity.

I started the dataflow over from scratch, copying and pasting from the advanced editor in Power BI. I also tweaked one setting...  in the On Premises Gateway, I checked off "Allow user's cloud data sources to refresh through this gateway cluster"  and removed the SQL database from the Gateway. I have no idea if this actually helped or anything, since I was too impatient to try one solution at a time.

After 12 minutes only...  the whole dataflow refreshed successfully.  We're going to incrementally lower the % and see if we start hitting errors again.

[GilbertQ]

Thanks for putting all your details on this.

I would be interested to see if you were able to run the same dataflow on shared capacity?

[Anonymous]

Refresh occurred at 50% memory allotment now. I really can't explain what the issues were, and I can't find any logs.  Unfortunately Im not an admin for our tenant so troubleshooting is limited.

I wasn't aware that Dataflows were available for non-premium workspaces. If I have time, I'll try it and see what my experience is.