Embedded capacity Gen2 Memory Usage

[kelvinngai]

Hi all,

Any one has idea how to track the memory usage of Gen2 embedded capacity? Is it still not available due to (preview) mode? 

One of our report has failed to refresh with below error:

Resource Governing: This operation was canceled because there wasn’t enough memory to finish running it. Either increase the memory of the Premium capacity where this dataset is hosted or reduce the memory footprint of your dataset by doing things like limiting the amount of imported data. More details: consumed memory MB, memory limit MB, database size before command execution MB. Learn more, see https://go.microsoft.com/fwlink/?linkid=2159753.

Obiviously, we have no enough memory, but in the message, it did not show the memory usage

More details: consumed memory ???? MB, memory limit ???? MB, database size before command execution ??? MB.

and I do not have the full picture of the memory usage for our capacity.

In the Azure protal, there is no such metric for Gen2.  Is it something I missed or it is really not available yet? if so, anyone knows the ETA of such metric for Gen2?

Thanks 

Kelvin Ngai

[nicm-cc]

I am dealing with the same issue. Tiny dataset but will not refresh due to resource governing constraints. Upon contacting microsoft support, I was simply told to upgrade my tier but with no way to monitor my workload, it is very difficult to trace the issue here. I have bigger datasets that refresh just fine. 

[datnguye]

Hi @v-henryk-mstf  @kelvinngai 

Do you have any insightful information about this issue now?

I'm facing to the very exact the same issue, and there's no number saying about the memory usage at all. My datasets are also very small <5MB.

I'm trying to install Metrics app to see if any clues. But if we're already having something out, can you help to post here?

Apprecited

[kelvinngai]

Hi @datnguye 

Unfortunately, I dont have more insightful information. We still have the issue that the file size and memory usage is small, but it failed due to insufficient memory, and also no number saying the memory usage at all.

However, we scale up the capacity to next level and it works.  

Cheers,

Kelvin

[LasseMr]

@kelvinngai : I have the exact same problem with 2 particular datasets out of approx 50, running on Premium Embedded Gen 2 Capacity. Started 16th of September.

@v-henryk-mstf 

The capacity util apps are not very informative, and deems rather buggy with misleading information. In our case it shows the accumulated artifact size well below the limits, while 2 datasets still fails with the error message mentioned by @kelvinngai 

I have tried even to upgrade capacity to 2 tiers above normal, tried creating extra capacity and moving the failed datasets workspace to this and perform manual refresh - none of the actions have resolved the issue.

We need better insight on the cause of failures, please offer better resolutions - for now we are leaving our customers hanging with inexplainable reasons why they cannot have their reports up to date.

[LasseMr]

Also, agree with @kelvinngai that it would be useful to have the gen 2 metrics available for the memory usage - we use the CPU metrics a lot to monitor continuously and catching potential problems ongoing. In terms of memory we are in the blind, especially considering that there are no other useful tools at the present.

[v-henryk-mstf]

Hi @kelvinngai ,

For this kind of problems, 

you can select the gear icon Gear icon in the upper-right corner Manage group storage. To check the size of your storage capacity, and the size of the specific dataset imported. When you try to perform an action that will reach one of the limits,  you can manage your storage to reduce your storage amount and get past the limit.

For workspaces in Premium capacity: There is a limit of 100 TB per Premium capacity and there is no per-user storage limit. According to your needs, choose different types of SKUs to manage your resource allocation.

In additional, you can use Monitor Premium capacities with the app to making informed decisions on how best to utilize your Premium capacity resources. 

For more details, you can read related document:

Manage data storage in your workspaces - Power BI | Microsoft Docs

Capacity and SKUs in Power BI embedded analytics - Power BI | Microsoft Docs

Power BI Premium Metrics app - Power BI | Microsoft Docs

If the problem is still not resolved, please provide detailed error information and let me know immediately. Looking forward to your reply.

Best Regards,
Henry

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[kelvinngai]

Hi @v-henryk-mstf ,

Thank you so much for reply.

For my case, it is not the storage issue. The dataset size is very small, only 5MB. 

It is the memory issue. I read a lot documents 

There are still some questions:

1.  Why the error message did not show the memory usage? In the sample error message, it shows the how many memory is consumed, the limt etc...

2. How to keep track the embeded Gen 2 capacity's memory usage? 

There is no such information in metric App and Azure.  Only CPU usage information is available

Thanks 

Kelvin

[v-henryk-mstf]

Hi @kelvinngai ,

I think you can use the Power BI Premium Capacity Metrics app. to manage the health and capacity of your Power BI Premium subscription. There is a key indicator in it is to monitor active memory.

To determine whether there is enough memory to complete its workload, you can refer to the A: Percentage of memory consumption visualization at the top of the page, which represents the memory consumed by the data set being actively processed. Then make a series of adjustments.

For more details, you can read related document:

Power BI Premium Metrics app - Power BI | Microsoft Docs

Best Regards,
Henry

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[kelvinngai]

Hi @v-henryk-mstf 

Thank you for your reply. 

Premium Capacity Metrics app is what I expected it can show, but it did not. 

That is why I asked here someone else have the same issue that the metrics app is not working for Gen2 capacity yet?

The Metric App cannot show any Memory usage for Gen2 capacity. It can only show CPU usage

Thanks 

Kelvin

[Anonymous]

I have the same out of memory errors, and it's weird we cannot monitor memory usage in Premium Capacity Metrics app nor in Azure monitoring... we're blind waiting a long time for Microsoft engineering support - with no solution yet.

[AGeek]

Hi @v-henryk-mstf  @kelvinngai 

Almost a year has passed since the last post. Is the situation still the same? We have the same issue ... Is it possible that Microsoft removed this important metric present in Generation 1?
😭

Thanks for the support
Antimo