Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
Anonymous
Not applicable

how to determine uncompressed table size in Desktop

I have a data source with a table that is ~15 [GB] in size (uncompressed).  Unfortunately, I need to pull the entire table.  My pbix file somehow has compressed the data down to 150 [MB].  I published my pbix file successfully to the service.  But, when I attempt to refresh the corresponding dataset in the service, I get the following error message:

 

The amount of uncompressed data on the gateway client has exceeded the limit of 10 GB for a single table. Please consider reducing the use of highly repetitive strings values through normalized keys, removing unused columns, or upgrading to Power BI Premium.

 

I understand that I need to limit the result set of my query to <= 10 [GB].  However, that means that, moving forward, I need to determine the size of my result set before I publish, as even though I can successfully publish the compressed size (150 [MB]), the subsequent refresh in the service will fail.  Is there some feature in Power BI Desktop that can show me the uncompressed data size per table?  Or, do I have to determine this myself, outside of the Desktop client?

4 REPLIES 4
Greg_Deckler
Super User
Super User

I am not aware of anything in the desktop that will give you the uncompressed size. 


@ me in replies or I'll lose your thread!!!
Instead of a Kudo, please vote for this idea
Become an expert!: Enterprise DNA
External Tools: MSHGQM
YouTube Channel!: Microsoft Hates Greg
Latest book!:
Mastering Power BI 2nd Edition

DAX is easy, CALCULATE makes DAX hard...

Hi there

It would appear that in your question is your answer.

You said that your table is 15GB, so that is what it needs to import before it can complete the compression. I would suggest splitting the table into 2 in the Power Query Editor and then appending them together




Did I answer your question? Mark my post as a solution!

Proud to be a Super User!







Power BI Blog

Anonymous
Not applicable

I know that I need to do something to resolve the uncompressed data volume obstacle.  However, that is not my issue.  My issue is...  I need / want something in the Desktop client that warns me that the resulting uncompressed table is > 10 [GB].  I need / want this because, except in the very rare scenario that I exceed the 1 [GB] compressed limit, in general, I can publish a pbix file successfully for an uncompressed table that is > 10 [GB], only to be burned later on in the refresh of the dataset.  It would be nice to know that I'm going to have this problem in advance in the Desktop client.  Otherwise, I have to convert the M code to SQL code and run the SQL code in a SQL client to see if I'm exceeding the 10 [GB] uncompressed limit in the result set (table in the pbix file).  Since I work on data warehouse sources, my result sets are generally going to be > 10 [GB].  And, I don't want to purchase Premium to utilize the incremental refresh feature.

Hi there

From your original request you said that it was 150MB compressed?

I would always suggest removing all the columns and data that are not required using TSQL. Best practise is always to push back what you can to the data source.




Did I answer your question? Mark my post as a solution!

Proud to be a Super User!







Power BI Blog

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors