Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
Anonymous
Not applicable

I need a new approach for data loading

I need to append/merge:

Table1 (static data) set to not refresh

Table2 (static data) set to not refresh

Table3 (dynamic data) set to refresh daily

 

Combine into 1 table for analysis without getting the 10Gig limit of data error. We don't have 10 gigs of data and I don't know why we keep getting the error. I have posted my data results and no one has commetted where the 10 gigs are originating. I'm frustratrated with Power BI data loading capabilities and the data flow. I work for a small company that has 10 million rows of normalized data (integers as keys). We can't afford the Premium subscription. All we need is 10 - 20 Pro licenses. If I can't solve this issue, I will have to recommend Tableau because it will be a better cost solution for our smaller company.

2 REPLIES 2
v-piga-msft
Resident Rockstar
Resident Rockstar

Hi @Anonymous ,

If I understand your scenario correctly that you have 10GB limit error but your dataset is not more than 10 GB?

If it is convenient, could you describe your scenario in more details.

When do you get the error? What is your data source and how do you connect it with?

In addition, please have a check your storage in Power BI Service.

Best  Regards,

Cherry

 

Community Support Team _ Cherry Gao
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Anonymous
Not applicable

Thank you for helping. Any suggestions will be appreciated. I really like the visualizations PBI has, I'm just struggling with the data loading process. We have a gateway setup. I have tried using direct query, but the reports are too slow. I really like the import mode process and this is what I am trying to use. The error is uncompressed data for any one table exceeds 10 gigs. I get the error when I click "Refresh" in PBI online to refresh new data and i'm trying to schedule refresh on the online app.powerbi.com which will give our 10 users access to the reports.

 

If I combine all three fact tables (Year 1, 2, 3 ) into one view that gets loaded, I receive the 10 gig error. If I break up the tables into 3 fact tables using views to load the data, I do not recieve the error, until I try to combine the tables into 1 table using merge or append funcation in PBI. I need the data combined to run the reports over a maximum of the three year period. I am guessing that I receive the error when the append triggers in the refresh.

 

My goal is to try a new approach and i'm looking for help on the new approach and only load/refresh data for Year 3. Y1 and Y2 are static. How can I combine Y3 into a merged Y1 and Y2 to not get the refresh error message.

 

I load my fact table from views to limit the data to 3 years.

 

My total storage online is 500 MB (small).

 

The error I receive - no table can have a limit over 10 gig.

 

Attached is a picture of the my table storages. The table sizes are listed as bytes. If you look at my image, what table is causing my issue? None of them are over 10 gig and the image shows my fact table as combined 3 year data not the individual, FactVwDwEoSTarPBIOrders. My total dataset containing all my tables is over 10 gig, but that isn't what the error states. The error states any one table. And if I just have my three fact tables broken up by year1, 2, 3, I don't receive the error, until I try to combine the tables.

PowerBI04.JPG

PowerBI05.JPG

I like PBI and want to recommend it use for our organization, but we are small and on a limited budget and are looking for a cost effective visualization tool.

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors