How to calculate an average of sums

[Kashuo]

Hello,

I have multiple buildings. Each building has multiple offices. Each office contains desks.

Each row of my table is an office. Each office has a desk count. The desk counts for each office range from 1 desk to over 200. I have created a set of groups to put segmented ranges of offices into buckets according to their desk count. Here are the buckets:

When I create a bar chart to visualize numbers of offices with various desk counts, I get this:

That chart show that the building in question has 41 ofices with either 1 or 2 desks in them, 76 offices with 3 or 4 desks in them, and so on. Here's a second building:

This building has a different breakdown. It has 93 offices with 1 or 2 desks, 318 offices with 3 or 4 desks, and so on. When I select to include both buildings, it sums up the desk count totals so that I get this:

As you can see, it gives me the total desk count as a sum. This is not what I want, though, I want an average. However, when I change the drop down on the Value from Sum to Average, I don't get 67 and 159 and so on (the average of the total desk count for that room type per building)). I get the average of the individual desk counts themselves, like this:

I've tried using some DAX formulas to try to get the initial SUM per building to stick, and then divide that by number of rows, but it just keep reverting to the individual desk count. It seems like I want to use the "Don't Summarize" option but it's not available.

Any ideas? Thanks!!