Reply
Highlighted
Visitor
Posts: 1
Registered: ‎12-08-2016
Accepted Solution

Too many values

Hello!

 

I work for a school district, and we are trying to work on being able to analyze data. We use SQL server extensively, and would love to use Power BI, but it seems very limited compared to Excel + Analysis Services.

 

The biggest issue I have, is that when I try to do just about anything, I seem to always get the message saying that there are “Too many values” and that Power BI isn’t displaying all the data. I can’t even try to demo this to any decision makers, they will see missing data and not trust anything!

 

This seems like there is potential for an awesome tool, but at the moment, we will almost certainly have to go with some other solution.

 

Are there any plans to make this more robust??

 


Accepted Solutions
Frequent Visitor
Posts: 4
Registered: ‎02-06-2017

Re: Too many values

The solution to this problem is to specify a count of values that you want to show in your Visual.

 

this is possible by going into the capabilities and setting the 'count' property of the datareduction algorithm to whatever you want. We tried this with 100.000 and it worked fine, albeit a tad slower than usual.

 

It will affect performance, but otherwise it will work.

View solution in original post

Frequent Visitor
Posts: 9
Registered: ‎02-08-2017

Re: Too many values

@PerfEval: I've applied it to the capabilities.json. See sample below:

 

{
    "dataRoles": [...],
    "dataViewMappings": [
        {
            "conditions": [...],
            "table": {
                "rows": {
                    "select": [...],
                    "dataReductionAlgorithm": { "sample": {"count"100000} }                    
                }
            }
        }
    ],
    "objects":{...}
}

View solution in original post


All Replies
Power BI Team
Posts: 18
Registered: ‎08-25-2015

Re: Too many values

Yes, the limit on data points in our visualizations is something we've gotten a lot of feedback on.  We are currently working on ways to visualize the complete data.

Frequent Visitor
Posts: 9
Registered: ‎02-08-2017

Re: Too many values

Hi,

 

We're also facing this limitation, particulary in custom visuals where we need to plot data with more that 1000 points (wich seems to be the current limitation).

 

We also see that apperently some visuals accept more data than others, for instance the World Visual can plot more Lat/Long points that what we can plot with our custom visual. Is there a way to remove this limit in a custom visual?

 

Could we expect this limitation to be removed in a near future? Any ETA?

 

Thanks,

Nelson

 

Moderator
Posts: 12
Registered: ‎03-23-2016

Re: Too many values

Thanks for the feedback. This is something on our roadmap but I can't give an ETA just yet.

 

In the interim, you have a few options when you run into these limits:

  • Filter the data to a smaller set. 
  • Aggregate the data in your model - you can do this using groups to combine categories, or in the edit queries view, you can create custom columns that aggregate in any way that makes sense for your data set. 
  • You can use the binning or clustering features to automatically group items into meaningful sets. 
  • Once items are grouped together, you can use hierarchies and drill to navigate into different levels of your data and get more detail. 

Even when you are getting this data point warning, we are running a sampling algorithm to provide a representative sample of your data. So while you aren't seeing every single point, the shape of your data set should be represented. 

Frequent Visitor
Posts: 9
Registered: ‎02-08-2017

Re: Too many values

Hi,

 

Thanks for the quick reply.

In our case we cannot filter the data to a smaller set and I also think that there's no way to combine the data into categories.

Our case is the following:

- We want to display several Latitude/Longitude points on a map (in a custom visual).

- We are rounding our Latitude/Longitude points to close to 0.11 meters precision, because this is the level of detail we need.

- With this precision we have a lot of Latitude/Longitude points to show on a map (and this is intentional because we want to display this level of detail)

- We cannot display all the points with this level of precision, so we are considering rounding the Latitude/Longitude to 1.1 meters precision, but this is far from optimal for our use case, and even with this precision we will hit the limitations quickly for a lot of scenarios.

 

We can see that the "Map" and the "ARCGis Maps for PowerBI (priview)" visuals can display all Latitude/Longitude points without saying that there are too many points, but this is not the case in our custom visual. So it seems that there's a way to workaround the limitation we face in the custom visual.

 

 I can send you an email with a picture if it's more easy to understand.

Moderator
Posts: 12
Registered: ‎03-23-2016

Re: Too many values

Hi there,

 

Custom visuals can decide which data reduction algorithm to use based on their needs and request more points. See the documentation here: https://github.com/Microsoft/PowerBI-visuals

 

In addition, you can aggregate your lat / long and create multiple columns, with different levels of granularity. Then use those columns as a hierarchy to drill into the area on the map in which you want more detail. 

Moderator
Posts: 12
Registered: ‎03-23-2016

Re: Too many values

Hi there,

 

Custom visuals can decide which data reduction algorithm to use based on their needs and request more points. See the documentation here: https://github.com/Microsoft/PowerBI-visuals

 

In addition, you can aggregate your lat / long and create multiple columns, with different levels of granularity. Then use those columns as a hierarchy to drill into the area on the map in which you want more detail. 

Frequent Visitor
Posts: 4
Registered: ‎02-06-2017

Re: Too many values

The solution to this problem is to specify a count of values that you want to show in your Visual.

 

this is possible by going into the capabilities and setting the 'count' property of the datareduction algorithm to whatever you want. We tried this with 100.000 and it worked fine, albeit a tad slower than usual.

 

It will affect performance, but otherwise it will work.

Frequent Visitor
Posts: 9
Registered: ‎02-08-2017

Re: Too many values

@Lizardous This really made it work!

Frequent Visitor
Posts: 3
Registered: ‎02-28-2017

Re: Too many values

Could you elaborate on how you did this a little more?  I found several of the capabilities files but am a little unsure of the syntax (dont know javascript).  I found something like this (the count part) in someone's custom visual so I copied this part and pasted into those capabilities files and changed the number:

"dataReductionAlgorithm": {
                        "top": {"count":100000}
                    }

Doesn't seem to do anything though. Do you know where I am going wrong?

 

Before the capabilities file looked like this for reference:

  "dataReductionAlgorithm": {
                        "top": {}
                    }