cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Schizzomarino
Frequent Visitor

Problem using "d3-array".rollup (and it spread argument) in a Power BI Custom Visual

Hello to you all,

I am hoping someone will have a solution for a problem I have encountered using "d3-array" in a custom visual for Power BI.
I have been using d3 v5 as it's got really nice data munching functions that keep my code clean and give good performance; but some of these are now giving me trouble: the rollup function from "d3-array" in particular. Here a description of my situation:


After importing d3 and rollup from d3-array (as rollup), and installing the typescript definitions, when I try to use rollup with two keys, i.e.:
Array.from(rollup(data, ([d]) => d.value, d => +d.date, d => d.name))
I get the following error for the above line: "Expected 3 arguments, but got 4"
If I remove the second or first key the above works, but then I don't get the Map I am looking for and would have liked to keep the above code as it is just so clean...

A small part of the data for clarity:

[{"date":36526,"name":"Coca-Cola","category":"Beverages","value":72537},{"date": 36526,"name": "Microsoft","category": "Technology", "value":70196},...]

 

Any typescript advice on how to get this to work?

 

Notes:
Rollup takes arguments in this form: (values, reduce, ...keys); so that keys can be multiple.

This works perfectly fine in a standard web page with the same d3 and d3-array libraries.

 

Thanks in advance for any advice on the above,

 

Daniel

1 ACCEPTED SOLUTION

Hi @Schizzomarino ,

 

Sorry, but I took a shot before trying it myself. But if I change the key attribute to ...keyany[] I can get your code working.

I do realise that the typing are a bit off now, but looks like TypeScript doesn't work correctly with this type of parameter binding.

But the most important thing: it compiles!

 

-JP

View solution in original post

11 REPLIES 11
Schizzomarino
Frequent Visitor

Hi Daniel @dm-p ,

thanks for the reply. The reasons I don't do the grouping with dataViewMapping:

Reuse: The pbiviz libraries change so much with every new version, I prefer using d3's latest v5 version so I can reuse it in future projets while updating to the newest and best pbiviz library.
I would like to know how to remove limitations that d3V5 encounters in Power BI.
Many objects are needed as to animate the visual: This is only one of the many data munching processes I need to do, including ranking the data and creating before and after snapshots for animation. I fear using dataViewMapping would be a lot more complicated than 4 lines of code in d3.
Finally, I will be using this also on standalone webpage visualisations (to embed directly with data within webpages, with no Power BI) and would much rather use the same processes within the custom visualisation.


The output I am looking for here is an array of arrays, with the deepest arrays made up of a map (the original data:

[{"date":36526,"name":"Coca-Cola","category":"Beverages","value":72537},{"date": 36526,"name": "Microsoft","category": "Technology", "value":70196},...]

Should become, this (consider the date formatting here is of no consequence):
RollupOutput.png

 

Thanks in advance for any guidance you may be able to provide


Dan

Hi Dan,

 

"I would like to know how to remove limitations that d3V5 encounters in Power BI."

 

If you use the 3rd version of tools there cannot be any limitations for external libraries usage.

If you built your porject using the 2nd version of tools, there is an instruction how to migrate.

 

Kind Regards,

 

Evgenii Elkin,
Software Engineer
Microsoft Power BI Custom Visuals
pbicvsupport@microsoft.com

Thanks Evgenii @v-evelk ,

I am using pbiviz 3.1.5 and have imported the rollup function from d3-array as such:

  • import { rollup } from "d3-array"

(I have also installed the types for d3 and it)

  • npm install --save @types/d3
  • npm install --save @types/d3-array

 

And when only one key is used the rollup function works, it looks to me the problem resides in recognising and using the spread operator that is part of the arguments for rollup.

I am thinking it might be a typescript limitation rather than Power BI, but was hoping for a workaround that would allow me to use the data functions within d3 (rather than recoding them in typescript myself)

 

best,

 

Daniel

Hi @Schizzomarino ,

 

It looks to me that it is an issue with the current typedefinitions (@types/d3-array) as the current only accepts one key, see: https://github.com/DefinitelyTyped/DefinitelyTyped/blob/cbcfdd4b12b863a82f5f1903c47fe22fa3b6975c/typ... is already an issue reported, but with not much happening: https://github.com/DefinitelyTyped/DefinitelyTyped/issues/32162

 

This kind of errors are usually related to outdated typedefinitions used by TypeScript and you can easily update the current typedefiniton and add the missing '...'

 

-JP

Thanks @jppp ,

I had not noticed this... Just goes to show how I still need to learn about definitions in typescript

So looking at the type definition I found this

/* 

@param a The array to group. 
@param reduce The reduce function.  
@param key The key function.
*/
export function rollup<TObjectTKeyTReduce>(aIterable<TObject>, reduce: (valueTObject[]) => TReducekey: (valueTObject=> TKey) : Map<TKeyTReduce>;
 
Unfortunately I can't figure where to insert the spread operator here, would someone know how to change it, so the above line from the type definitions file provides the third argument as a spread one (key/TReduce)?
 
Thanks again JP
 
Daniel

Hi @Schizzomarino ,

 

You can adjust your local version in the node_modules/@types/d3-array folder.

And I think you only need to add the spread '...' before the key parameter:

export function rollup<TObjectTKeyTReduce>(aIterable<TObject>, reduce: (valueTObject[]) => TReduce, ...key: (valueTObject=> TKey) : Map<TKeyTReduce>;

 

-JP

Hi @jppp ,

I had tried that amongst a whole pletora of other combinations; but Visual Studio will not compile this... I get the following tooltip:

ProblemWithSpread.pngI have tried so many variations but can't find one that works; and I found this discussion on this:
https://stackoverflow.com/questions/51737041/mapping-generic-spreaded-argument-types-in-typescript

For now I can't quite wrap my head around the technical topic in the above thread... I finsd it rather complicated, probably why it has taken two years, so far, to not have it fixed  :-0

Would anyone know how to get this to work? It would be great after two years to find this solution and update the types mapping for d3-array in the git repository

 

Thanks again @jppp for pointing me in the right direction, but before I set yours as the correct answer I want to see if anyone has code that will work to define spread arguments when exporting types definitions in Typescript

Hi @Schizzomarino ,

 

Sorry, but I took a shot before trying it myself. But if I change the key attribute to ...keyany[] I can get your code working.

I do realise that the typing are a bit off now, but looks like TypeScript doesn't work correctly with this type of parameter binding.

But the most important thing: it compiles!

 

-JP

View solution in original post

Thanks so much @jppp 
That worked!

 

If "any[]" (and "any" for that matter) will enable loose type setting but (seem to) work flawlessly, what is stopping everyone from creating such loose typescript definitions?

dm-p
Super User I
Super User I

Hi @Schizzomarino,

I can help with this, however it's not clear what your expected results are from your post. Can you perhaps share what you would expect your data to look like once processed, or explain how you expect the aggregations to work? I assume you're trying to sum the value and group by name and then by date?

And just to check-off: if this data is coming from the data model to begin with, is there a reason you're not setting up the capabilities and dataViewMapping to do the grouping for you? This way, Power BI will do the computation before the data even hits your visual code and you won't need to do the computation client-side. There will obviously be reasons to do this client-side (for instance, if you're consuming external data sources other than the data model), but just wanted to confirm whether you had considered this as it's a big difference between straight-porting some d3 code into the framework, vs. leveraging the capabilities to get your data in the right shape to begin with.

Either way, let me know what your expected outcome is for the data and I'll do my best to help.

Regards,

Daniel





Did I answer your question? Mark my post as a solution!

Proud to be a Super User!


My course: Introduction to Developing Power BI Visuals


On how to ask a technical question, if you really want an answer (courtesy of SQLBI)




Hi Daniel, @dm-p , 

thanks for the reply. The reasons I don't do the grouping with dataViewMapping:

  • Reuse: The pbiviz libraries change so much with every new version, I prefer using d3's latest v5 version so I can reuse it in future projets while updating to the newest and best pbiviz library.
  • I would like to know how to remove limitations that d3V5 encounters in Power BI.
  • Many objects are needed as to animate the visual: This is only one of the many data munching processes I need to do, including ranking the data and creating before and after snapshots for animation. I fear using dataViewMapping would be a lot more complicated than 4 lines of code in d3.
  • Finally, I will be using this also on standalone webpage visualisations (to embed directly with data within webpages, with no Power BI) and would much rather use the same processes within the custom visualisation.

 

The output I am looking for here is an array of arrays, with the deepest arrays made up of a map (the next step would be that the pairs are iterated over using d3.pairs functionality, e.g.  for ([[kaa], [kbb]] of d3.pairs(datevalues)){...} ), so the original data, this:

[{"date":36526,"name":"Coca-Cola","category":"Beverages","value":72537},{"date": 36526,"name": "Microsoft","category": "Technology", "value":70196},...]

Should become, this:

RollupOutput.png

 

 

 

 

 

Thanks in advance for any guidance you may be able to provide

 

 

Dan

Helpful resources

Announcements
PBI User Groups

Welcome to the User Group Public Preview

Check out new user group experience and if you are a leader please create your group!

MBAS on Demand

Microsoft Business Applications Summit sessions

On-demand access to all the great content presented by the product teams and community members! #MSBizAppsSummit #CommunityRocks

MBAS Attendee Badge

Claim Your Badge & Digital Swag!

Check out how to claim yours today!

Top Solution Authors
Top Kudoed Authors