Can't setup Incremental Refresh for Dataflow to SnowFlake

[Anonymous]

I get an error when I try to setup incremental refresh to a Snowflake view in a Power BI Dataflow. 

Error Message: Model is invalid because it contains duplicate query names. Error: This document contains a duplicate member 'RangeStart'.

When I try to setup the incremental refresh, I am seeing duplicates for every datetime column in the dropdown, but this isn't in the actual data. I wonder if Power BI Service is finding the names in the view and in the table referenced by the view.

[Anonymous]

Unlike pbix data model, you do not need to define RangeStart/RangeEnd parameters in a dataflow to setup incremental refresh. You just need a date column in datetime format. 

In your case, you should just delete the rangestart parameter from your Dataflow. 

Please mark this as resolved if this helps. 

[lbendlin]

Let me rephrase that.  

Unlike a Power BI dataset (where you need to define the RangeStart and RangeEnd parameters in Power Query on the desktop) for dataflows the Power BI service creates these parameters for you when you indicate that you want incremental refresh (and specify the datetime column for it).  Power BI service will also automatically add the filter step to the end of the dataflow M code (which is stupid, in my opinion - it should be possible to specify that filter much earlier in the code).

If you had manually created the parameters and then try to enable incremental refresh for a data flow all hell will break loose. Most of the time you will lose access to the dataflow and will have to delete and redo it.

[juanserm]

I understand the response you are giving (makes total sense), but what I can't figure out is how do I go about limiting the table size for design purposes if I can't add any parameters? When working with datasets that's not a problem because RangeStart and RangEnd are override by the incremental refresh ones, but in this case I have no other choice but load all the data every time I need to work on the dataflow design? (BTW, in my case I'm working on a Datamart, but it behaves pretty much like a dataflow in this sense).

[lbendlin]

you can have parameters. What you cannot have is control over the RangeStart and RangeEnd parameters - these are entirely controlled by the dataflow incremental refresh setup process. As @Anonymous mentioned your responsibility is to provide the datetime (or dateinteger) field that these parameters will be applied to. Not by you, by the dataflow checker.

[juanserm]

Sure, I can have parameters, but if I use them to limit my query result to work on the datamart design, then I have to -at some point- remove all temp parameter limitations so the incremental refresh ones can take full control during refresh, which takes a long time to load. I was hoping there was an alternative way (like they did with datasets), but I guess I'll have to live with that.

Thanks for your response, dough.

[lbendlin]

That's a much deeper question.  As you noticed unlike with datasets there is no way to change parameter values for dataflows in the service.  If this is important to you please consider voting for an existing idea or raising a new one at https://ideas.powerbi.com

In the meantime, set a ridiculously low threshold for incremental refresh (one month/one day)  for testing and then once you confirmed it working you can set the true thresholds.

[V-lianl-msft]

Hi @Anonymous ,

The Table you are drawing a reference to has duplicate values. 

Please try to fix the issue in Relationships by changing the Cardinality to 'Many to One'.

Best Regards,
Liang
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[lbendlin]

This looks more like a duplicate parameter.  Did you define two RangeStart parameters/queries in Power Query?