I'm trying to use a custom connector in a dataflow and it isn't going well.
I'm not a gateway admin just a workspace admin.
I was given access to a data source that was created using a custom connector.
I'm able to successfuly publish datasets and then configure/link them to the data source that was pre-defined for my use.
But if I try to copy the same power query into a dataflow (blank query) then it fails! First it tells me credentials are required. Then when I attempt to enter the credentials it says "only admins of an on-premise data gateway can create connections". Please note that the dataflow editor seems to recognize that a gateway is available to me. And the dataflow editor seems to understand that I'm using a custom connector. But it won't allow me to configure/link the data source. And it won't let me save my work to get to the settings page (where all the gateway configuration is stored).
IMHO, This dataflow editing environment is really poor. It is slow, it gets confused very easily, it has an inadequate subset of the functionality you woud expect. My productivity in this environment is a small fraction of what it is on the desktop.
If anyone has ever used a "custom connector" in a dataflow, please let me know the trick to making this work. Perhaps this development experience is only available for gateway admins? Is that a common requirement for dataflow development? I am somewhat new to the experience.
Solved! Go to Solution.
The biggest problem with these dataflows is that the U/I gives all kinds of confusing and ambiguous cues. The security dialogs are pretty bad too. When you think that you are being prompted for an existing datasource, it turns out that the dataflow is trying to create a brand new connection.
In many cases things don't work unless you type a server name or URL exactly right. So you spend massive amounts of time flailing about, and hoping you are have mis-typed one character in your data source properties. You hope that once you've typed the character properly then things will "just work". Unfortunately that doesn't seem to be the case for custom connectors. They just don't work, period. I need to take a break from dataflows for a while. Maybe after waiting another year or two, they will be less frustrating...
All power query developers cannot be gateway admins. That is a non-starter. The point is that the data source already exists and I shouldn't need to be a gateway admin to use it. I can already configure/link the datasource for a given dataset. But not for a dataflow.
So why is the editing experience (and the security requirements) so different in a dataflow? Why can't I configure/link my dataflow to the same datasource?
Is this considered a bug? It seems like an arbitrary inconsistency. I'm happy to open a support issue, if it would be productive and wouldn't take too much effort to reach a resolution. I get the feeling that dataflows are pretty immature technology and this may be considered a "corner case". Perhaps the expected behavior is undefined. Ideally the dataflow stuff would work in a more well-defined and predictable way.
Opened a support ticket with Microsoft and they confirm that dataflows do NOT support custom connectors. The U/I experience gives all kinds of confusing and ambiguous cues and it wasn't clear either way.
Hope this helps anyone else who has trouble with dataflows. It doesn't make sense for Power BI developers to spend large amounts of time struggling with dataflows, and finding all these inconsistent behaviors as compared to datasets. I wish Microsoft would make it a priority to make Power Query more consistent between datasets and dataflows.
Power Platform release plan for the 2022 release wave 1 describes all new features releasing from April 2022 through September 2022.
Click here to read more about the December 2021 Updates!
Mark your calendars and join us for our next Power BI Dev Camp!