cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
mark_carlisle Regular Visitor
Regular Visitor

PBI Dataflows Folder data source & custom columns

Appreciate that Dataflows is currently in Preview so not expecting everything immediately. Could I just ask the following;

 

  1. Are custom columns going to be available as an option within the in future. Currently I only seem to be able to create a Conditional Column or a Index Column via the UI and a custom column needs to be created via the Advanced Editor.
  2. After adding the custom columns the data took longer to refresh, increasing from 39 seconds to 104 seconds is this performance hit to be expected when adding custom columns?
  3. Will Folder be getting added as a datasource?

 

This is a possible bug - After setting up Incremental refresh on the dataflow it automatically creates the necessary RangeStart and RangeEnd parameters and adds a step to titled #"AgentByTeamBySkillByDay-54686544617465-autogenerated_for_incremental_refresh". Renaming this step for readability and placing it earlier in the list of steps causes a new auto generated incremental refresh step to be added.

10 REPLIES 10
Ian-H New Member
New Member

Re: PBI Dataflows Folder data source & custom columns

I would also love to see 'Folder' as a data source for Dataflows!

otravers Member
Member

Re: PBI Dataflows Folder data source & custom columns

Microsoft said they intended to bring Dataflows' Power Query to parity with Power BI Desktop.

maya Occasional Visitor
Occasional Visitor

Re: PBI Dataflows Folder data source & custom columns

Hi Mark, the autogenerated step for incremental refresh (in this case #"AgentByTeamBySkillByDay-54686544617465-autogenerated_for_incremental_refresh") should remain as is - without renaming it or placing it earlier in the list of steps.

mmace1 Member
Member

Re: PBI Dataflows Folder data source & custom columns

For the Custom Column - you can also create it in Power BI desktop using the interface, then just copy the Advanced editor code out, and into Dataflows. 

 

To me that seems easier than messing with non-intellisense M language.  

mmace1 Member
Member

Re: PBI Dataflows Folder data source & custom columns

Twice my searches have led me to this thread - similar problems seemingly. 

 

Another vote for adding 'Folder' as a Datasource option.  Is it meaningful to start suggested-votes for these, or no since parity is the goal anyway...? 

mark_carlisle Regular Visitor
Regular Visitor

Re: PBI Dataflows Folder data source & custom columns

@mmace1 Just thought I'd mention that I've done some testing with a folder data source and PBI dataflows and it is possible to get it working.

 

You need to connect to the Folder data source in PBI Desktop and then copy each of the querries, along with the parameter and function from the Advanced Editior in PBI desktop in to a Blank query within dataflows. The only thing I had to do was to remove the Filtered Hidden Files step within the final query

 

Be great if MS add it as a source so the process is automated but this is an acceptable work around. 

AlP Frequent Visitor
Frequent Visitor

Re: PBI Dataflows Folder data source & custom columns

@mark_carlisle  

As I described in the post below, I have an issue where I cannot save a dataflow containing queries/parameters/function to get csv data from a folder - do you know if there is a way to fix it?

 

https://community.powerbi.com/t5/Service/Unable-to-save-a-dataflow-status-400/m-p/643658#M64081

 

Thanks in advance.

 

mark_carlisle Regular Visitor
Regular Visitor

Re: PBI Dataflows Folder data source & custom columns

I've not tested CSV from OneDrive but I've found you can't save Data flows that have duration column types.
AlP Frequent Visitor
Frequent Visitor

Re: PBI Dataflows Folder data source & custom columns

Thank you for the response, I do not have any duration type columns (tried converting all columns to text but didn't resolve the issue)...