cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
SamRock Regular Visitor
Regular Visitor

Re: Dataflow +Dataset = NEED more Clarity!

Thanks for the twitter link! This was a topic we dicussed in last week's PBI governance call of my company. Glad its being addressed! 🙂

SamRock Regular Visitor
Regular Visitor

Re: Dataflow +Dataset = MAJOR Confusion!

Thank you @RBunting for the link!
I read some of the comments on this idea. Glad to see I am not alone here 🙂
Hope MS takes this up soon

Power81 Visitor
Visitor

Re: Dataflow +Dataset = MAJOR Confusion!

@SamRockSo I've been following this post now for a couple weeks as I believe I'm in the same boat with our current PBI environment. For my understanding, are you, essentially, importing your dataflows into the PBI Desktop? If so, aren't you waiting a long time for things to load? I thought the whole point of Dataflows was to have a "connection" to the cloud where resource utilization would take place on the Azure Data Lake level (scratching my head here)?

 

I thought as a work around I would create a Dataflow dataset (wait a half day for everything to import 🙂 ). This way I have one central dataset the masses could leverage as well as me having one central ETL point.  

 

Any additional clarification would be greatly appreciated! 

SamRock Regular Visitor
Regular Visitor

Re: Dataflow +Dataset = NEED more Clarity!

Here is another shining example of how the way Dataflow fetches data on Desktop forever and ever. Not an ideal way to start development of a report. Why cant it just give us a preview or sample of few 1000s?

 

MAKE ABSOLUTELY NO SENSE to download every data to the desktop!

dataflow-download.PNG

 

Re: Dataflow +Dataset = NEED more Clarity!

I would start in Power BI Service, create a derived Dataflow that filters to the data you need and then connect to that new Dataflow from Power BI Desktop.

 

You could also create a "Sample" Dataflow, with a few hundred records to enable report creation and then switch to the full Dataflow before publishing the report to service.

SamRock Regular Visitor
Regular Visitor

Re: Dataflow +Dataset = NEED more Clarity!

Thanks. But the problem is every-time you switch it starts downloading the data again right?

parishdapbi Frequent Visitor
Frequent Visitor

Re: Dataflow +Dataset = NEED more Clarity!

I'd hope that in Add Data for the dataflow  if you bring it in with the Edit option instead of Load, that you could filter by a date field and that would be used to limit the data?

Highlighted

Re: Dataflow +Dataset = NEED more Clarity!

Correct, @SamRock you'll eventually need to download all the data to Desktop to publish--but at least this eases report creation.

 

I'm pretty sure DirectQuery is coming to Enhanced Dataflows Compute Engine, so maybe there will be options to limit data during report creation too 🙂

 

EDIT: Actually, no--you don't have to download everything to Desktop if you don't remove the filter from the derived Dataflow until after publish.  From @MatthewRoche 's blog

 

"Quick Tip: Developing with large dataflows

  1. Develop and test the dataflows in the Power BI service
  2. Add filters to the too-large dataflow entities’ queries[2] to reduce the number of records in the entities
  3. Refresh the dataflow
  4. Develop the model and reports in Power BI Desktop
  5. Publish the model and reports to the Power BI service
  6. Remove the filters from the filtered dataflow entities
  7. Refresh the dataflow
  8. Refresh the dataset"
SamRock Regular Visitor
Regular Visitor

Re: Dataflow +Dataset = NEED more Clarity!

@jeffshieldsdevThank you so much! This sounds promising! 🙂

Sad we have to beat around bushes to find such solution. Wish MS made it clear from day 1.

Also happy to hear DirectQuery is coming to Dataflows!
However, is the DirectQuery from the data loaded to a Flow?
Or DirectQuery from the source DB via Dataflow?

Re: Dataflow +Dataset = NEED more Clarity!

My understanding is the dataflows we create will have DirectQuery support in Power BI Desktop, via the enhanced compute engine.  It was mentioned at MBAS in June, however I'm not seeing anything in the release notes now, so who knows 😞 https://docs.microsoft.com/en-us/power-platform-release-plan/2019wave2/index

Helpful resources

Announcements
Ask Amanda Anything Q&A

Ask Amanda Anything Q&A

Learn the answers to some of the questions asked during the Amanda Triple A event.

October 2019 Community Highlights

October 2019 Community Highlights

October was a busy month in the community. Read the recap article to learn about some of the events and content.

New Solution Badges

New Solution Badges

Two waves of brand new solution badges are coming! Read the article for more information on our new community badges.

Analytics in Azure virtual event

Analytics in Azure virtual event

Experience a limitless analytics service built to ingest, prep, manage, and serve data for immediate use in Power BI.

Top Kudoed Authors
Users Online
Currently online: 204 members 2,158 guests
Please welcome our newest community members: