Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn a 50% discount on the DP-600 certification exam by completing the Fabric 30 Days to Learn It challenge.

Reply
nrowey
Helper I
Helper I

DataMart with 3 M rows of data very slow tiring to use Transform

I am using Premium Per User licence, Added a new DataMart in a Premium Workspace with a Postgres connector.

The data has aprox 3 million rows , Caching is turned on but itis very slow to load the Transform function to build joins to a second datamart. Is there any way to make this faster.  Taking aprox 15 to 20 mins to load every time I select transform.

   

4 REPLIES 4
otravers
Community Champion
Community Champion

The patterns I'd go in most cases, depending on whether I want to use Import, are:

 

- Dataflow(s) > Datamart > SQL endpoint > Import (or composite) dataset > reports

- Dataflow(s) > Datamart > auto-generated DQ dataset > reports

 

This is based on my current understanding of datamarts, I reserve the right to change my mind!

------------------------------------------------
1. How to get your question answered quickly - good questions get good answers!
2. Learning how to fish > being spoon-fed without active thinking.
3. Please accept as a solution posts that resolve your questions.
------------------------------------------------
BI Blog: Datamarts | RLS/OLS | Dev Tools | Languages | Aggregations | XMLA/APIs | Field Parameters | Custom Visuals
otravers
Community Champion
Community Champion

In their present state it is completely impractical to do any ETL within the datamart, you want to do all your transformations in upstream dataflows.

------------------------------------------------
1. How to get your question answered quickly - good questions get good answers!
2. Learning how to fish > being spoon-fed without active thinking.
3. Please accept as a solution posts that resolve your questions.
------------------------------------------------
BI Blog: Datamarts | RLS/OLS | Dev Tools | Languages | Aggregations | XMLA/APIs | Field Parameters | Custom Visuals

It sounds like i am using this Datamart incorectly, it automaticly created a Dataset with the same name.

Are you saying I should create a Dataflow and connect to the autocreated dataset, do all the ETL there and point my report to the Dataflow?

I guess I dont get the advantage?

GilbertQ
Super User
Super User

Hi @nrowey 

 

Yeah this is expected to happen because it has to first load all the data before it can do the join. So each time this happens it has to load the data to then find the joins.

 

The only way to do this faster is to do the joins in the PostGres database query?





Did I answer your question? Mark my post as a solution!

Proud to be a Super User!







Power BI Blog

Helpful resources

Announcements
LearnSurvey

Fabric certifications survey

Certification feedback opportunity for the community.

Top Solution Authors
Top Kudoed Authors