Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
spider_Monkey
Helper I
Helper I

Automatically Split One Table into Smaller Tables based on a row value (Name)

Hello, 

 

I am working with a dataset with 15mm+ rows, so needless to say I am running into issues while writing some formulas. I have found if I make the data set smaller the formulas work. 

 

So what I am trying to figure out is if there is a way to automatically split a table into smaller tables based on the name column from the table below. Basically I want to get this information into 4 seperate tables automatically as it refreshes daily. 

 

NameDateStateEventCompany
Charles G1/1/2022 CallABC 
Dave1/1/2022 TextXYZ
Steve S1/1/2022 EmailEnron
Katie1/1/2022 EmailLife
Charles G1/1/2022 CallABC 
Dave1/1/2022 CallXYZ
Charles G1/1/2022 TextABC 
Dave1/1/2022 EmailXYZ
Charles G1/3/2022 EmailABC 
Charles G1/3/2022 CallABC 
Charles G1/1/2022 CallABC 
Katie1/2/2022 TextLife
Katie1/2/2022 EmailLife
Katie1/1/2022 EmailLife
Dave1/1/2022 CallXYZ
Dave1/1/2022 CallXYZ

 

8 REPLIES 8
ppm1
Solution Sage
Solution Sage

While it is likely possible to do what you are asking, you probably shouldn't. Instead, please share the measures you have tried that are not performant and/or further describe your data model. 15M rows is not that much for Power BI but it is not hard to get poor performance if your DAX doesn't follow best practices.

 

Pat

Microsoft Employee

@ppm1  just wanted to see if you had any other thoughts 

Hey Pat,

 

Below is the formula, currently it's just on the table, I was wondering if I did it at the query level that would help.  If it helps when I limit the table to about 16-20k rows the formula works no problem. 

 

Data set is about 15mm rows and 26 columns. 

 

Previous Action = CALCULATE(MAX('Test'[Date]),
(Filter('Test',EARLIER('Test'[ID])='Test'[ID] &&
EARLIER('Test'[Date])>'Test'[Date])

Do you know of a way to write that formula to make a custom column in the query?

Please try this column expression instead.

Previous Action =
VAR thisdate = 'Test'[Date]
RETURN
    CALCULATE (
        MAX ( 'Test'[Date] ),
        ALLEXCEPT ( 'Test', 'Test'[ID] ),
        'Test'[Date] < thisdate
    )

 

Pat

Microsoft Employee

Morning @ppm1 ,

 

No luck it tells me not enough memory to complete the operation. I tried a version of that formula first and got the same error. 

 

Is there anyway to write this in Mquery?

spider_Monkey
Helper I
Helper I

So the problem is I am dealing with 150+ names so I can't manully do the queries. 

FataiSanni
Helper I
Helper I

Hi,

 

You can create 4 different queries with each having its own 'name' filter, so for instance Query 1 is filtered to 'Dave' alone, and you can do this for other queries as well.

 

Alternatively, you can reduce the amount of data you work with in the Power BI desktop that won't affect performance then expand the data to the full scope when you publish to the service. This way, you won't need to split the tables. You can see how this works here.

 

If this post helps, then please consider Accepting it as the solution to help the other members find it more quickly

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors