Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!
Hello,
I am working with a dataset with 15mm+ rows, so needless to say I am running into issues while writing some formulas. I have found if I make the data set smaller the formulas work.
So what I am trying to figure out is if there is a way to automatically split a table into smaller tables based on the name column from the table below. Basically I want to get this information into 4 seperate tables automatically as it refreshes daily.
Name | Date | State | Event | Company |
Charles G | 1/1/2022 | Call | ABC | |
Dave | 1/1/2022 | Text | XYZ | |
Steve S | 1/1/2022 | Enron | ||
Katie | 1/1/2022 | Life | ||
Charles G | 1/1/2022 | Call | ABC | |
Dave | 1/1/2022 | Call | XYZ | |
Charles G | 1/1/2022 | Text | ABC | |
Dave | 1/1/2022 | XYZ | ||
Charles G | 1/3/2022 | ABC | ||
Charles G | 1/3/2022 | Call | ABC | |
Charles G | 1/1/2022 | Call | ABC | |
Katie | 1/2/2022 | Text | Life | |
Katie | 1/2/2022 | Life | ||
Katie | 1/1/2022 | Life | ||
Dave | 1/1/2022 | Call | XYZ | |
Dave | 1/1/2022 | Call | XYZ |
While it is likely possible to do what you are asking, you probably shouldn't. Instead, please share the measures you have tried that are not performant and/or further describe your data model. 15M rows is not that much for Power BI but it is not hard to get poor performance if your DAX doesn't follow best practices.
Pat
Hey Pat,
Below is the formula, currently it's just on the table, I was wondering if I did it at the query level that would help. If it helps when I limit the table to about 16-20k rows the formula works no problem.
Data set is about 15mm rows and 26 columns.
Previous Action = CALCULATE(MAX('Test'[Date]),
(Filter('Test',EARLIER('Test'[ID])='Test'[ID] &&
EARLIER('Test'[Date])>'Test'[Date])
Do you know of a way to write that formula to make a custom column in the query?
Please try this column expression instead.
Previous Action =
VAR thisdate = 'Test'[Date]
RETURN
CALCULATE (
MAX ( 'Test'[Date] ),
ALLEXCEPT ( 'Test', 'Test'[ID] ),
'Test'[Date] < thisdate
)
Pat
Morning @ppm1 ,
No luck it tells me not enough memory to complete the operation. I tried a version of that formula first and got the same error.
Is there anyway to write this in Mquery?
So the problem is I am dealing with 150+ names so I can't manully do the queries.
Hi,
You can create 4 different queries with each having its own 'name' filter, so for instance Query 1 is filtered to 'Dave' alone, and you can do this for other queries as well.
Alternatively, you can reduce the amount of data you work with in the Power BI desktop that won't affect performance then expand the data to the full scope when you publish to the service. This way, you won't need to split the tables. You can see how this works here.
If this post helps, then please consider Accepting it as the solution to help the other members find it more quickly