Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.
Hi,
I am having a processing issue when the following column calculation is run on a large data set. I had to scale the data set down to 6,000 records to work. Ideally I would like to run this on a data set larger then 600,000+ records. (also my desktop is running 32bit 😞 ).
Do you have any suggestions on a different way to write this function that will reduce the processing power needed?
The function is intended to grab the Latitude value in the record with the most recent date/time closest to the current record and find the difference in order to identify if the current record is a valid data point.
LatAccuracy = if(abs((TBL_CROESystems[LATITUDE] - LOOKUPVALUE(TBL_CROESystems[LATITUDE],TBL_CROESystems[Date/Time],CALCULATE(max(TBL_CROESystems[Date/Time]),filter(all(TBL_CROESystems), TBL_CROESystems[Date/Time] < EARLIER(TBL_CROESystems[Date/Time])))))) >= .110, "OutofBounds", "InBounds")
Thank you in advance
Hey,
not sure if there is a "low" impact algorithm, this is due to the nature of the algorithms that are basically used.
If you are familiar with R scripting you could give it a try, maybe this gets you started
By the way the rgeos package unfortunately is not supported on Power BI Service, for this reason it can just be used locally or in combination with Power BI gateway in personal mode
If you need help, I'm willing, but please be aware that I will request a sample of your dataset, with at least ID (just an integer) | LAT | LONG columns 🙂
Cheers
Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City
Check out the April 2024 Power BI update to learn about new features.
User | Count |
---|---|
113 | |
100 | |
77 | |
74 | |
49 |
User | Count |
---|---|
146 | |
108 | |
106 | |
90 | |
62 |