I bring in dimension tables from a oracle db but the dimension tables contain historic data, how do I filter the dimension tables as I do not want rows in my dimension table that are unrelated to my fact table. I could do inner joins but that gets very expensive as I have a lot of dimension tables for my reporting. Each dimension table has a created on and amended on field should I maybe filter by those fields but then there may be customers who were created or amended a long time ago and i end up filtering them out.
Consider creating surrogate key for your dimensions. Then you have unique key for each value in your dimension table which is much easier to use when creating relationships between your historical dimension and fact table.
Other thing you could do is filtering out the dates where your historical values are invalid using validfrom/validto columns if they are available in your data 🙂
The are keys in the fact and dim tables therefore that is not a problem, lol that would be to easy if there were valid from and valid to columns. There are created on and amended on date fields however. I was thinking about using them but was wondering what if rows have neither been amended or created but are required by the fact table