Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.
Hi,
We are storing our reporting datasets in Teradata (in the cloud) and then loading them into Power BI, but am finding the load speed is very slow; this isn't too much of a problem for our more aggregated datasets but it is becoming a problem for granular datasets that we would like to work with. As an example, we recently tried to load a transaction (ish) level table of ~150m rows: the load ran for 12 hours, getting to about 120m rows (about 3k rows per second), and then our VPN dropped out. We're finding this speeds up when we load to the service, but not enough for it not to be an issue.
As a comparison, we have been experienting with Amazon Redshift as a data store and the load rates for that on similar data sets is of the order or 100k rows per second. However, Redshift isn't going to be a viable choice for us at the moment, so we're sticking with Teradata.
My question then is, what is the issue with Teradata? Is it that Power BI just doesn't work well with it, or should we be looking at something else like the network?
Any pointers would be greatly appreciated!
Thanks for your response @lbendlin
Interesting. Your load should have conked out at 300 minutes - that is supposedly a hard limit in the Power BI service.
Have you considered using incremental refresh and partition bootstrapping?
Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City
Check out the April 2024 Power BI update to learn about new features.