Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.
I have built out fairly involved dashboard which is pulling in YTD data from our Snowflake platform. 12 seperate queries some pulling in as many as 18M rows of data. As you can imagine the refresh process takes some time, about 3 hours from the desktop. So I have set up incremental refresh and a private gateway to push this to the service portal. Now it begins the refresh, I can see the jobs running in Snowflake but they time out. Is this due to the inital load being so heavy? Will it get lighter once the initial YTD data is in and I'm just running 10 days incrementally? For my initial load, is there a way for me to break this up into maybe 3 month batches? Need help, our Snowflake platform and PBI don't seem to be playing well together.
Solved! Go to Solution.
Hi @Anonymous ,
“As you can imagine the refresh process takes some time, about 3 hours from the desktop. ”
I think you need to reset your incremental refresh strategy and optimize your data model.
Best regards,
Lionel Chen
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Hi @Anonymous ,
“As you can imagine the refresh process takes some time, about 3 hours from the desktop. ”
I think you need to reset your incremental refresh strategy and optimize your data model.
Best regards,
Lionel Chen
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
@Anonymous the initial full load will take longer, I'm not sure what you mean by 12 different queries, are these different tables or the same on? 18 million is not that many rows. Are you using direct query?
Subscribe to the @PowerBIHowTo YT channel for an upcoming video on List and Record functions in Power Query!!
Learn Power BI and Fabric - subscribe to our YT channel - Click here: @PowerBIHowTo
If my solution proved useful, I'd be delighted to receive Kudos. When you put effort into asking a question, it's equally thoughtful to acknowledge and give Kudos to the individual who helped you solve the problem. It's a small gesture that shows appreciation and encouragement! ❤
Did I answer your question? Mark my post as a solution. Proud to be a Super User! Appreciate your Kudos 🙂
Feel free to email me with any of your BI needs.
Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City
Check out the April 2024 Power BI update to learn about new features.
User | Count |
---|---|
113 | |
97 | |
80 | |
69 | |
59 |
User | Count |
---|---|
150 | |
119 | |
104 | |
87 | |
67 |