Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
Andrey28
Regular Visitor

In-between Import and DirectQuery

Hi Everyone,

 

I am still relatively new to Power BI and I am trying to understand specifics related to data import. Currently I am primarily using Import function for my data (files saved at my PC or shared drives), however I am trying to understand better connections to data sources i.e. to Hive. I was poking around with the Direct Query option which looks nice, but based on my understanding it doesn't save data in Power BI model hence when I publish it, nothing is there. I understand that this somehow can be solved by scheduling data refreshes. However here is my question:

Is there an option in Power BI to query certain amount of data from tables, i.e. from Hive and keep it in the Model ? Basically I am dealing with the tables that have millions of records. I don't want to import the whole table but also want to avoid Direct Query. I am searching for some kind of Select From Where SQL-type language that I can add to the query editor that will allow me to keep data in my model. So I am looking for something in between. Can you point me to the right article ?

 

Thanks,

Andrey

1 ACCEPTED SOLUTION
mahoneypat
Employee
Employee

You should be able to stay in Import mode and set up a dynamic query that returns only a subset of date (e.g., last 30 days).  Depending on your source, you may also benefit from query folding where most of the work (filtering, column selection, etc.) is pushed back to the source.

 

If you need all the data but don't want to refresh the whole dataset everytime, you can explore the incremental refresh option (which is now available for pro licenses).

 

https://docs.microsoft.com/en-us/power-bi/admin/service-premium-incremental-refresh

 

Regards,

Pat

 





Did I answer your question? Mark my post as a solution! Kudos are also appreciated!

To learn more about Power BI, follow me on Twitter or subscribe on YouTube.


@mahoneypa HoosierBI on YouTube


View solution in original post

3 REPLIES 3
v-lionel-msft
Community Support
Community Support

Hi @Andrey28 ,

 

You need to transform the data in 'Edit Query', then apply and close.

 

Best regards,
Lionel Chen

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

mahoneypat
Employee
Employee

You should be able to stay in Import mode and set up a dynamic query that returns only a subset of date (e.g., last 30 days).  Depending on your source, you may also benefit from query folding where most of the work (filtering, column selection, etc.) is pushed back to the source.

 

If you need all the data but don't want to refresh the whole dataset everytime, you can explore the incremental refresh option (which is now available for pro licenses).

 

https://docs.microsoft.com/en-us/power-bi/admin/service-premium-incremental-refresh

 

Regards,

Pat

 





Did I answer your question? Mark my post as a solution! Kudos are also appreciated!

To learn more about Power BI, follow me on Twitter or subscribe on YouTube.


@mahoneypa HoosierBI on YouTube


Pat, this helped, thanks

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.