Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn a 50% discount on the DP-600 certification exam by completing the Fabric 30 Days to Learn It challenge.

Reply
GulianiG
Helper II
Helper II

Can Storage Engine answer Simple Queries Independently?

** Queries on Formula & Storage Engine for Power BI Experts **
Excerpt from the 2 articles that led to confusion:

EnterpriseDNA article says:
"Simple queries (SUM, MIN, MAX) can be answered by the Storage engine, otherwise a datacache is sent to Formula Engine for it to compute an answer."
It adds later that "Storage Engine, being faster, handles simple queries and can do it all by itself"

SQLBI article says:
"The queries sent to the storage engine might vary from a simple retrieval of the raw table data to more complex queries aggregating data and joining tables. The storage engine returns data in an uncompressed format, regardless of the original format of the data"

Questions
1) Does Storage Engine (SE) answer simple queries (EnterpriseDNA) or provides raw data (SQLBI) for Formula engine (FE) to compute an answer?

2) If SE is capable to answer few simple requests, is FE then relegating itself when it finds the optimal Query Plan and delegates Storage Engine to compute?

2) For such simple queries, by saying "SE can do it all by itself", is EnterpriseDNA implying that SE does not need FE and can send the answer directly to Summary Table for Power BI visual or are they indicating that SE can process queries using its in-memory data without relying on its compressed data?

I asked the same question on my LinkedIn Profile- would update if I find an answer there

1 ACCEPTED SOLUTION
Fowmy
Super User
Super User

@GulianiG 

Both EnterpriseDNA and SQLBI agree that Storage Engine (SE) deals with simple queries, SE is faster and use multiples cores, it has a limited/basic functions as mentioned but it can filter, join and summarize very well.  EnterpriseDNA says SE can handle them alone, while SQLBI adds that SE deals with different types of queries and gives the data to Formula Engine (FE) for further work. FE can handle sophesticated and complex calculations over the data cache (raw data)  retuned by SE, it's singles threaded (slower).

Yes, when FE figures out the best way to handle a query, it can let Storage Engine (SE) do the actual work because SE is good at it.

 

When EnterpriseDNA says "SE can do it all by itself," it means SE can handle simple queries alone without relying too much on Formula Engine (FE), such as when there are no callbacks. It doesn't skip FE but shows SE can manage simpler tasks independently, whether it's using in-memory or compressed data. SE returned non compressed data into data cache from the vertipacq engine. 


Did I answer your question? Mark my post as a solution! and hit thumbs up


Subscribe and learn Power BI from these videos

Website LinkedIn PBI User Group

View solution in original post

4 REPLIES 4
vanessafvg
Super User
Super User

additionally a good way to understand what the storage engine does and what the formula engine does is it to play with it in dax studio and run queries to get a feel for it, its a lot of information.  Its one of my favourite things to spend hours on 😅

 

effectively the formula engine and storage engine(s) work together, but they each have their own 'jobs'.





If I took the time to answer your question and I came up with a solution, please mark my post as a solution and /or give kudos freely for the effort 🙂 Thank you!

Proud to be a Super User!




Fowmy
Super User
Super User

@GulianiG 

Both EnterpriseDNA and SQLBI agree that Storage Engine (SE) deals with simple queries, SE is faster and use multiples cores, it has a limited/basic functions as mentioned but it can filter, join and summarize very well.  EnterpriseDNA says SE can handle them alone, while SQLBI adds that SE deals with different types of queries and gives the data to Formula Engine (FE) for further work. FE can handle sophesticated and complex calculations over the data cache (raw data)  retuned by SE, it's singles threaded (slower).

Yes, when FE figures out the best way to handle a query, it can let Storage Engine (SE) do the actual work because SE is good at it.

 

When EnterpriseDNA says "SE can do it all by itself," it means SE can handle simple queries alone without relying too much on Formula Engine (FE), such as when there are no callbacks. It doesn't skip FE but shows SE can manage simpler tasks independently, whether it's using in-memory or compressed data. SE returned non compressed data into data cache from the vertipacq engine. 


Did I answer your question? Mark my post as a solution! and hit thumbs up


Subscribe and learn Power BI from these videos

Website LinkedIn PBI User Group

Thanks @Fowmy 
Your answer helps with Q2 & Q3
I'll see if someone else responds to Q1- or else marks yours as an answer

From your overall comment, for Q1, I feel the word "Answer" is an inapt word by EnterpriseDNA, it supplies the answer or datacache back to FE- so I will assume SQLBI is using the right terminology there

@GulianiG 

To answer Q1, The formula engine turns a DAX query into a query plan with steps for execution. Each step corresponds to an operation by the formula engine. Storage Engine (SE) fetches data from the Tabular model and sends it to FE for further calculations. When you asked about "SE answering simple queries,", yes it true, it means SE handles straightforward queries directly for visualization without needing FE for additional calculations. For the second part about "SE providing raw data," it's always true as FE can't access data; SE retrieves and supplies data to FE, regardless of the query complexity.

So, there two statements in the question, both are true.

Did I answer your question? Mark my post as a solution! and hit thumbs up


Subscribe and learn Power BI from these videos

Website LinkedIn PBI User Group

Helpful resources

Announcements
LearnSurvey

Fabric certifications survey

Certification feedback opportunity for the community.

PBI_APRIL_CAROUSEL1

Power BI Monthly Update - April 2024

Check out the April 2024 Power BI update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.