Power BI Community Blog articles

Why To Move All Fabric Connections to Service Principal Authentication

InsightsByV — Wed, 21 Oct 2026 13:37:07 GMT

The Problem with Personal Account Dependencies

Many organizations start with data connections created using developer or analyst accounts because it is quick and convenient.

Over time, this creates several challenges:

Report refreshes depend on a specific employee account.
Password changes can break connections.
Employee transfers or resignations create operational risks.
Troubleshooting ownership becomes difficult.

In short, critical business reporting ends up depending on an individual's credentials.

Why Service Principal?

A Service Principal acts as a non-human identity that can authenticate securely against Fabric resources.

Instead of:

Report → User Account → Data Source

The architecture becomes:

Report → Service Principal → Data Source

This means the connection belongs to the organization rather than to a specific user.

No dependency on user accounts
Easier onboarding and offboarding
Better governance and security
Reduced refresh failures caused by credential changes
Centralized ownership of reporting assets
Improved business continuity

Implementation Approach

Step 1: Create a Service Principal

Create a Service Principal through Azure App Registration in Azure Portal.

This will generate:

Client ID
Tenant ID
Client Secret

These credentials would later be used for authentication.

Copy the secret Value, not its Secret ID, and store it in your approved secret store. The value cannot be retrieved after leaving the screen

Please refer this great article for setting up your Service Principal : Link

Step 2: Create a Security Group

Instead of granting permissions directly to the Service Principal, add it to a dedicated security group.

Create a security group in Azure Portal and and add this service principal in this se.

Benefits:

Easier permission management
Better governance
Future scalability

If additional Service Principals are needed later, they can simply be added to the same group.

Step 3: Grant Workspace Access

Add the security group as a member of the required Fabric workspaces.

This will ensure that the Service Principal is having sufficient permissions to access workspace artifacts and related resources.

Step 4: Create New Connections Using the Service Principal

Power BI Admin need to enable below 2 settings in the Admin portal for the security group we created:

Service principals can access read-only admin APIs
Service principals can call Fabric public APIs

Step 5: Create New Connections Using the Service Principal

For each report and semantic model connection:

Create a new Fabric connection.
Select Service Principal authentication.
Provide:

Tenant ID
Client ID
Client Secret (value)

Save and validate the connection.

If configured correctly, the connection should authenticate successfully.

Step 6: Rebind Semantic Models

Existing semantic models can be updated to use the newly created Service Principal connection instead of personal credentials.

This is the step where ownership effectively moves away from individual users.

How To Validate:

Simply saving the connection is not enough, multiple validation checks are required to ensure the configuration success.

Validation 1: Connection Test

The connection was created successfully and authenticated without errors.

Validation 2: Semantic Model Refresh

Trigger a manual refresh using the Service Principal-based connection. The refresh should complete successfully.

Validation 3: Refresh History

Refresh history will show successful execution after the migration. This confirms that the authentication was working end-to-end.

Validation 4: Cross-Workspace Testing

The same approach can be repeated in another workspace and semantic model. The refresh must complete successfully there as well. This eliminats the possibility that the first success was due to a workspace-specific configuration.

Validation 5: User Independence Check

The biggest test:

The connection no longer depended on the creator's account.

The reporting solution could continue functioning even if the original developer was unavailable.

Lessons Learned:

A few observations:

Using a security group makes permission management cleaner than assigning permissions directly.
Test with a non-production semantic model first.
Validate refreshes after every change.
Document Tenant ID, Client ID, and ownership information securely.
Standardize this approach for all future reporting projects to avoid technical debt.

Final Thoughts

Service Principal authentication is one of those improvements that users never notice, but administrators will appreciate for years.

The migration does not change report functionality, visuals, or user experience. What it does change is reliability, governance, and supportability behind the scenes.

If your Fabric environment still relies on personal accounts for report connections, now is a good time to review your architecture. Moving to Service Principal authentication can significantly reduce operational risk and ensure that critical business reports continue running regardless of personnel changes.

The best infrastructure is often the infrastructure nobody notices. After this migration, our reports simply kept working, and that was exactly the outcome we want!

Turn Your Fears into Data: Join Our Halloween Data Visualization Contest!

slindsay — Mon, 12 Oct 2026 19:18:56 GMT

Turn your fears into data! Join our Halloween data visualization contest and transform a community-sourced dataset of fears into a creative, single-page A4 Power BI poster. The contest opens October 12, submissions close October 21, and the winner will be revealed October 31.

Microsoft Fabric Git Integration: New Ways to Compare, Commit, and Branch

techies — Mon, 05 Oct 2026 16:14:56 GMT

Git integration in Microsoft Fabric has come a long way. What started as a way to connect Fabric workspaces with source control has gradually evolved into a more complete development workflow with better synchronization, branching, comparison, and now more granular control over what gets committed.

The September 2026 Fabric updates introduce several preview capabilities that make Git-based development more practical, especially for teams with multiple developers, long-running changes, and shared Fabric workspaces.

In this article, I’ll walk through the Git workflow from the beginning and highlight additions that caught my attention: enhanced compare and conflict resolution, file-level commits, selective branching, branch workspaces, bulk import/export APIs, and the new Branch workspace admin profile.

If you have already worked with Git integration in Power BI and Fabric, the basic workflow will look familiar. For a Power BI report, first save the project as a Power BI Project (.pbip). Once the project is ready, you can connect your Fabric workspace to Git.

Connecting a Fabric Workspace to GitHub

In Microsoft Fabric, create or open the workspace you want to connect.

Go to:

Workspace settings → Git integration

Select GitHub as the Git provider and choose Add account.

Fabric will ask for information such as:

GitHub account/name
Personal access token
Repository URL
Branch

Creating the GitHub access token

For GitHub, the access token can be created from:

GitHub → Settings → Developer settings → Personal access tokens → Fine-grained tokens

When creating the token, select the appropriate repository and configure the required repository permissions.

For the Git integration scenario, the Contents permission needs to allow the required read/write operations.

Copy the generated token for Git integration in Fabric.

Working with a Feature Branch

After connecting Git, create a feature branch instead of changing the main branch directly.

For example:

git checkout -b feature/dashboard

This creates and switches to a new branch called feature/dashboard

You can then connect and synchronize this branch with the Fabric workspace.

After the initial setup, the familiar Git workflow continues.

For example:

git push origin feature/dashboard

After pushing the project, you can view its structure under the relevant branch on GitHub.

At the same time, the connected Fabric workspace contains the corresponding Fabric items, such as the report and semantic model.

For example, suppose I remove the Partner slicer from the report.

The Fabric workspace changes; the Git branch updates its version.

This is where the enhanced compare experience becomes particularly useful.

Instead of syncing and assuming alignment, check the differences between your workspace and the Git branch.

Enhanced Compare and Conflict Resolution

The September update features an enhanced comparison experience.

Before committing changes, updating the workspace, or resolving conflicts, you can inspect what has actually changed between the workspace and the connected Git branch.

The comparison experience provides clearer context around changes, allowing you to understand the differences before taking action.

File-Level Commit — Preview

This is one of the additions I find particularly interesting.

With file-level commit, you can select specific files within a supported Fabric item and commit only those changes.

The remaining changes stay in the workspace until you are ready to commit them.

Selective Branching and Branch Workspaces

Workspace-level branching is another key topic for September updates.

When using Fabric, developers need a suitable workspace to build and test changes. Branch workspaces provide isolated environments linked to specific Git branches, making them ideal for teams working on independent features.

Branch Workspace Admin Profile — Preview

One notable addition is the new Branch workspace admin profile.

This feature addresses an important governance challenge.

Developers can create branch workspaces using the configured profile without needing the same elevated permissions.

The benefits include:

Developers can create branch workspaces without broad workspace-creation permissions.
Capacity assignment can remain centrally governed.
Branch workspaces can follow a consistent configuration.
Development environments remain more isolated.
Administrators retain control over the underlying setup.

This creates an interesting balance between developer flexibility and platform governance.

What About Changes Made Directly in Git?

The workflow also works in the other direction.

Suppose you make a change directly in the project files through VS Code.

For example, you modify the title or another supported definition in the project.

You can then synchronize the change through Git.

A typical Git workflow could look like:

git pull --rebase origin feature/dashboard

followed by:

git push origin feature/dashboard

Changes pushed to GitHub trigger detection in the Fabric workspace.

You can then review the changes and choose to update the workspace.

Bulk Import/Export APIs

Another addition worth watching is the availability of bulk import/export APIs.

Bulk APIs can streamline these processes, particularly when integrated with Git-based development, CI/CD, and deployment automation, aligning Fabric with modern software engineering practices.

The important shift is that Git integration is becoming more than a simple connect → commit → sync experience.

We now have more control over:

What changed → What to commit → What to update → Where to develop → How development is governed

The September 2026 Fabric updates show an interesting evolution in Git integration for Microsoft Fabric.

It combines multiple capabilities to create a more controlled development workflow.

Skills for Fabric in practice: tune them, write your own, close the synonyms gap (Part 2 of 2)

Natarajan_M — Thu, 01 Oct 2026 15:00:00 GMT

How to use Microsoft’s Skills for Fabric day to day, tune them to your team’s standards without forking them, and write your own. Then a full walkthrough of a new skill that adds Q&A and data-agent synonyms to a Power BI semantic model, the one AI-readiness setting that neither the Power BI MCP server nor the official skills will touch. Tested end to end in Power BI Desktop.

Microsoft Fabric Data Agent Governance: From POC to Production

Natarajan_M — Wed, 30 Sep 2026 17:00:00 GMT

In Part 1, we discussed — Who can access what? Click Here to read Part 1

We looked at identity, source permissions, RLS, CLS, OLS, query-path security, and why AI instructions should never become the authorization layer.

But a Data Agent can be perfectly secure and still be poorly governed. Once we move from a POC into production, a different set of questions appears:

Who can modify the agent?
Which identity is used when the agent is called from another application?
How do we test an instruction change before releasing it?
What changes when Code Interpreter is enabled?
Where does conversation history live?
What happens when the Data Agent leaves Fabric?

That is where the conversation moves from security into governance.

Security and Governance Are Not the Same Thing

SECURITY
Who can access the agent and the data?
What rows, columns, or objects can they see?

GOVERNANCE
Who can change the agent?
How are changes tested and quality measured?
Which runtime is used?
Where is conversation data stored?
How is the agent consumed outside Fabric?

Security defines the boundary. Governance controls how the system is operated inside that boundary. Both are required for production.

Fig. Security Vs Governance

Code Interpreter Introduces Another Execution Surface

The normal Data Agent source-query path (SQL, DAX, KQL) is read-oriented, so a prompt like "Delete all customers with zero revenue" doesn't turn the agent into a database administrator. But there is a nuance: Fabric Data Agent can optionally use Code Interpreter, which generates and executes Python in a Microsoft-managed sandbox for statistics, transformations, and charts.

Underlying source
      │ Governed retrieval
      ▼
Fabric Data Agent
      │
      ▼
Sandboxed Python execution
      │
      ▼
Calculated / transformed / visualized result

The source-query path remains governed and read-oriented, while Code Interpreter introduces an additional analytical execution environment.

Rather than treating the word "sandbox" as sufficient evidence, we tested the execution path directly.

What We Observed in Our POC

Governed data was retrieved before Python executed. For "Total Sales by Region", the Run Steps showed the semantic model queried with DAX first. The authorized result was saved as a local JSON file under /mnt/data and then read by pandas. Code Interpreter was not independently querying the model.
The sandbox could create local artifacts. Generated Python wrote files such as CSV and TXT into /mnt/data, and the Run Steps exposed the code used.
Artifacts persisted across chats for the same user. A file created in one chat was still visible in a new chat. We do not infer a retention period from this; the lifetime remains unknown.
A second user did not see the first user's artifact. User B enumerated files and found only their own test file. This is POC evidence consistent with user-level isolation in our environment, not a universal guarantee.
Runtime details were visible, but sensitive introspection was constrained. Python version, OS, and package versions were exposed, but a request for environment-variable names was refused before execution.
Outbound internet was treated as disabled. A requested HTTPS call was not issued and the environment reported internet access as disabled. Since no connection was attempted, this is not packet-level verification of a network block.
A controlled Python error exposed nothing sensitive. Referencing a nonexistent column returned only a KeyError, with no credentials, connection strings, or unrelated file contents.

Fig. Run Step showing the DAX query and generated Python reading the result file

Fig. Same-user artifact persistence across chats

Fig. New chat shows file exists.

Fig. In our test, User B could enumerate their own isolation artifact but not User A’s artifact.

Fig. Area of observations

These are POC observations, not universal guarantees. For production, treat Code Interpreter as its own governed execution surface and ask: What data reaches Python? What code is generated? What artifacts persist, and for how long? Are users isolated? What can the environment reach? What appears in errors?

Read-only source access does not mean there is only one execution surface.

The Calling Identity Becomes Critical Outside Fabric

Inside Fabric, identity is easy to reason about. A published Data Agent can also be called by custom applications, MCP clients, and automation, each with a different identity path. One architecture question should always be documented:

Whose identity ultimately reaches Fabric?

Service principals suit automation, CI/CD, and background processing, but they carry their own permissions. If User A, B, and C all reach the agent through one shared service principal, the platform evaluates that single identity, not each user's entitlements. For interactive apps where per-user entitlements matter, a delegated (on-behalf-of) identity is usually easier to reason about. Neither is universally better:

The identity model must be intentional.

Fig. Calling Identity

Governance Starts Before the Agent Is Published

Who can change the Data Agent is almost as important as who can use it. Creators work in a draft version (schema, instructions, examples, sources); consumers use the published version. A normal consumer shouldn't need edit rights to a definition like "Revenue means Net Revenue excluding intercompany" just to ask "What was revenue last month?" Separate four responsibilities: consume, inspect, edit, publish.

Treat Data Agent Configuration Like Code

Once instructions define business behavior, changing them is a production change. "Change instruction, save, hope nothing broke" is not a governance process. A stronger lifecycle:

Development → Source control → Peer review
→ Test workspace → Regression validation
→ Production → Publish

You should be able to answer who changed an instruction, what changed, which version was tested, and which version is running in production.

Fig. Lifecycle

Standard vs. Preview Runtime Is a Governance Decision

A POC may use Preview capabilities for newer functionality; production may prefer the stable path. But even a standard runtime doesn't mean the underlying AI stack stays frozen. "We tested it once" is not a strategy, so regression testing has to become part of normal operation.

Security Is Not the Same as Accuracy

A response can be perfectly secure and still wrong. For "What did we spend on marketing last quarter?", an authorized user still depends on the agent interpreting "spend", "last quarter", which accounts count as marketing, and which source is authoritative. Production governance needs two questions: Was this user allowed to see the answer? and Was the answer actually correct?

Native Evaluation Should Be Part of the Release Process

"We asked five questions and the answers looked okay" is not a release gate. Maintain expected-answer test cases and run them on every change. A good set includes core KPIs, time-intelligence questions, synonym variations, ambiguous prompts, cross-source questions, restricted-data questions, expected failures, and known edge cases.

When something fails, Run Steps help locate the divergence: wrong source, wrong query, wrong business interpretation, or a misleading explanation. Instead of "the AI got it wrong", ask where did the execution path diverge from what we expected?

Fig. Security vs. Quality Evaluation

Conversation History, Residency, and Networking

Conversation history is part of the data estate. A prompt like "Why is revenue falling for Customer ABC while we negotiate their confidential renewal?" may be more sensitive than the result. Decide what users may enter, how long context persists, and who can investigate interactions.

Data residency needs an explicit decision. Document more than where the capacity lives: where AI processing occurs, where conversation history and telemetry are stored, and where downstream applications process the response.

Networking must be evaluated per source. Private Link (how do we privately reach a resource?) and Outbound Access Protection (which destinations may this workspace call?) solve different problems, and not every source follows the same path. As in Part 1: validate the actual path, not just the existence of a security feature.

Fig. Conversation History, Data Residency, Networking

What Happens When the Data Agent Leaves Fabric?

A Data Agent can be used through Microsoft Foundry, Copilot Studio, Microsoft 365 Copilot, MCP clients, and custom applications. Once the response leaves Fabric, the receiving service brings its own identity model, retention policy, logging, compliance boundary, and geography. "Secure inside Fabric" does not mean "secure everywhere downstream."

The same applies to identity. If the application calls Fabric as the user, the user's permissions apply. If it calls as a shared or maker identity, answers reflect that identity's permissions instead. The second pattern may be intentional, but it must be documented as a security and governance decision.

Fig. When the Data Agent Leaves Fabric

A Production Governance Model

IDENTITY → AUTHORIZATION → DATA SECURITY
→ QUERY-PATH SECURITY → AI SCOPE
→ SEMANTIC GOVERNANCE → EXECUTION GOVERNANCE
→ QUALITY GOVERNANCE → LIFECYCLE GOVERNANCE
→ FABRIC DATA AGENT

Part 1 covered the upper security layers. Part 2 adds the controls needed to operate the agent as a production analytical product.

Fig. Data Agent Ready for Production

Before Production, I Would Review These Areas

Identity: confirm which identity reaches Fabric; document user, delegated, and service-principal paths separately.
Data security and scope: validate RLS/CLS/OLS with real restricted identities on every query path; expose only the sources and fields required.
Execution: document the runtime; review Code Interpreter separately (data passed to Python, artifact persistence, isolation, network limits).
Quality: maintain expected-answer tests; regression-test critical questions.
Lifecycle: separate consumers from creators; use draft and published versions; promote through Dev, Test, and Production; version instructions.
Privacy, residency, network: define prompt rules, document AI processing and telemetry locations, review Private Link per source.
External consumption: re-run the review for every channel and confirm the effective identity for each.

Governance Still Does Not Tell Us What Happened

The agent may now be secure and well governed, but production still raises questions: Who asked what? What did the agent answer? Which source did it choose? Where did execution fail? Can compliance investigate an interaction? These map to three disciplines: Audit (who asked what, and what was answered), Trace (how a request executed), and Observe (whether the agent is healthy in production).

Next: Part 3 — Auditing, Tracing and Observing Microsoft Fabric Data Agents in Production

Part 3 will combine Microsoft Purview, DSPM for AI, Microsoft Foundry, Application Insights, Fabric diagnostics, capacity telemetry, and MCP/application telemetry to answer not just what the Data Agent is allowed to do, but what it actually did. That is where security and governance turn into operational accountability.

Power BI MCP, Fabric MCP and Skills for Fabric in VS Code: setup and limits (Part 1 of 2)

Natarajan_M — Tue, 29 Sep 2026 14:58:25 GMT

A practical setup guide for Power BI and Fabric MCP servers and Microsoft’s Skills for Fabric in VS Code, covering the extension route, the remote servers, all four ways to install the skills (including the ZIP download and the trap that comes with it), and an honest list of what these tools can and can’t do. Checked against Skills for Fabric v0.3.18.

Microsoft Fabric Data Agent Security: What Actually Protects Your Data?

Natarajan_M — Mon, 28 Sep 2026 18:56:28 GMT

Most Microsoft Fabric Data Agent demos begin the same way: connect a semantic model, Lakehouse, or Warehouse, add some instructions, ask a natural-language question, and get an answer.

That's the exciting part. But the moment we move from a demo to a real enterprise implementation, the questions change:

Whose identity is actually querying the data?
Does sharing the Data Agent also share the underlying data?
Can the agent bypass RLS or CLS?
Does Object-Level Security still protect metadata?
What happens if the same data is reachable through multiple query paths?

These are the questions we set out to answer during our Fabric Data Agent POC. The most important principle that came out of it was surprisingly simple:

The Fabric Data Agent should not be your security layer. Your data platform should be.

That one principle makes the rest of the architecture much easier to reason about.

A Data Agent Does Not Get Unlimited Access to Fabric

When someone uses a Fabric Data Agent directly in Fabric, the interaction normally runs in the context of the authenticated Microsoft Entra ID user.

Fabric manages the underlying Azure OpenAI service, so users don't need to supply an OpenAI API key, and the Data Agent doesn't suddenly receive unrestricted access to every table in the workspace. When the agent needs data, it generates the appropriate SQL, DAX, or KQL and queries the underlying source using the permissions available to the calling identity.

Fig. Security Path

The important part is the middle: the agent generates the query, but the data platform decides whether that query can return the requested data. That is exactly where enterprise security should live.

Sharing a Data Agent Does Not Mean Sharing the Data

There are effectively two authorization checks:

Can the user access the Data Agent itself?
Can the user access the underlying data source?

Fig. Two Authorization Checks

Fabric doesn't elevate a user's access just because a Data Agent has been shared with them. Users still need the applicable permissions on the underlying source. Queries that touch data they can't access can fail authorization or return no data, depending on the source and its security model.

This separation is critical for enterprise deployments: you can distribute one common analytical assistant while still preserving different data entitlements for Finance, Sales, Operations, or regional teams.

Semantic Models Have an Interesting Permission Model

Power BI semantic models are especially interesting here. Microsoft documents that a consumer can query a semantic model through a Data Agent with Read permission. They don't need:

Workspace Member
Workspace Contributor
Build permission
Write permission

simply to ask questions through the Data Agent. (Build or Write permissions remain relevant when someone needs to modify the model or use capabilities that change its AI configuration, such as Prep for AI.)

From a least-privilege perspective, this is a very useful improvement. Instead of granting broad workspace or semantic-model permissions just because somebody needs conversational analytics, you can grant only the narrower access required for consumption.

What Happens to RLS and CLS?

This is probably the security question that comes up most often around conversational analytics:

"If the AI generates the query, can it somehow bypass RLS?"

For supported Data Agent interactions, Microsoft explicitly documents that existing user permissions continue to apply, including Row-Level Security (RLS) and Column-Level Security (CLS).

Fig. RLS — One Agent Three Users

All three users can use the same published Data Agent, yet the semantic model can return different populations because the security context belongs to the user.

The agent doesn't need instructions like:

If User = John, only show Europe.

In fact, that would be the wrong way to implement security. The model should enforce the rule. The Data Agent should operate inside it.

What About OLS?

We tested Object-Level Security (OLS) rather than assuming how the Data Agent would behave. In our model, we applied OLS to Customer[Company Name] in the Power BI semantic model and tested the Data Agent using the restricted identity. We then asked:

"Show Total Sales by Company Name."

Fig. Data Agent does not expose the OLS-protected Company Name field.

The Data Agent responded that the model does not contain a Company Name field and suggested Customer Name instead.

Fig. Run Steps show the semantic model was analyzed, but the protected field was unavailable to the agent.

That was the key result. Company Name still physically exists in the semantic model, but for the OLS-restricted user, the Data Agent behaved as though the field didn't exist. In our tested path, OLS protected not only the data but also the object's metadata.

Our test confirmed OLS enforcement for the tested Fabric Data Agent → Power BI semantic model path, including metadata hiding.

There's an important qualification: this remains query-path specific. If the same underlying data is also exposed through a Lakehouse, Warehouse, SQL endpoint, or another source, that route needs its own security controls.

Fig. OLS security

Security Is Specific to the Query Path

This is one of the broader lessons from our POC. Imagine the same business data can be reached through:

Power BI Semantic Model
        │
        └── RLS / CLS / OLS

Lakehouse
        │
        └── OneLake / item security

Warehouse
        │
        └── SQL / OneLake permissions

Securing one route doesn't automatically prove that all the others are equally secure. The semantic model could correctly restrict a user's data through RLS or OLS while the same user has broader access to the underlying Lakehouse. If both are available to the Data Agent, the overall architecture may still be over-permissioned.

So the enterprise question isn't simply "Does my semantic model have RLS?" It's:

Is every query path available to this Data Agent secured for this identity?

Fig. Security Is Specific to the Query Path

AI Instructions Are Not Security Policies

Consider an instruction like:

Never display employee salary information.

That may be useful behavioral guidance, but it is not an authorization boundary. A user could phrase the question differently. The orchestration could change. A future runtime could interpret an instruction differently. The instruction could be accidentally edited or removed.

The secure implementation puts the control in the data platform (RLS/CLS/OLS and source permissions), with instructions layered on top for behavior.

Fig. AI Instructions Are Not Security Policies

The distinction is simple:

Instructions determine how the agent should behave. Permissions determine what the agent is allowed to see.

Never reverse those responsibilities.

Reduce the Agent's Data Surface

Security isn't only about identity — it's also about how much data you expose to the agent in the first place. When configuring a Data Agent, creators select relevant sources, tables, and schema context. For an enterprise Finance Data Agent, exposing:

Dim Date
Dim Account
Dim Cost Center
Fact Actuals
Fact Forecast

is usually better than exposing hundreds of unrelated HR, customer, and operational tables. It helps in two ways: it reduces unnecessary exposure, and it improves query generation because the agent has fewer irrelevant paths to reason through.

But schema selection still isn't authorization.

Fig. Three Boundaries

Data Agent Querying Is Read-Oriented — With an Important Nuance

The normal Data Agent source-query path is designed for analytical retrieval. Its SQL, DAX, and KQL tools retrieve and analyze data rather than modify the underlying source. So a prompt like:

Delete all customers with zero revenue.

doesn't turn the Data Agent into a transactional database administrator. That lowers risk significantly compared with giving an autonomous agent unrestricted write-capable database tools.

But there's a nuance.

Code Interpreter Changes the Execution Surface

Fabric Data Agent can optionally use Code Interpreter (currently in Preview), which generates and executes Python in a Microsoft-managed sandbox for tasks such as statistics, advanced calculations, data transformation, visualization, and analytical processing.

That does not make the underlying source write-enabled. A better mental model:

Underlying Source
       │
       │ Governed read
       ▼
Fabric Data Agent
       │
       ▼
Sandboxed Python
       │
       ▼
Calculated / visualized result

Fig. Code Interpreter Surface

Source querying remains governed and read-oriented, while Code Interpreter adds an additional analytical execution environment.

That extra execution surface should be reviewed deliberately, especially when considering Preview capabilities for sensitive production workloads.

Discussed in Part 2 with detailed analysis. Check Out.

What We Took Away From the Security POC

The cleanest security model for us became:

IDENTITY
Microsoft Entra ID
        ↓
AUTHORIZATION
Fabric item + source permissions
        ↓
DATA SECURITY
RLS / CLS / OLS / OneLake / SQL
        ↓
QUERY-PATH SECURITY
Validate each available source
        ↓
AI SCOPE
Selected schema
        ↓
FABRIC DATA AGENT

The Data Agent sits on top of the security architecture. It should not replace it. That's the difference between building an impressive AI demo and building an enterprise analytical agent.

Fig. Security Architecture

Security Is Only the First Layer

Once the basic security model is in place, a new set of questions appears:

Who can modify the agent?
Which runtime should production use?
How do we test changes?
What happens with Code Interpreter?
How should service principals be handled?
Where is conversation history stored?
What happens when the agent is used from Copilot Studio, Foundry, MCP, or a custom application?

Those are no longer just security questions — they're governance questions.

Next: Part 2 — Governance, From POC to Production

Part 2 focuses on how a production Data Agent should be changed, tested, released, consumed, and governed — including identity models, lifecycle management, evaluation, privacy, data residency, networking, and external consumption: Microsoft Fabric Data Agent Governance: From POC to Production.

The central idea remains the same:

Do not make the AI responsible for protecting the data. Make the data platform responsible for protecting the data, and make the AI operate inside those boundaries.

Click Here to go to Part 2

From Chaos to Clarity: How the Manufacturing Dashboard Helps Everyone Understand the Business

AnalyticPulse — Mon, 28 Sep 2026 14:22:31 GMT

Let's walk through what each page actually shows, and why it matters.

Page 1: Customer Insights — "Who are we selling to, and are they happy?"

Every business lives or dies by its customers, but it's surprisingly hard to keep track of hundreds of relationships in your head. This page acts like a customer relationship "scoreboard."

It shows things like:

* How many active customers the business currently has

* Which customers are considered healthy (buying regularly, paying on time) versus at risk (going quiet, showing warning signs)

* How revenue is spread across different customers and regions — are we relying too heavily on just a few big accounts?

Think of it like a doctor's checkup, but for relationships instead of health. If ten customers suddenly go from "healthy" to "at risk," that's a signal to act before they walk away for good — not after the revenue has already dropped and everyone's asking why.

Page 2: Production & Supply — "Are we making enough, and can we deliver it?"

This is the page for anyone who cares about what's actually happening on the factory floor and getting product out the door. It's less about money and more about operations.

Key questions it answers:

* How much did we produce this month, and is that on target?

* Are shipments going out on time, or are we falling behind on delivery promises?

* Is one particular plant underperforming compared to the others?

If you're a plant manager, this is likely the first thing you check every morning — before coffee, even. It tells you at a glance whether today is a "business as usual" day or a "we need to fix something now" day.

Page 3: Overview — "How's the whole business doing, in 30 seconds?"

This is the page built for someone with almost no time to spare — a CEO, an investor, or anyone who just needs the headline numbers without digging through details.

At the top sit eight simple cards, each showing one important number and whether it went up or down compared to last month:

* Total Revenue — how much money came in

* Total Production — how much product was made

* Capacity Utilization — how much of the factories' potential is actually being used

* Gross Profit — how much money is left after production costs

* Supply Fulfillment — how reliably orders are being delivered

* Inventory Value — how much stock is sitting in the warehouse

* Active Customers — how many customers are currently buying

* Overall Operational Efficiency — a single score summarizing how smoothly everything is running

Below that, charts break revenue and production down by month, by plant, and by region — so if a number drops, you don't just see that it dropped, you can immediately see where. Was it one plant having a bad month, or a slowdown across an entire region?

There's also a simple "Alerts & Insights" section that puts the numbers into plain words — things like "Supply on track: fulfillment is 94% and improving" — so nobody has to guess what a chart is trying to tell them.

Page 4: Inventory & Working Capital — "Do we have too much stock, or too little?"

This page tackles a balancing act every manufacturing business faces. Keep too much inventory sitting around, and you're tying up cash and warehouse space that could be used elsewhere. Keep too little, and you risk running out of product right when a customer needs it — losing sales and trust.

This page shows:

* Total Inventory Value — how much money is currently tied up in stock

* Inventory Turnover — how quickly that stock is being sold and replaced (a higher number generally means things are moving efficiently)

* Days Inventory Outstanding — roughly how many days' worth of stock is sitting around unused

* Raw Material vs. Finished Goods Stock — how much is still waiting to be turned into product versus ready to ship

* Slow-Moving Inventory — stock that isn't selling and may need attention

* Stockout Risk — a warning flag for items at risk of running out

* Inventory Accuracy — how well the recorded stock counts match what's physically in the warehouse

There's even a detailed table listing specific materials by name, plant, and status (like "Slow-Moving" or "Healthy"), so instead of a vague warning, someone gets a precise to-do list of exactly what needs a closer look.

Why This Approach Works for Everyone

The real magic of this dashboard isn't any single chart — it's the consistency. Every page follows the same basic structure:

1. Filters at the top (date range, region, plant, product category) so anyone can narrow the view to exactly what matters to them

2. A handful of key numbers, shown as simple cards with an up or down arrow — no complicated formulas to interpret

3. A plain-language "Alerts & Insights" section that explains, in a sentence or two, what changed and why it matters

That means a machine operator, a plant manager, a customer success rep, and a CEO can all open the same dashboard and immediately find what's relevant to them — no translation needed, no waiting for someone else to "run the numbers."

In a business with as many moving parts as manufacturing, that kind of shared clarity isn't a luxury. It's what keeps everyone — from the shop floor to the boardroom — pointed in the same direction.

How to Use Verified Answers in Power BI Visuals

uzuntasgokberk — Thu, 24 Sep 2026 16:01:06 GMT

Step-by-Step Walkthrough

Start by opening Power BI Desktop. If you prefer to work in the Cloud, you can enable the Large Semantic Model setting to proceed in a similar way. In the Desktop interface, navigate to the section labeled "3.3" (a UI navigation point within the verified answers setup) and select "Verified Answers." This opens a set of optional choices where you can select which visuals should serve as verified answers for specific questions.

If your goal is to generate a specific visual as a complete answer for a particular question, add that visual as a verified answer along with an example question. Once configured, click the Apply button to save your selection.

From there, move on to additional visuals and repeat the same process. For example, you can ask a question related to revenues and configure a trend graph to appear as the answer whenever that question is prompted. You can also use your own custom input when setting up verified answers, giving you flexibility beyond the default options.

In a full demo scenario, three questions were configured — each mapped to a specific visual. When those questions were entered as prompts in Fabric Corporate, the corresponding visuals appeared as output exactly as configured.

What to Take Away

Targeted Responses: Verified answers ensure Power BI returns only the visuals you have specifically approved, making your reports more focused and relevant.
Flexible Setup: You can configure this feature in both Power BI Desktop and the Cloud, with the option to enable the Large Semantic Model on the Cloud side.
Language Support: The feature was tested in both English and Turkish, confirming that verified answers work across languages.

Tools and Concepts Used

Power BI Desktop
Power BI Cloud (with optional Large Semantic Model)
Verified Answers Feature
Prompting and Question-Based Visuals
Language Support (English and Turkish)

Frequently Asked Questions

Q: Can I use verified answers in Power BI Cloud as well as Desktop? A: Yes, verified answers work in both environments. If you are working on the Cloud side, you have the option to enable the Large Semantic Model setting to proceed in a similar way to Desktop.

Q: Does verified answers support languages other than English? A: Yes. The feature was tested with Turkish and produced the expected results, confirming multi-language support.

Conclusion

Verified answers in Power BI offer a practical way to ensure that only the visuals you want are shown in response to specific prompts. Whether you are working in Power BI Desktop or through Microsoft Fabric on the Cloud, this feature helps streamline your reports and improves relevance for your audience. Try configuring verified answers in your next Power BI project to see the benefits firsthand.

If you have questions or comments, feel free to share them below. See you in the next post!

Lakehouse vs. Warehouse in Microsoft Fabric: Which One Should You Actually Use?

AnalyticPulse — Wed, 23 Sep 2026 13:55:56 GMT

If you're new to Microsoft Fabric, you've probably hit this wall already:

you go to create an item, and Fabric hands you two very similar-sounding options - Lakehouse and Warehouse. Both store tabular data in Delta format in OneLake and provide SQL access, but they offer different development and transactional experiences. Both let you query with SQL.Fabric presents both as analytical data-store options, and their capabilities overlap enough that the choice is not always obvious. So which one do you pick?

Short answer: it depends on who's writing the queries and what shape your data is in. Long answer: keep reading — and try the quick self-check below before you scroll to the recommendation.

60-Second Self-Check

Answer these three questions honestly before you create your next Fabric item:

1. Who will write the transformation logic?

- A) Python/Spark notebooks, data engineers comfortable with PySpark

- B) SQL analysts and BI developers who live in T-SQL

2. What does your source data look like?

- A) Mixed — JSON, Parquet, CSV, streaming events, semi-structured

- B) Mostly clean, structured, relational-shaped data

3. What's the end consumption pattern?

- A) A mix of ML, notebooks, ad-hoc exploration, and reporting

- B) Primarily Power BI reports and governed semantic models

Mostly A's → Lean Lakehouse.

Mostly B's → Lean Warehouse.

Mixed bag? → You're not alone — see the "Can I use both?" section below.

Lakehouse: The Flexible One

A Lakehouse stores data as files (Delta Parquet) in OneLake, and gives you two ways to work with it:

- Notebooks (PySpark, Spark SQL) for engineers who want full control

- A SQL analytics endpoint that auto-generates so SQL folks can still query the same tables

Pick Lakehouse when:

- Your data arrives messy, semi-structured, or in large volumes that benefit from Spark's distributed processing

- Your team already thinks in notebooks and data science workflows

- You want schema flexibility - Delta tables support schema enforcement and controlled schema evolution, offering flexibility while preserving data reliability.

- You're building a medallion architecture (bronze → silver → gold) and need engineering muscle at the bronze/silver layers

Watch out for:

- The SQL endpoint is read-only - The SQL analytics endpoint is read-only for Lakehouse table data, so it does not support INSERT, UPDATE, or DELETE against those Delta tables. Modify or load the data through Spark or another supported ingestion and transformation experience. You can still create supported SQL objects such as views, functions, and stored procedures in the endpoint.

- Fabric provides automatic Delta table optimizations, but advanced workloads may still benefit from deliberate file sizing, data layout, partitioning, or optimization strategies. (OPTIMIZE, V-Order, partitioning)

python

#Typical Lakehouse bronze-to-silver pattern in a notebook

df = spark.read.format("json").load("Files/raw/events/")

df_clean = df.dropDuplicates().withColumn("load_date", current_date())

df_clean.write.format("delta").mode("overwrite").saveAsTable("silver_events")

Python

from pyspark.sql.functions import current_date()

df = spark.read.format("json").load("Files/raw/events/")

df_clean = df.dropDuplicates().withColumn("load_date", current_date())
df_clean.write.format("delta").mode("overwrite")

saveAsTable("silver_events")

Warehouse: The Familiar One

Fabric Warehouse provides a rich T-SQL-first data warehousing experience. — think of it as a cloud data warehouse that happens to store data in OneLake under the hood.

Pick Warehouse when:

- Your team's primary skill is T-SQL, not Spark/Python

- You need full DML (`INSERT`, `UPDATE`, `DELETE`, `MERGE`) with transactional guarantees

- You're modeling a governed, relational structure — star schemas, stored procedures, views

- You want a more traditional data-warehouse development experience (cross-database queries, You want a familiar relational warehouse development experience with T-SQL, views, stored procedures, and support for compatible SQL tools.)

Watch out for:

- Less flexible with wildly semi-structured or streaming-first data — you'll typically land that in a Lakehouse first, then move it in

Warehouse is designed primarily for T-SQL rather than Spark-native development. If Spark is central to your transformation logic, Lakehouse is usually the more natural starting point.

sql

-- Typical Warehouse transformation pattern

MERGE INTO dbo.DimCustomer AS target

USING staging.CustomerUpdates AS source

ON target.CustomerID = source.CustomerID

WHEN MATCHED THEN UPDATE SET target.Email = source.Email

WHEN NOT MATCHED THEN INSERT (CustomerID, Email) VALUES (source.CustomerID, source.Email);

Can I Use Both?

Yes - and honestly, A supported and commonly discussed architecture is to use a Lakehouse for engineering-oriented layers and a Warehouse for a curated relational serving layer. do exactly this:

- Lakehouse for bronze/silver ingestion and heavy transformation (Spark does the messy work)

- Warehouse for the polished gold layer that analysts and Power BI consume with familiar T-SQL

Both use OneLake, and Fabric supports patterns such as shortcuts and cross-database queries that can reduce or avoid unnecessary data duplication. If you physically load curated data into separate Warehouse tables, however, that creates another stored representation.

Try It Yourself

Before your next project kickoff, run this checklist with your team:

[ ] Who owns the transformation code - engineers or SQL analysts?

[ ] Does the source data need Spark-level flexibility, or is it already relational?

[ ] Do you need multi-table transactions and T-SQL DML, or can table changes be implemented through Spark and Delta operations?

[ ] Could a hybrid (Lakehouse → Warehouse) actually be the real answer?

No data to test with yet? A quick way to practice both patterns above is to grab a free sample dataset (or a ready-made dashboard layout to reverse-engineer) from a site like Docynx.

Over to you

I'd love to hear how your team decided:

Did you go Lakehouse, Warehouse, or both? What tipped the decision — team skill set, data shape, or something else entirely? Drop your setup in the comments — especially if you've got a "we picked wrong and had to migrate" story, those are always the most useful ones.

Stop Hosting Report Images Publicly Just to Show Them in Power BI

pankajnamekar25 — Mon, 21 Sep 2026 13:56:40 GMT

The problem- showing an image meant making it public

Say you're building a product catalog report, and you want each row in a table to show a small product photo next to its name and price. Power BI has always supported this through an image URL field you give it a web address, and it displays the image. Simple enough, until you think about where that image actually has to live.

For Power BI to load an image from a URL, that URL generally needs to be publicly reachable on the internet. Which means product photos, employee headshots, or internal logos things that often shouldn't be sitting on a public website at all - ended up hosted somewhere publicly accessible anyway, purely so a report could display them. For a genuinely public product catalog, that might be fine. But for an internal HR dashboard showing employee photos, or an internal parts catalog with proprietary product images, publishing those images to a public URL just to satisfy a formatting requirement was never a comfortable trade-off.

The alternative some people used embedding images as base64 text directly in the data - avoided the public hosting problem but came with its own cost =base64-encoded images are large blocks of text, and packing many of them into a semantic model bloats its size and slows things down, especially at any real scale.

What changed

In the August 2026 update, Power BI added support for using image files stored in OneLake directly as an image source - anywhere Power BI already accepts an image URL. This includes the image visual itself, table and matrix cells, the card visual, and button and list slicers.
OneLake is Fabric's underlying storage layer think of it as your organization's own secure file storage that lives inside your Fabric tenant, not on the public internet. The important shift here is that your images now stay inside that same secure boundary as the rest of your data. Nothing has to be exposed publicly, and nothing has to be crammed into your model as bloated encoded text.

A real example
Picture that same product catalog report, but this time for an internal parts catalog your company doesn't want publicly visible - only your own staff should see these product images.

1. Upload your product photos to a Lakehouse in your Fabric workspace, the same way you might upload any other file to OneLake storage.
2. In your semantic model, add a column that holds the OneLake path to each product's image, matched to the right row (similar to how you'd normally store a web image URL).
3. In your table or matrix visual, set that column's data category to Image URL, the same step you'd already take for a normal web-hosted image.
4. Power BI now loads each product's photo directly from OneLake, displayed right in the table - with no image ever needing to be publicly hosted, and no bloated encoded text sitting in your model.

The report looks and behaves exactly the way it would with any other image URL - the only thing that's changed is where the image actually lives, and who can reach it.

Where this helps the most

- Internal catalogs and HR-style reports, where showing a photo matter for usability, but the image itself shouldn't be public
- Organizations with strict data governance requirements, where "the image is on a public URL" was never going to pass an internal security review
- Reports with a lot of images, where base64 embedding was making the model uncomfortably large

What to keep in mind

This requires your images to actually be organized in a Lakehouse within your Fabric workspace first, so it's a bit more setup than just pasting a public URL - worth it specifically when keeping images private and secure actually matters for your use case. For a genuinely public-facing report where the images are meant to be seen by anyone anyway, a normal public image URL still works fine and doesn't need this extra step.

If you've been quietly uncomfortable about hosting internal images publicly just so a report could display them, this is worth switching to the next time you build a report that needs pictures.

Thanks for reading!

Connect with me on:
LinkedIn |
Data With Pankaj - YouTube

Dataflow Gen2 vs Copy Job vs Pipeline: Choosing by Workload, Not by Habit

FarhanJeelani — Wed, 16 Sep 2026 19:35:15 GMT

Every Fabric team has a default tool. People who came from Azure Data Factory build a pipeline for everything. Power BI people open Dataflow Gen2 for everything. Newer teams might put every table into a Copy Job because the wizard is quick.

Each tool does its own job very well. Problems start when it gets used for another tool's job: pipelines full of hand-built watermark logic, dataflows used only to copy tables, and Copy Jobs expected to handle transformations they were never built for.

This post gives you a simple way to pick the right tool based on what the workload needs.

One sentence each

Copy Job moves data from A to B, including incremental loads, with as little setup as possible.

Dataflow Gen2 shapes data. It cleans, merges, reshapes and applies business rules using Power Query.

Pipeline coordinates work. It runs steps in order, handles dependencies and failures, and ties everything together.

If you remember only one thing, remember this: Copy Job moves, Dataflow shapes, Pipeline orchestrates.

Start with three questions about the workload

Ask these before you open any editor.

Am I moving data or changing it?
If the data arrives at the destination looking mostly like the source, with only column mapping or type changes, you are moving data. If you are joining, deduplicating, deriving columns or applying business logic, you are transforming it.
How does the source change?
A one-time or full reload is a different problem from an ongoing incremental sync. Incremental loads based on change data capture (CDC) are different again.
How many steps depend on each other?
A single load on a schedule is one thing. "Load these, then validate, then transform, then run a stored procedure, then alert someone if it fails" is a workflow.

Your answers usually point clearly to one tool.

Copy Job: when the job is data movement

Copy Job is the newest of the three and the one most often overlooked by teams stuck in old habits. Microsoft's decision guide lists its main scenarios as incremental copy and replication (both watermark-based and native CDC), data lake and storage migration, medallion ingestion, and out-of-the-box multi-table copy. Microsoft Learn

The incremental support is the main reason to use it. In a pipeline, the Copy activity handles incremental copy through pipeline expressions and control tables, and only with watermarks. That means you build and maintain the control table, the lookup, the parameterized query and the watermark update yourself. Copy Job does this for you. Microsoft Learn

Choose Copy Job when:

You need to ingest many tables from a database into a lakehouse or warehouse.
You want an initial full load followed by incremental updates, and you don't want to build watermark logic.
The source supports CDC and you want inserts and updates merged into the destination automatically.
The destination is outside Fabric. Copy Job supports 40+ destination connectors. Microsoft Learn

Microsoft's own example fits this well: an analyst who needs multi-table selection across regional SQL Server instances, a bulk initial load, and then CDC-based incremental merges picks Copy Job because it supports both watermark-based and native CDC incremental copying through a wizard, and automatically detects CDC-enabled tables. Microsoft Learn

Don't choose Copy Job when:

You need real transformation. Its transformation support is rated low, and that is intentional. Microsoft Learn
The load is one step in a larger workflow with conditions, retries and downstream dependencies. That belongs in a pipeline.

Dataflow Gen2: when the job is shaping data

Dataflow Gen2 is Power Query running at Fabric scale. It is the right tool when the value lies in the transformation logic. It offers 170+ built-in connectors, 300+ transformation functions in a visual interface, and data profiling tools for checking data quality. Microsoft Learn

A common complaint is that dataflows are slow for large volumes. That used to be a fair criticism, but Fabric has added several performance features aimed at specific workloads:

Fast Copy is for direct, high-throughput copies from a supported source with no transformations. It uses the same backend as the pipeline Copy activity. microsoft microsoft
Modern Evaluator helps when you are shaping data from connectors that don't fold, or only partly fold. microsoft
Partitioned Compute is for large, partitioned or multi-file datasets that can be processed in parallel. microsoft
Staging lets you land raw data first and transform it afterwards (ELT), so ingestion and transformation don't compete in one pass. microsoft

Choose Dataflow Gen2 when:

Business logic is the main work: cleansing, standardizing codes, merging sources, calculated columns.
The people who own the logic know Power Query and would struggle to maintain Spark or SQL.
You are combining files, APIs, SharePoint lists and databases into one clean dataset.
You want visual data profiling while you build.

Don't choose Dataflow Gen2 when:

You are only copying tables. Fast Copy makes this workable, but Copy Job is simpler, gives you incremental loads without extra work, and has less to maintain.
Your destination isn't supported. Dataflow Gen2 lists around 7+ destination connectors, compared with 40+ for the copy tools. Microsoft Learn
The transformations are very complex or code-heavy. At that point a notebook is usually the better choice.

Pipeline: when the job is coordination

A pipeline is not mainly a data movement tool. It is the orchestrator. Microsoft describes it as low-code orchestration that groups several activities together to complete a task. The Copy activity inside it is powerful, and it remains a strong option for very large migrations. The guide describes it as the best low-code choice for moving petabytes of data into lakehouses and warehouses, either ad hoc or on a schedule. Microsoft Learn Microsoft Learn

The real reason to use a pipeline is the control flow: dependencies, branching, retries, failure handling, parameters, and calling other items.

Choose a pipeline when:

Several steps must run in a set order, where step B only runs if step A succeeds.
You need logic such as If/Else, ForEach over a metadata list, or waiting on an external event.
You are combining different item types, such as a Copy Job, a dataflow, a notebook, a stored procedure and a web call.
You need error handling and notifications around the whole process.
You are doing a large, custom migration where you want detailed control over the Copy activity.

Microsoft's scenario for pipelines describes a workflow that runs stored procedures, calls web APIs, moves files and executes other pipelines. That is orchestration, not simple ingestion. Microsoft Learn

Don't choose a pipeline when:

It would contain a single activity that just runs one dataflow on a schedule. Dataflows can be scheduled on their own.
You would be rebuilding incremental loading by hand when Copy Job already supports your source.

Side-by-side

	Copy Job	Dataflow Gen2	Pipeline
Main job	Move and replicate	Transform and shape	Orchestrate
Incremental loads	Built in (watermark and CDC)	Possible, but not its strength	Manual (watermark and control tables)
Transformation	Low	High	None itself (calls other items)
Destinations	40+	~7+	Depends on activities
Authoring	Wizard	Power Query	Visual canvas and expressions
Best owner	Data integrator, analyst	Analyst, data engineer	Data engineer
Warning sign you picked wrong	Adding transformation workarounds	Dataflow has no transformation steps	Pipeline has one activity

Common mistakes that come from habit

The "everything is a pipeline" team. Every source table gets a Lookup, a ForEach, a parameterized Copy activity and a stored procedure to update the watermark. It works, but you now maintain a small custom framework that Copy Job gives you ready-made. Keep the pipeline and let it call the simpler pieces.

The "everything is a dataflow" team. Dataflows with forty queries that just select a table and load it. Move the raw ingestion to Copy Job and keep dataflows for the layer where logic actually happens.

The "Copy Job does it all" team. Trying to handle business rules with column mappings, then adding SQL views downstream to fix what should have been transformed properly. Once logic appears, add a Dataflow Gen2 or a notebook.

The single-activity pipeline. A pipeline whose only purpose is to run one item on a schedule adds a layer to monitor without adding control. Use the item's own schedule until you really need dependencies.

The pattern that usually wins: use all three, each for its own job

For a typical medallion architecture, the tools fit together naturally:

Bronze with Copy Job. Ingest source tables with built-in incremental or CDC loads. No custom watermark logic.
Silver and Gold with Dataflow Gen2 (or notebooks for heavy, code-first logic). Apply cleansing, conformance and business rules where they are visible and easy to maintain.
Pipeline around everything. Run ingestion, then transformation only if ingestion succeeded, then refresh or post-processing steps, and send an alert if anything fails.

Each tool does what it is best at, and each is simpler because it isn't doing another tool's job.

A 30-second decision checklist

No transformation, and data must stay in sync over time? → Copy Job
One-off or very large custom migration needing fine control? → Copy activity in a pipeline
Transformation logic is the main work, and owners know Power Query? → Dataflow Gen2
Complex, code-first transformation at scale? → Notebook
Multiple steps, dependencies, branching or error handling? → Pipeline, calling the tools above

Closing thought

The right question isn't "which tool do we use?" It is "what does this workload need?" Movement, shaping and coordination are three different problems, and Fabric gives you a dedicated tool for each. Teams that choose by workload build less custom plumbing, find problems faster, and hand solutions over more easily, because each piece does one clear job.

Next time you start a new load, answer the three questions first, then pick the tool.

Mouse over effects for buttons

ibarrau — Tue, 15 Sep 2026 18:34:29 GMT

Before we begin, I'd like to clarify a golden rule for creating a great button experience: users should always have a clear indication of which button is currently selected compared to the others. It may sound simple, and many will argue that the Navigator visual available today already solves this problem. However, when we want to go further with custom effects, we will need a bit of retro creativity with our old friends, bookmarks.

NOTE: If you are not familiar with how to easily highlight a selected button by changing its background color, you can review that article first.

Next, we will look at two effects separately, although they could easily be combined.

Underlining Instead of Background Highlighting

When creating buttons, it is natural to focus on their background, both for the selected state and the hover state. However, if we pay attention to the web, we can see that a much lighter visual effect is often achieved with underlined titles.

With a simple underline, I can quickly identify which button is currently selected. I can also use the same effect when the cursor hovers over the other measures.

To build this, we will need two buttons (the active and inactive versions, just as in the previous article) and a shared horizontal line. The inactive button, in its normal state, has a solid background (0% transparency) that exactly matches the page background color, along with gray text. When the mouse hovers over it, the button changes to white text and a 100% transparent background, making the line behind it visible.

When we click the active button, meaning the one that represents the currently displayed state, it remains hidden until the button is clicked. After the click, we will make it visible using bookmarks. This button is positioned exactly on top of the inactive one. While the active button is visible, we will hide the inactive button. The active button will have the same properties in its Default state as the On Hover state of the inactive button, making the horizontal line visible and displaying the text in white. Let's see how the complete setup looks:

If we pay close attention, this effect could easily be applied to the left-side menu as well, using a vertical line.

Motion Spacing

Although Power BI does not provide many visual elements that allow us to experiment with motion effects, in some cases we can simulate them. In this example, we will create the illusion of the text moving slightly to the right when the cursor is placed over the button. To achieve this, we will once again use three elements: two buttons and a vertical line. The difference this time is that instead of simply covering the line as we did before, we will hide and show it. Why make it more complicated this way? Simply because the background has a gradient. In those situations, we cannot have the button in its default state hide the line, because reproducing the exact background effect on the button would be very difficult. Therefore, we will keep the lines hidden until the button is pressed, at which point they will be displayed.

To create this effect, we will set up the inactive button with gray text and no background. The trick will be in its On Hover state. In that state, we will add invisible characters.

What is an invisible character? Invisible text, or invisible characters, refers to special characters that cannot be seen on the screen but are treated like normal letters by the system.

When the mouse hovers over the text, it will begin with invisible characters, creating the effect that the text shifts slightly to the right. Something like this:

Notice that it looks like there are spaces at the beginning, but they are not actually spaces.

Here are the characters if you'd like to copy them:‏‏‎ ‎‏‏‎ ‎‏‏‎ ‎← after the arrow. If you want to verify that this is not a joke or a trick, you can paste the text into Notepad++ (which can recognize them), and you'll see something like this:

The movement effect is now in place. So, how do we continue enhancing the user experience? By following the golden rule. When the user clicks, we will hide the inactive button to reveal the vertical line and the active button, which in this case has its text colored the same as the line.

Let's see the final result:

That's all for this article. I hope it gives you some new ideas for creating user experiences that feel more modern, friendly, and similar to the web applications we interact with every day.

Remember that these two effects can be combined. For example, you could have a vertical navigation menu that not only displays the vertical indicator line for the selected option, but also applies the movement effect when users hover over the items.

Original Post in spanshi

How to Connect SAP to Power BI Through SAP BTP

metrica — Mon, 14 Sep 2026 13:57:05 GMT

Learn how to connect SAP to Power BI through SAP BTP using Metrica Software’s connector. This step-by-step guide covers creating reusable OData data sources, managing access, and loading SAP data into Power BI.

Power BI Smart Table Visual: Excel style Column Filtering, Dynamic Column Headers, Grouping Columns

tharunkumarRTK — Mon, 14 Sep 2026 13:55:10 GMT

For a smoother reading experience with clearer screenshots and easy code copying, I suggest [reading this article on my website](https://www.techietips.co.in/articles/powerbi-managed-6500measures-smartly). It’s free, ad-free, distraction-free, and dedicated solely to sharing knowledge.

The Table is the most commonly used visual in Power BI. Anyone who has spent years in Excel expects to click a header and filter that column. They expect related columns to sit under a common heading. The native table visual does none of this.

There are grid visuals in the Microsoft visual app source that do. Most developers never get to use them, for two reasons that have nothing to do with the features. The good ones are licensed per developers or user, and that cost has to be justified to someone. The free ones are usually not Microsoft certified, which means an uncertified third party script is running inside your report, and that is a fair thing for a security team to say no to.

So I built one. It is called Smart Table, and I built it with Claude. This post walks through what it does.

1. Excel style column filtering

Every column header carries a funnel button in its own divided cell. Click it and you get what you would expect from Excel: sort, a set of conditions appropriate to the column’s type, and a searchable checkbox list of that column’s values. The menu names the column it belongs to, counts what you have ticked, and stays open while you sort.

The conditions follow the column type rather than offering one generic list:

Sync: does the filter stay in the grid or reach the report

Sync is on by default. Filters applied in the header are pushed to the model, so every other visual on the page responds, exactly as if the user had used a slicer. Turn Sync off: Filtering then narrows this grid only.

Global search

The toolbar has a search box with its own operator dropdown: contains, is exactly, starts with, ends with, does not contain. It searches across every column at once.

Power BI: Smart Table Excel Style Column Filtering

2. Dynamic column headers, driven by a measure

A column header name is normally a static label. Here it can be a DAX measure.

Arrival Month is set to April, and the Key Metrics headers read Lead Time (Avg: 30.00), Room Nights (Avg: 1.00), ADR (Avg: 160.50). Change the month and they recompute.

The header honors the filter context, including the filters applied inside the grid itself. Filter Property down to Resort Hotel with the funnel and the averages in the header follow.

Power BI Smart Table: Measure Driven Column Headers (Dynamic Column names) - YouTube

3. Column groups, without any extra tables

Four groups across eleven columns: Reservation, Channel, Room Type, Key Metrics. Each one is a spanning header above its columns with its own colour.

The thing to notice is what is not involved. No disconnected table. No field parameters. No two table visuals stacked on top of each other. No shapes placed in the background: which means your column groups move to the right along with your cursor.

Power BI Smart Table: Grouping columns in a table without using a static table and a matrix visual - YouTube

Known limits

All of these come down to one idea: some filtering reaches the model and some is local to the grid.

30,000 rows That is the Power BI cap for a table mapping with this data reduction algorithm. Header filters push a real model filter, so filtering down to a workable set works fine. Do not point it at an unfiltered fact table.
Value lists cap at 2,000 distinct values per column**, built from the loaded rows. On a high cardinality column like a guest name, use the search box in the menu.
Ends with, Does not end with, and blank selections filter the grid only.** Power BI’s advanced filter operators have `StartsWith` and `Contains` but no `EndsWith`, and a blank has no equivalent in a model side `In` list. The menu tells you when you pick one of these.
Bookmarks restore the data but not the ticked checkboxes.** The model filter is persisted by Power BI, so the rows come back correctly. The header checkboxes just will not show as ticked.

I wrote this to make one point: the gaps in the built-in visuals are not permanent. Build your own, and with vibe coding that is no longer a months-long project.

I am not sharing the .pbiviz file. It is an experiment, not tested for production grade reports. Feel free to checkout my [website](https://www.techietips.co.in) and reach out to me if you are interested to do these experiments along with me,

I would like to continue developing this visual and will write detailed blogs in features explaining its features and implementations.

I hope you learned something new. Feel free to share your thoughts in the comments section.

Happy Learning!!!

My button & slicer disappeared in Power BI Service. Here's why.

Hans-Georg_Puls — Thu, 10 Sep 2026 21:28:14 GMT

Imagine you have created a beautiful table in Power BI with a slicer and a button placed on top of it, overlapping the table area.

The slicer is used to search for specific content. The button becomes clickable when exactly one row is selected and opens a detail page for that row.

In Power BI Desktop, everything works perfectly.

However, as soon as you publish your report to the Power BI service and start interacting with it, strange things may happen.

If you select one row, both the slicer and the button disappear.

If you leave the table area, both reappear.

However, as soon as you hover over them, they disappear again.

Even more strangely, if you move in and out of the table a few times, it may happen that the button is available again.

As it turns out, this is documented behaviour, not a bug. In reading view, Power BI brings the selected visual to the front layer, above anything that overlaps it. Selecting a table row selects the table, so the button and the slicer, which actually sit in front of the table, end up behind it and disappear.

What I can't explain is the button becoming available again after moving in and out a few times. I haven't found anything documented on that.

The good news: this default behaviour can be changed.

To change it, there is an option. It's called 'Maintain layer order' and can be found in a visuals format pane under Properties > Advanced options.

Switch it on for the visual that jumps to the front, in my case the table.

Have you run into this one? And on which visual did you end up switching it on?

Ask vs. Act: Fabric Data Agents and Fabric Operations Agents Explained (with Industry Scenarios)

FarhanJeelani — Thu, 10 Sep 2026 21:26:37 GMT

Feature status (September 2026): Fabric data agents and operations agents are both generally available. Investigator insights in operations agents is still in preview. Always check Microsoft Learn for the latest.

The core difference in one line

A Fabric data agent waits for a question and answers it from governed data. A Fabric operations agent doesn't wait for anyone: it continuously watches live data against business goals and raises its hand, or triggers an action, when a condition is met.

A simple mental model: the data agent is the expert analyst you can message at any time; the operations agent is the control-room operator who never goes off shift.

Fabric data agent: conversational analytics on governed data

The data agent is a configurable Fabric item that lets anyone ask plain-English questions about data in OneLake. Behind the scenes, it decides which source can answer the question, then generates and runs a read-only query: SQL for lakehouses and warehouses, DAX for Power BI semantic models, and KQL for KQL databases (including Eventhouse). It can also work with ontologies and Microsoft Graph.

One data agent can combine up to five sources. Authors improve accuracy by selecting relevant tables, writing agent instructions in business language (for example, "route revenue questions to the Finance semantic model"), and adding example question-and-query pairs, up to 100 per source (examples aren't currently supported for semantic models).

Security is where the data agent is deliberately conservative. Every query runs with the asking user's own credentials, so row-level and column-level security still apply, and Microsoft Purview controls such as DLP and access restriction policies are respected. It never creates, updates, or deletes data, and it doesn't trigger notebooks or other workflows.

Once published, the agent can be consumed well beyond Fabric: in Microsoft 365 Copilot, Teams, Azure AI Foundry, or as a tool inside Microsoft Copilot Studio agents, where the integration is now generally available through the Fabric IQ Data MCP tool.

It's built for conversational insight, not bulk extraction: responses are capped at 25 rows and 25 columns, and unstructured files like PDFs aren't supported.

Fabric operations agent: autonomous monitoring with human-in-the-loop action

The operations agent lives in the Real-Time Intelligence workload. Instead of answering ad-hoc questions, you give it business goals, instructions, a knowledge source (an Eventhouse KQL database or an ontology), and the actions it's allowed to recommend.

From that configuration, the agent generates a playbook: the business entities and properties it will track and the rules that define what "needs attention" looks like. You can build this manually or describe your intent to the built-in Copilot chat ("monitor the turbines and alert me when motor temperature gets too high") and refine the proposed rules iteratively. Each rule is backed by an inspectable query that the agent evaluates every five minutes, using either state conditions (such as is above 80, which keeps signaling while true) or transition conditions (such as crosses above 80, which signals only on the change).

When a rule is met, the agent sends a Teams message, to a person or a channel, through the Fabric Operations Agent Teams app, summarizing what it saw and what it recommends. Beyond messaging, you can configure actions that run a Fabric notebook or trigger a Power Automate flow, which opens the door to ServiceNow tickets, SAP work orders, emails, or any other Power Automate connector. For anomalies, Investigator insights (preview) adds correlated patterns and a likely root-cause summary directly in Teams.

Two governance details matter. First, each operations agent gets its own Microsoft Entra Agent ID, so its activity is auditable separately from human users. Second, it runs in delegated mode with its creator's permissions: when a recipient approves a recommendation, the action executes with the creator's access. Choose agent owners deliberately.

Side-by-side comparison

Dimension	Fabric data agent	Fabric operations agent
Core purpose	Answer questions about data	Monitor data and drive action
Interaction model	Pull: a user asks	Push: the agent notifies
Trigger	A human prompt	A rule condition being met
Data sources	Lakehouse, warehouse, semantic model, KQL database, mirrored database, ontology, Microsoft Graph (up to 5)	Eventhouse KQL database or ontology
Configuration	Tables, instructions, example queries	Goals, instructions, playbook rules, actions
Output	Natural-language answers, small tables, summaries	Teams alerts with recommendations, notebook runs, Power Automate flows
Write / action capability	Strictly read-only	Executes approved actions
Identity	Asking user's credentials (RLS/CLS enforced)	Entra Agent ID using creator's delegated permissions
Time orientation	"What happened, and why?"	"What's happening now, and what should we do?"
Where users meet it	Fabric, Teams, M365 Copilot, Copilot Studio, Foundry	Primarily Teams (alerts, approvals, investigations)

Industry scenarios

1. Manufacturing: from vibration spike to work order

A discrete manufacturer streams CNC machine telemetry (spindle vibration, motor temperature, cycle time) through Eventstream into an Eventhouse.

Operations agent: Its goal is to minimize unplanned downtime. A rule watches for vibration crossing above tolerance on any machine. When it fires, the maintenance supervisors' Teams channel receives an alert with the machine, recent readings, and a recommendation to schedule an inspection. On approval, a Power Automate flow creates a work order in the maintenance system.

Data agent: Preparing for a quarterly review, the plant manager asks: "Which lines had the most unplanned downtime last quarter, and how did OEE compare to target?" The agent queries the production lakehouse and OEE semantic model and returns a ranked answer, with no report-building required.

2. Retail and CPG: protecting a flash promotion

During a weekend promotion, point-of-sale and inventory events land in Eventhouse in near real time.

Operations agent: It monitors store-level inventory for promoted SKUs and flags any item dropping below safety stock. The regional replenishment lead gets a Teams recommendation to trigger an emergency transfer from a nearby store or distribution center, executed through Power Automate after approval.

Data agent: On Monday, the category manager asks: "How did the promotion perform versus the same weekend last year, by region, and which stores ran out of stock?" The agent blends the sales semantic model with lakehouse inventory history to answer in seconds.

3. Energy and utilities: pipeline pressure and grid load

An oil and gas operator ingests SCADA readings (pressure, flow rate, compressor status) from field assets; a utility does the same for substations.

Operations agent: Rules detect pressure exiting its normal operating range at a pumping station, or transformer load crossing a critical threshold during a heatwave. The control room is alerted with context and a recommended response, such as dispatching a field technician via a flow. Human approval stays in the loop. Note that operations agents complement, and never replace, certified safety instrumented systems.

Data agent: An asset integrity engineer asks: "Which ten assets had the most pressure excursions in the last 12 months, and when were they last serviced?"

4. Financial services: payment gateway degradation

A bank streams card authorization events by gateway, merchant category, and channel.

Operations agent: Its goal is protecting payment success rates. When a gateway's decline rate crosses above its threshold, the payments operations team gets an alert with a breakdown and a recommendation to reroute traffic to a secondary processor, an action that runs only after an engineer approves it.

Data agent: A relationship manager asks: "Which of my corporate clients saw failed payments above 2% this month?" Because the agent runs under the manager's own identity, row-level security ensures they only see their own portfolio.

5. Pharma logistics: cold-chain integrity

A logistics provider moving vaccines and biologics collects IoT temperature and GPS data from refrigerated containers.

Operations agent: A transition rule fires the moment a shipment's temperature exits the 2–8°C range. The control tower is notified with location and excursion details, a notebook action flags the batch for quality review, and a flow alerts the receiving site.

Data agent: A quality lead asks: "Which lanes and carriers had the highest excursion rates this year?"

Better together: detect, decide, explain

The strongest architectures use both agents on the same data foundation. Picture an airport operations team. An operations agent watches baggage-system events in Eventhouse and alerts the duty manager when belt throughput drops below target during a peak arrival wave. The manager approves the recommended action: rerouting bags to an alternate belt via Power Automate.

Then, without leaving Teams, the manager asks a Copilot Studio agent grounded by a Fabric data agent over the same Eventhouse and historical lakehouse: "How often has this belt degraded at this time of day over the past 90 days?"

The operations agent handles detection and response. The data agent handles explanation and learning. Because both point at the same governed data in OneLake, the alert and the analysis never disagree about the numbers.

How to choose

Choose a data agent when the need is exploratory and question-driven: executive self-service, analyst acceleration, or bringing trusted data into Copilot experiences. Choose an operations agent when you have a specific, repeatable business process with measurable conditions, where minutes matter and someone should be told (or something should happen) without anyone having to ask.

A quick test: if you hear "someone should check this dashboard every hour," you need an operations agent. If you hear "I just need to know X before Friday's meeting," you need a data agent.

Practical tips before you build

For data agents: curate tightly by selecting only the tables users need, write instructions in business terms, and invest in example queries, which are often the biggest accuracy lever. Use Git integration and deployment pipelines to promote agents from development to production.

For operations agents: scope one agent to one business process, flatten nested JSON columns in Eventhouse tables before configuring, and review each rule's generated query before starting the agent. Remember that approvals execute with the creator's permissions, trial capacities aren't supported, and agents consume capacity units, so track them in the Fabric Capacity Metrics app.

For both: confirm your tenant admin has enabled the required Copilot and Azure OpenAI settings, including cross-geo AI processing where your capacity region requires it.

Wrapping up

Fabric data agents and operations agents are two sides of the same coin. The data agent democratizes understanding: anyone can ask a question and get a governed answer. The operations agent operationalizes response: the business stops depending on people watching dashboards and starts reacting to events as they happen.

Start with the scenario that hurts most today, build on a shared OneLake foundation, and grow from there.

How are you using these agents in your industry? Share your scenarios in the comments!

You Can Create Calculated DAX Columns in Direct Lake — But Should You?

amitchandak — Wed, 09 Sep 2026 14:03:44 GMT

Adding calculated columns in DAX for Direct Lake gives a lot of flexibility and feels like an easy option... BUT.....

Building and Enriching a Microsoft Fabric Data Agent on a Power BI Semantic Model

Praful_Potphode — Tue, 08 Sep 2026 13:46:21 GMT

Microsoft Fabric’s Data Agents are one of the most exciting additions to the Fabric ecosystem for Power BI practitioners. They let business users ask questions in plain English and receive grounded answers directly from a governed semantic model, without needing to write DAX or SQL.

In this post, I’ll explain why you might place a Data Agent on top of a Power BI semantic model, how to set one up, and, most importantly, how to enrich it so it provides high-quality, trustworthy answers.

The big idea: Business users ask questions in plain English. The agent reasons over a curated semantic layer and returns grounded, governed answers rather than invented results.

1. What Are Data Agents, and Why Use Them?

Data Agents let people ask plain-English questions about their data and receive context-rich answers. Under the hood, they use Azure OpenAI models to interpret the question, determine which data source is most relevant, and generate an appropriate query: SQL for a warehouse or lakehouse, DAX for a semantic model, and KQL for event data.

The result is presented as a readable response containing tables, summaries, or insights while respecting the user’s existing security permissions. Because Data Agents connect directly to governed data in OneLake, they make analytics accessible to users who may never open Power BI Desktop or write a query.

1.1 What Is the Difference Between Copilot and a Data Agent?

Both Fabric Copilot and Fabric Data Agents use generative AI, but they are designed for different scenarios.

Copilot is embedded within the current Fabric experience, such as a Power BI report, and helps users complete context-specific tasks such as creating visuals or writing DAX. A Data Agent is a standalone conversational analytics experience that can connect to as many as five data sources and can be integrated with experiences such as Microsoft Teams, Microsoft 365 Copilot, Copilot Studio, and custom applications.

Data Agents support custom instructions, business terminology, verified answers, and reusable integrations. They are read-only and honor existing security. This makes them suitable for domain-specific, governed self-service analytics, while Copilot remains useful for assistance within the current Fabric item.

2. Prerequisites and Setup

To create a Data Agent, you need Fabric capacity and at least one supported data source containing data, such as a lakehouse, warehouse, Power BI semantic model, KQL database, mirrored database, or ontology.

You should also have a well-designed Power BI semantic model with governed measures, meaningful relationships, clear naming, and appropriate security. Cross-geo processing must be enabled when your data is stored in a different region. Read permission on the semantic model is sufficient; Build permission is not required.

3. Preparing the Semantic Model: The Most Important Step

Preparing the semantic model is the foundation of a reliable Data Agent. Before adding the model to the agent, configure it using Power BI’s Prep for AI capabilities.

Use business-friendly names. Give tables, columns, and measures clear names. Hide technical keys, housekeeping fields, and other objects that users should not query.
Organize the model. Group related columns and measures into display folders, such as Customer, Sales, Profit, and Time Intelligence.
Add descriptions and synonyms. Explain what each important table, column, and measure represents, and include the terminology users are likely to use.
Validate relationships. Ensure the model has the correct active relationships, filter direction, and date behavior.
Apply security. Configure RLS and OLS where required. The Data Agent respects the semantic model’s security.
Configure Prep for AI. Define the AI data schema, semantic-model instructions, and verified answers. When a Data Agent queries a semantic model, its DAX-generation tool relies on the model metadata and Prep for AI configuration. Data Agent-level instructions are not used to generate DAX, so model-specific calculation and filtering rules must live in the semantic model.

4. Creating and Configuring the Data Agent

In your Fabric workspace, select New item, choose Fabric data agent, give the agent a descriptive name, and create it. The OneLake catalog will appear, allowing you to add supported data sources. Select the Power BI semantic model and any other required sources, then choose only the tables the agent needs for the intended business questions.

4.1 Configuring the Data Agent

Creating the agent is only the starting point. To improve accuracy and keep the experience useful over time, treat it as an evolving system and use an iterative process of testing and refinement.

1. Add Data Agent instructions. These instructions govern orchestration and presentation. Use them to define source-routing priorities, response structure, clarification behavior, business terminology, and how results should be summarized or visualized.

2. Add semantic-model instructions. These belong in Prep for AI and explain how AI should understand and query the model. Include governed measures, date rules, relationships, filter logic, definitions, and important safeguards.

One important separation: Semantic Model AI Instructions explain how AI should understand the model. Data Agent Instructions explain how the agent should orchestrate and communicate.

3. Start with a benchmark set. Build a representative set of business questions with expected queries and answers. Use it to evaluate accuracy and expand coverage systematically.

4. Diagnose incorrect responses. Determine whether an incorrect answer was caused by missing metadata, unclear instructions, incomplete examples, an ambiguous user question, or a semantic-model issue.

5. Refine the instructions. Clarify data-source priorities, business definitions, date behavior, summary-versus-detail rules, and expected output formatting.

4.2 Best Practices and Recommendations

Use descriptive table, column, and measure names so the agent can understand the schema.
Build specialized agents for specific domains and limit each source to the tables and columns needed for that domain.
Be explicit about what the agent should do, not only what it should avoid.
Define business terms, abbreviations, synonyms, date rules, and important calculation logic.
Provide representative verified answers for common and complex questions.
Select only the sources and objects needed to answer the expected questions. A focused configuration improves query accuracy and efficiency.
Use SQL, DAX, or KQL examples only where they materially clarify complex joins, filters, or date logic.
Validate results under different user identities to confirm RLS and OLS behavior.

These practices help the Data Agent interpret questions correctly, query the right source, and return clear, context-rich answers.

5. Enriching and Evolving the Data Agent

You can enrich the agent with instructions, examples, terminology, and response rules. Data Agent instructions should define tone, routing, clarification, response formatting, and visual policy across sources. Avoid placing semantic-model-specific calculation logic at this level.

Use verified answers to anchor common business questions to the expected interpretation and governed query. As you test the agent, review both the returned result and the generated query. If an answer is incorrect, refine the semantic model metadata, AI data schema, verified answers, or instructions based on the actual cause.

Evaluate and Iterate with the Fabric Data Agent SDK

Evaluation is essential to the ongoing development of a Data Agent. Use the Fabric Data Agent SDK to test representative questions at scale, compare actual results with expected answers, and identify patterns in failures.

An effective evaluation set should include common questions, ambiguous questions, unsupported requests, date and comparison scenarios, hierarchy navigation, security-sensitive questions, and follow-up questions that depend on conversational context.

Add Code Interpreter

Code Interpreter gives the Data Agent a secure, sandboxed Python environment for analyzing retrieved data. With it enabled, the agent can support calculations, deeper analysis, and visualizations that go beyond the initial query result.

The governed data source should still produce the underlying result first. Code Interpreter should analyze that result rather than recreate business logic outside the semantic model.

6. How Data Agents Work at Runtime

When a user asks a question, the Data Agent uses Azure OpenAI to interpret the request and determine the most suitable connected source. It accesses each source under the user’s identity and security context.

For a Power BI semantic model, the agent uses the model’s metadata and Prep for AI configuration to generate and execute a DAX query. It then formats the governed result as a conversational response. Because the query runs under the user’s permissions and honors model and workspace security, the agent only returns information that the user is authorized to access.

7. Benefits and Limitations

Data Agents make governed analytics more accessible by reducing the need for users to write queries or build a report for every question. They can bring multiple supported data sources into one conversational experience while honoring existing security and governance.

However, a Data Agent is not automatically accurate simply because it is connected to trusted data. Its results depend on semantic-model quality, relevant metadata, clear instructions, verified answers, focused source selection, and continuous evaluation. Complex root-cause or predictive questions may also require additional analytical logic beyond a straightforward governed query.

8. Conclusion

Microsoft Fabric Data Agents provide a powerful new way to interact with enterprise data. By combining a well-prepared Power BI semantic model with focused Data Agent configuration, verified answers, clear instructions, and ongoing evaluation, you can create a conversational analytics experience that users can trust.

The goal is not merely to connect AI to data. The goal is to connect AI to governed business meaning.

Building a Better Microsoft Fabric Data Agent: Instructions, Visual Policy, and Cost-Aware Design

Praful_Potphode — Tue, 08 Sep 2026 13:34:16 GMT

Connecting a semantic model to a Fabric Data Agent is the easy part. The real work starts when the agent needs to understand what your business actually means.

A Fabric Data Agent can generate technically valid DAX and still return the wrong business answer. The missing piece is often not the model itself; it is the context we provide around it.

Consider this question:

Which region performed best last quarter?

It sounds straightforward, but the agent must make several decisions before it can answer.

What does performed best mean?

Highest Sales?
Highest Profit?
Best Profit Margin?

Which date defines last quarter?

Order Date?
Ship Date?

Should the agent return only the leading region, compare every region, or create a chart? If a governed Profit Margin measure already exists, should the agent use it or recreate the calculation from raw columns?

These are not primarily DAX questions. They are context and governance questions.

After working with Fabric Data Agents beyond the proof-of-concept stage, I stopped treating instructions as one large prompt. A reliable implementation has several layers:

Data enrichment
Semantic Model AI Instructions
Data Agent Instructions
Routing and ambiguity handling
Response and visual policy

Each layer has a different responsibility. Getting these layers right improves answer quality and can also reduce unnecessary reasoning, retries, query execution, and overly verbose responses.

1. There Is a Cost to Ambiguity

Fabric Data Agent consumption is token-based. The user’s question is only one part of the context processed by the AI. Agent instructions, source instructions, example queries, conversation history, and supporting context can also contribute to the input.

At the time of writing, the published consumption rates are:

1,000 input tokens = 100 CU seconds
1,000 cached input tokens = 10 CU seconds
1,000 output tokens = 400 CU seconds
Data-source query execution is charged separately

This creates an important architectural consideration.

A Poorly Prepared Data Agent

Imagine that a user asks:

Which region performed best?

The agent assumes that performance means Sales, executes a query, and returns an answer. The user then replies, I meant profitability.

Now we have another interaction, another round of reasoning, another response, and potentially another underlying query.

A better approach is to define the business interpretation once:

When performance is used without a specific KPI, evaluate both Sales and Profit.

The objective should not be to use the fewest tokens possible. It should be to use the minimum useful context required to produce the right answer with the least unnecessary reasoning.

A small prompt that produces the wrong answer is not an optimization. Adding thousands of characters of unnecessary instructions to every interaction is not an optimization either.

Optimize for cost per successful answer, not tokens per prompt.

2. Data Enrichment Comes First: Start with the Model

Semantic-model enrichment was covered in our previous article, so I will not repeat the full process here. The main principle is that a focused AI Data Schema, clear descriptions, relevant synonyms, governed measures, and a clean model reduce ambiguity and help the AI focus on the objects that matter.

Important: Instructions should add business context. They should not be used to repair a poorly designed semantic model.

Data Enrichment Guide - https://community.fabric.microsoft.com/blog/community_blog/why-your-fabric-data-agent-gives-wrong-answers-and-how-data-enrichment-fixes-it/5361211

3. The Most Important Separation: Semantic Model Instructions vs. Data Agent Instructions

Semantic Model AI Instructions and Data Agent Instructions are not the same, and they should not contain the same information.

Data Agent-level instructions are not passed to the DAX-generation tool used for semantic-model query generation. This changes how the instruction architecture should be designed.

Consider this rule:

When a user asks for margin, always use the governed Profit Margin measure.

If this rule exists only in the global Data Agent instructions, it is placed at a layer that the DAX-generation tool does not directly use. The rule belongs in the Semantic Model AI Instructions.

The easiest way to remember the separation is:

Semantic Model AI Instructions explain how AI should understand and query the model.
Data Agent Instructions explain how the agent should orchestrate and communicate.

Once this separation is clear, the overall framework becomes easier to maintain.

4. Semantic Model AI Instructions: Give the Model a Clear Purpose

Use Semantic Model AI Instructions for rules that affect how the model should be interpreted and queried. The business rules will change across Finance, Sales, HR, Supply Chain, Operations, and other domains, but the structure does not have to change.

Instead of one long block of rules, use a repeatable framework:

SEMANTIC MODEL AI INSTRUCTIONS
1. Purpose and Scope
2. Measure Priority
3. Authoritative Objects
4. Business Definitions
5. Governed Measure Mapping
6. Business Rules
7. Time Rules
8. Source Priority
9. Default Behaviour
10. Ambiguity Rules
11. Domain Restrictions
12. Cross-Domain Rules

The principle is straightforward:

If a rule changes the query, it probably belongs close to the semantic model.

Reusable Semantic Model Instruction Framework

5. Keep the Data Agent Focused on Orchestration

Once semantic logic lives at the correct layer, the Data Agent instructions become much cleaner. A generic framework can look like this:

DATA AGENT INSTRUCTIONS
1. Purpose
2. Scope
3. Data Source Behaviour
4. Response Policy
5. Visual Policy
6. Insight Generation
7. Recommendation Policy
8. Comparison Rules
9. Large Result Handling
10. Conversation Style
11. Explainability
12. Formatting
13. Missing Data
14. Follow-up Questions
15. Guardrails

The production pattern should reinforce one key rule:

Retrieve governed results from the semantic model and never override the model’s business rules, measure selection, relationships, or calculations.

Reusable Data Agent Instruction Framework

6. Make the Response Policy Adaptive

A fixed seven-section response can be useful for deep analysis, but it is unnecessary for a simple question such as:

What were Total Sales in 2025?

An adaptive response policy is more effective:

Simple KPI Question

Answer directly.
State the important period and filter context.
Keep the response concise.
Do not force a chart.

Analytical Question

State the scope.
Provide a short summary.
Show the supporting result.
Create one useful visual.
Highlight the key takeaways.

Deeper Analysis

Add recommendations and follow-up questions only when they provide real value. A useful analytical flow is: Scope, Executive Summary, Supporting Details, Visual, Key Takeaways, Recommendations, and Follow-up Questions.

7. Visual Policy Should Be Governed Too

Fabric Data Agent can generate native visual responses, and Data Agent Instructions can influence when a visual should be used and which supported chart type should be preferred.

At the time of writing, the supported visual types include:

Line chart
Multi-line chart
Column chart
Multi-column chart
Stacked column chart
Pie chart
Scatter plot
Area chart
Stacked area chart

A simple intent-based policy can map trends to line charts, category comparisons to column charts, composition to stacked columns, small part-to-whole comparisons to pie charts, and numeric relationships to scatter plots.

The safeguards matter just as much:

Do not create a visual when it adds no value.
Create a visual when the user explicitly requests one and the request is supported.
Do not generate several charts when one is sufficient.
Keep the KPI, filters, comparison, and time context consistent.
Never change the business calculation simply to produce a visual.

A chart should explain the result. The result should never be changed to fit the chart.

8. Guardrails Matter More Than More Instructions

A good instruction framework should make its non-negotiable behavior explicit.

Always

Respect semantic-model rules.
Respect the user’s filters.
Respect security.
Use governed results.

Never

Recreate governed KPIs.
Fabricate values.
Infer causes without evidence.
Override semantic-model logic.
Expose technical objects unnecessarily.
Change the user’s scope without permission.

9. What Changes When the Source Is SQL?

Now imagine that the same Data Agent also has access to a Fabric Warehouse.

A semantic model already provides relationships, governed measures, business-friendly metadata, DAX calculations, and a semantic layer. A raw SQL source is different. The Data Agent may need more direct guidance about:

What each table represents.
How tables should be joined.
Which columns own particular business concepts.
Which date should be used.
Common stored values and their meaning.
The grain of each fact table.
Query patterns that are not obvious from object names.

For SQL sources, use Data Source Instructions as a concise onboarding guide for someone seeing the database for the first time.

SQL Data Source Instruction Pattern

SOURCE PURPOSE
Use this source for Sales, Customer, Product,
and Inventory analysis.

TABLE GRAIN
FactSales contains one row per Order Line.
FactInventory contains Product inventory by location.

JOIN RULES
FactSales.CustomerKey → DimCustomer.CustomerKey
FactSales.ProductKey → DimProduct.ProductKey
FactSales.DateKey → DimDate.DateKey

BUSINESS RULES
Revenue = SUM(SalesAmount)
Order Count = COUNT(DISTINCT OrderID)
Do not use COUNT(*) for Order Count because
one Order can contain multiple Order Lines.

DATE RULES
Use OrderDate for normal Sales analysis.
Use ShipDate only for shipping or fulfillment questions.

QUERY RULES
Use explicit joins.
Avoid SELECT *.
Return only required fields.
Do not expose technical keys unless requested.

The goal is not to document the entire database. Include only the context that materially affects query generation.

Example Queries Demonstrate the Pattern

Consider the question:

Who are the top 10 customers by revenue?

SELECT TOP 10
    c.CustomerName,
    SUM(s.SalesAmount) AS Revenue
FROM FactSales s
INNER JOIN DimCustomer c
    ON s.CustomerKey = c.CustomerKey
GROUP BY
    c.CustomerName
ORDER BY
    Revenue DESC;

This single example teaches the agent:

Which fact table contains Sales.
How Customer should be joined.
How Revenue is aggregated.
How customer-level grouping works.
How Top N ranking should behave.

Instructions explain the rule. Example queries demonstrate the pattern.

A few carefully chosen examples are usually more valuable than dozens of near-identical queries.

10. Limitations to Keep in Mind

Data Source Instructions improve context, but they do not turn a SQL source into a governed semantic model.

Instructions do not repair poor data modeling.
Do not place every business KPI inside SQL instructions.
Stored values may need to be documented explicitly.
Example queries are patterns, not hard-coded answers.
Too many examples can create noise.
SQL Data Source Instructions are different from Semantic Model AI Instructions.

11. Final Thought

Building a reliable Fabric Data Agent is not about writing the longest possible prompt. It is about putting the right context at the right layer.

Use metadata to explain the data. Use Semantic Model AI Instructions to govern how the model is interpreted. Use Data Source Instructions and examples to guide SQL generation. Use Data Agent Instructions to govern orchestration, response behavior, visuals, and guardrails.

The goal is not more instructions. It is less ambiguity.

That is what moves a Fabric Data Agent from a successful demo toward governed conversational analytics.

References

Microsoft Learn: Data Agent consumption

Microsoft Learn: Semantic model best practices for Data Agent

Microsoft Learn: Configure your Data Agent

Microsoft Learn: Visual responses from a Fabric Data Agent

Microsoft Learn: Improve Data Source routing

GitRep - AI Instruction Framework

What have you found most difficult when moving a Fabric Data Agent from a proof of concept into a governed production experience? Share your experience in the comments.