Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
PedroC88
Frequent Visitor

Run Data Gateway from container

Currently my company has a data gateway installed on a physical server on-prem. This server is being decommissioned and I need to run the datagateway from Azure where we have the VPNs configured and access to the on-prem servers. I know I could spin a VM and install the gateway there but this isn't cost-effective as that VM would need to be up and running 24x7 only to serve the gateway. I think it's best to prepare a container image with the gateway (or an empty windows image that installs and configures the gateway on load) but I don't have a clue where to start.

 

How can I spin a container to serve the data gateway? Or is there another way to access on-prem sql servers from the power bi platform?

16 REPLIES 16
Anonymous
Not applicable

Hi everyone
I'm facing the same problem as @PedroC88 of trying to set up a gateway in a container, to let powerBI refresh properly, but without the use of a VM.
So I was wondering if any solutions were found in the end.
What I'm interested in is if it's really possible install the gateway in a container and how I can achieve this.
If someone could help a newbie, it would be very appreciated.
Thx

Greg_Deckler
Super User
Super User

@PedroC88 - Well, in theory you could use automation to only spin up the VM say, twice a day and schedule your refreshes to run in that window?


@ me in replies or I'll lose your thread!!!
Instead of a Kudo, please vote for this idea
Become an expert!: Enterprise DNA
External Tools: MSHGQM
YouTube Channel!: Microsoft Hates Greg
Latest book!:
Mastering Power BI 2nd Edition

DAX is easy, CALCULATE makes DAX hard...

@Greg_Deckler 

Well... this would technically work but a question I had for another day was how can I allow the users to refresh the datasources on demand? Note that the company is only now on-boarding PBI so this is all kinda new to us. If we have that scenario of allowing users to run the reports on demand how could we handle that?

As long as "on demand"  means "from 8am UTC to 10am UTC because that's when the gateway is spun up" - that would work. Doesn't really matter if the refresh is scheduled, on demand, or OneDrive. If the gateway is down the refresh request will fail.

@lbendlin that's why I'm not all bought on that "time window" approach. Normally users don't need to see live data, yesterday's data is usually fine, however, on month-end closings for example when accounting is working on reconciling their systems, if they make any entries or adjustments they need to be able to refresh the report on demand, highlighting the need for the gateway to be on 24x7.

Also - this whole discussion assumes that your data sources are accessed in import mode.  As soon as you use Direct Query you need your gateway to be on all the time anyway.

Indeed, which brings me back to the original question, is there a way to run the data gateway in a container to have it running 24x7 in a cost-effective way?

Only one way to find out - the VPN will be the critical piece of the puzzle.  I don't think the container needs to be joined to your AD domain but the user logging into the gateway management app needs to successfully authenticate against your company's AD.

Our AD is also hosted on Azure and we already have service accounts for the application in question that we can use for authentication.

I am not sure a service account will cut it.  I am not talking about the account the gateway service runs under, I am talking about the gateway config user.

 

lbendlin_0-1599613991331.png

 

 

 

@lbendlin I see what you mean now, and is there any way to manage the gateway from powershell?

yes, you can use PS5 with OnPremisesDataGatewayHAMgmt cmdlets, PS6.4+ with DataGateway cmdlets, or plain old Invoke-RestMethod calls for manual API wrangling.

 

Not sure if any of these allow you to set a gateway up from scratch though.

 

lbendlin
Super User
Super User

Are you saying that running an on-prem VM has a physical cost for you?

No it doesn't. I was saying I need to run the gateway from Azure as the company is trying to move away from on-prem servers except in instances where absolutely required. So I'd need to create a VM in Azure but it's really expensive to run a VM 24x7 thus I was thinking of creating a docker image to do it.

The problem is that gateways by definition have to be on prem (well, they have to be able to access on prem data sources, and they have to be able to see the on prem AD). And gateways are rather pointless if they are not 24x7.

 

Of course you can go crazy and run them on Azure with a VPN connection back into your network.  But that wholly defeats the purpose of putting the gateway as close as possible to your on prem data sources for performance reasons. 

 

Think of all the data travel to and fro that would result from placing the gateway into Azure.

@lbendlin 

The container or the VM would be joined to a VPN in Azure with access to the on-prem servers. About the performance, it wouldn't be worse than it is today because we already run the gateway from contry x and connect it to countries a, b andto serve those countries' data

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.