cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
New Member

Spooling Data Gateway

Hi,

 

We are just testing a new data gateway on an Azure VM, the data volume used in the current test data flow is much bigger than we'd tried previously (50 million records coming from an Azure SQL database). It seems that the whole dataset is spooling before its moved to the Power BI Service,taking a few hours,  is this the expected behaviour? I'd previously thought spooling was for a situation where overflow occurred and data wasn't moving fast enough through the gateway, the machine spec is inline with those recommended in the documentation, though there is only one virtual NIC so perhaps throughput may be a bottleneck.

 

Appreciate any clarification/advice!

 

Thanks

1 ACCEPTED SOLUTION
Microsoft
Microsoft

Hi  @D1234 ,

Data is temporarily stored on the gateway machine. This data storage continues until all data is received from the data source. The data is then sent back to the cloud service. This process is called spooling.

And we can't set the spooling location to a custom directory, which is located in the installation directory of the gateway:

C:\ Users \ PBIEgwService \ AppData \ Local \ Microsoft \ In-premises data gateway \ Spooler

Therefore, when installing the gateway,We recommend you use a solid-state drive (SSD) as the spooling storage.

 

Links of related gateways and spools:

https://docs.microsoft.com/en-us/data-integration/gateway/service-gateway-onprem-indepth

 

Best Regards,

Liu Yang

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

1 REPLY 1
Microsoft
Microsoft

Hi  @D1234 ,

Data is temporarily stored on the gateway machine. This data storage continues until all data is received from the data source. The data is then sent back to the cloud service. This process is called spooling.

And we can't set the spooling location to a custom directory, which is located in the installation directory of the gateway:

C:\ Users \ PBIEgwService \ AppData \ Local \ Microsoft \ In-premises data gateway \ Spooler

Therefore, when installing the gateway,We recommend you use a solid-state drive (SSD) as the spooling storage.

 

Links of related gateways and spools:

https://docs.microsoft.com/en-us/data-integration/gateway/service-gateway-onprem-indepth

 

Best Regards,

Liu Yang

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

Helpful resources

Announcements
secondImage

Congratulations!

We are excited to announce the Power BI Super Users!

Microsoft Ignite

Microsoft Ignite with Arun Ulag

Featured Session: Drive Data Culture with Power BI- Vision, Strategy & Roadmap. Register here https://myignite.microsoft.com #MSIgnite #PowerPlatform #Power BI ​

Microsoft Ignite

Microsoft Ignite

Join digitally, March 2–4, 2021 to explore new tech that's ready to implement. Experience the keynote in mixed reality through AltspaceVR!

secondImage

The largest Power BI virtual conference

100+ sessions, 100+ speakers, Product managers, MVPs, and experts. All about Power BI. Attend online or watch the recordings.

Top Solution Authors
Top Kudoed Authors