Power BI online - dataset sync fails for a "folder" as data source

[bichlert]

With Power BI Desktop I prepared a report which is accessing .xlsx files and also a folder to get the creation date of this .xlsx files. In Power BI Desktop the refresh is working fine (folder sync is slow but working). Now I published this report to Power BI Online. Here the sync of this dataset isn't working (sync is running via a data gateway, this one is working fine for all other reports, only this report with a folder as data source isn't working).

Any ideas?

Error:

There was an error when processing the data in the dataset.

Please try again later or contact support. If you contact support, please provide these details.

Data source error{"error":{"code":"DM_GWPipeline_Client_AsyncOperationStreamingRetriesExhausted","pbi.error":{"code":"DM_GWPipeline_Client_AsyncOperationStreamingRetriesExhausted","parameters":{},"details":[],"exceptionCulprit":1}}} Table: Folder.

[v-qiuyu-msft]

Hi @bichlert, 

From your description, there are two data sources: Excel and Folder in your report, right? 

Please find if there is any detail information in gateway logs: https://docs.microsoft.com/en-us/data-integration/gateway/service-gateway-tshoot#troubleshooting-too...

Best Regards,
Qiuyun Yu

[Anonymous]

I have the same problem: Setting up refresh in Power BI Service for sources of type folder does not work. Which also means, refresh of the whole dataset is not possible.

[v-qiuyu-msft]

Hi @Anonymous, 

What's the detail refresh failure message? Is the folder located in online site or local drive? 

Best Regards,
Qiuyun Yu

[Anonymous]

The files are on local drive. I just tried to reproduce it to catch the error and with a very plain vanilla use case it worked. But I had to install a personal gateway to make it run although I already had an enterprise gateway running on the same machine.

[bichlert]

Hi @v-qiuyu-msft 

you are correct - there are two data sources: Excel and Folder. The Folder data source path points to a local file server and NOT to Sharepoint online.

Details from the gateway error log:

DM.EnterpriseGateway Error: 0 : 2020-04-13T06:44:20.1846235Z DM.EnterpriseGateway MGAS [DM.Pipeline.Diagnostics] StackTrace: at Microsoft.PowerBI.DataMovement.Pipeline.Diagnostics.AsyncOperationStreamingCancelledException.TraceConstructor()
at Microsoft.PowerBI.DataMovement.Pipeline.Diagnostics.AsyncOperationStreamingCancelledException..ctor(Guid asyncOperationId, PowerBIErrorDetail[] errorDetails)
at Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.GatewayOperationResultSpooler.SpooledOperationController.SpooledOperationPacketManager.<StreamAsync>d__18.MoveNext()
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run()
at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(Action action, Boolean allowInlining, Task& currentTask)
at System.Threading.Tasks.Task.FinishContinuations()
at System.Threading.Tasks.Task`1.TrySetCanceled(CancellationToken tokenToRecord, Object cancellationException)
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.SetException(Exception exception)
at System.Threading.SemaphoreSlim.<WaitUntilCountOrTimeoutAsync>d__31.MoveNext()
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run()
at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(Action action, Boolean allowInlining, Task& currentTask)
at System.Threading.Tasks.Task.FinishContinuations()
at System.Threading.Tasks.Task`1.TrySetResult(TResult result)
at System.Threading.Tasks.TaskFactory.CompleteOnInvokePromise.Invoke(Task completingTask)
at System.Threading.Tasks.Task.FinishContinuations()
at System.Threading.Tasks.Task`1.TrySetCanceled(CancellationToken tokenToRecord, Object cancellationException)
at System.Threading.Tasks.Task.DelayPromise.Complete()
at System.Threading.CancellationCallbackInfo.ExecuteCallback()
at System.Threading.CancellationTokenSource.ExecuteCallbackHandlers(Boolean throwOnFirstException)
at System.Threading.CancellationTokenSource.NotifyCancellation(Boolean throwOnFirstException)
at System.Threading.CancellationTokenSource.LinkedTokenCancelDelegate(Object source)
at System.Threading.CancellationCallbackInfo.ExecuteCallback()
at System.Threading.CancellationTokenSource.ExecuteCallbackHandlers(Boolean throwOnFirstException)
at System.Threading.CancellationTokenSource.NotifyCancellation(Boolean throwOnFirstException)
at Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.GatewayOperationResultSpooler.SpooledOperationController.SpooledOperationPacketManager.<StreamAsync>d__18.MoveNext()
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
at Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.GatewayOperationResultSpooler.SpooledOperationController.SpooledOperationPacketManager.StreamAsync(StreamingCallbackDelegate transferCallback, GatewaySerializationContext serializationContext)
at Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.GatewayOperationResultSpooler.SpooledOperationController.<StreamAsync>d__32.MoveNext()
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
at Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.GatewayOperationResultSpooler.SpooledOperationController.StreamAsync(StreamingCallbackDelegate transferCallback, GatewaySerializationContext serializationContext)
at Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.Serialization.GatewaySerializer.<>c__DisplayClass14_0.<<SerializeImpl>b__0>d.MoveNext()
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
at Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.Serialization.GatewaySerializer.<>c__DisplayClass14_0.<SerializeImpl>b__0()
at Microsoft.PowerBI.DataMovement.Pipeline.GatewayPipelineTelemetry.PipelineTelemetryService.<ExecuteInActivity>d__5.MoveNext()
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
at Microsoft.PowerBI.DataMovement.Pipeline.GatewayPipelineTelemetry.PipelineTelemetryService.ExecuteInActivity(PipelineActivityType pipelineActivityType, Func`1 action)
at Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.Serialization.GatewaySerializer.<SerializeImpl>d__14.MoveNext()
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
at Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.Serialization.GatewaySerializer.SerializeImpl(GatewaySerializationContext serverContext)
at Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.Serialization.GatewaySerializer.<>c__DisplayClass13_0.<<Serialize>b__0>d.MoveNext()
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
at Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.Serialization.GatewaySerializer.<>c__DisplayClass13_0.<Serialize>b__0()
at Microsoft.PowerBI.DataMovement.Pipeline.GatewayPipelineTelemetry.PipelineTelemetryService.<ExecuteInActivity>d__5.MoveNext()
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
at Microsoft.PowerBI.DataMovement.Pipeline.GatewayPipelineTelemetry.PipelineTelemetryService.ExecuteInActivity(PipelineActivityType pipelineActivityType, Func`1 action)
at Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.Serialization.GatewaySerializer.<Serialize>d__13.MoveNext()
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
at Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.Serialization.GatewaySerializer.Serialize(GatewaySerializationContext serverContext)
at System.Threading.Tasks.Dataflow.ActionBlock`1.ProcessMessageWithTask(Func`2 action, KeyValuePair`2 messageWithId)
at System.Threading.Tasks.Dataflow.ActionBlock`1.<>c__DisplayClass7.<.ctor>b__1(KeyValuePair`2 messageWithId)
at System.Threading.Tasks.Dataflow.Internal.TargetCore`1.ProcessMessagesLoopCore()
at System.Threading.Tasks.Task.Execute()
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot)
at System.Threading.Tasks.Task.ExecuteEntry(Boolean bPreventDoubleExecution)
at System.Threading.ThreadPoolWorkQueue.Dispatch()

[v-qiuyu-msft]

@Anonymous Please run the latest on-premises data gateway on the same machine which has personal gateway run. And configure folder data source in Manage Gateways and set the dataset to use this on-premises data gateway. 

@bichlert Please update your gateway to the latest version as well to test again. 

Best Regards,
Qiuyun Yu

[bichlert]

@v-qiuyu-msft thank you for your help but newest version is already installed...

[RadimHampel]

Hi, I've got exactly same error, latest getaway, in my case model contains only SQL queries. I causes the refresh to fail randomly - scheduled refresh is set tu run 4 times a day and it ramdomly fails, please have a look.

Thanks.

[v-qiuyu-msft]

Hi all, 

Please create a support ticket to let engineers look into the issue on your side. 

Best Regards,
Qiuyun Yu