Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
Anonymous
Not applicable

Delay web query until condition met from another web query

I use an API that provides a Json response which includes a statusUrl and a downloadUrl.

If I immediately hit up the downloadUrl there is usually no data.

The statusUrl Json response advises when the report generation is "COMPLETE" and the reportUrl can be used to get the data.

 

Can I set up some sort of condition to check that a value in the statusUrl Json response (which I can convert to a normal table or list if easier) = "COMPLETE" or if it's "FAILED" or if it's "NOT READY"?

Then if it shows "COMPLETE" it runs Csv.Document(Web.Contents(downloadUrl),...) , if it's "NOT READY" it delays for a few seconds and tries again (a loop?) and if it's "FAILED" the output is an error message.

1 ACCEPTED SOLUTION
Nolock
Resident Rockstar
Resident Rockstar

Hi @Anonymous,

I've prepared a template which you can use for solving your problem.

You have to add your logic for processing JSON responses and modify the WaitForList according your needs.

let
    // request data
    RequestDataRecord = fnProcessJson(Web.Contents("request data URL")),

    // wait 10x for 5 seconds
    WaitForList = List.Repeat({5}, 10),
// wait until the response is COMPLETE, but max 50 secs WaitForStatus = List.MatchesAny( WaitForList, each Function.InvokeAfter( () => fnProcessJson(Web.Contents("http://StatusCheckURL")), #duration(0,0,0,_) ) = "COMPLETE" ), // download file if WaitForStatus was successful DownloadFile = if WaitForStatus then fnProcessJson(Web.Contents("download data URL")) else null in DownloadFile

View solution in original post

6 REPLIES 6
Nolock
Resident Rockstar
Resident Rockstar

Hi @Anonymous,

I've prepared a template which you can use for solving your problem.

You have to add your logic for processing JSON responses and modify the WaitForList according your needs.

let
    // request data
    RequestDataRecord = fnProcessJson(Web.Contents("request data URL")),

    // wait 10x for 5 seconds
    WaitForList = List.Repeat({5}, 10),
// wait until the response is COMPLETE, but max 50 secs WaitForStatus = List.MatchesAny( WaitForList, each Function.InvokeAfter( () => fnProcessJson(Web.Contents("http://StatusCheckURL")), #duration(0,0,0,_) ) = "COMPLETE" ), // download file if WaitForStatus was successful DownloadFile = if WaitForStatus then fnProcessJson(Web.Contents("download data URL")) else null in DownloadFile
Anonymous
Not applicable

The first response you wrote was correct, I just wasn't calling my function correctly!
Thanks for your help.

Anonymous
Not applicable

Thanks, @Nolock 
Looks like this will do the job, just working through it at the moment.


What should the output of the function in the middle be? Should it be text? a list?

I think that's where I'm failing now..

 

Hi @Anonymous,

do you mean this function? 

() => fnProcessJson(Web.Contents("http://StatusCheckURL"))

The result should be the status (i.e. COMPLETE, INPROGRESS, FAILED, ...) and its datatype is text because I compare it with the string "COMPLETE" on the next line.

But you can modify the code in a form which fulfils all your expectations. It is just a template how it could work.

Anonymous
Not applicable

Yep, that's the function. So.. I created the function, the output is text "COMPLETE" .
I'm not sure what odds going wrong, I still get the "null" output.

Hi @Anonymous,

a simplified version that waits just for one second and then returns "xxx". Maybe it helps.

let
    // wait 10x for 5 seconds
    WaitForList = List.Repeat({1}, 10),
    WaitForStatus = List.MatchesAny(
            WaitForList,
            each 
                Function.InvokeAfter(
                    () => "COMPLETE", 
                    #duration(0,0,0,_)
                ) = "COMPLETE"
            ),
            
    DownloadFile =
        if WaitForStatus then
            "xxx"
        else
            null
in
    DownloadFile

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors