Rest API - Loop to get results from 100+ pages

[Anonymous]

Dear community

I am working on a BI project based on Algolia. Algolia is a search engine like Elastic search with a Rest API.

I managed to do a Post call to Algolia with a Body that contains the required fields and the Page.

let
   body = "{    
    ""page"":0,
    ""distinct"":1,
    ""facetFilters"": [  [ ""store.FR001.qty > 0"",
    ""store.FR002.qty > 0"",
    ""store.FR012.qty > 0"",
    ""store.FR021.qty > 0"",
    ""store.FR022.qty > 0"",
    ""store.FR023.qty > 0"",
    ""store.FR041.qty > 0"",
    ""store.FR042.qty > 0"",
    ""store.FR051.qty > 0"",
    ""store.FR061.qty > 0"",
    ""store.FR071.qty > 0"",
    ""store.FR072.qty > 0"",
    ""store.FR073.qty > 0"",
    ""store.FR075.qty > 0"",
    ""store.FR101.qty > 0"",
    ""store.FR102.qty > 0"",
    ""store.FR103.qty > 0"",
    ""store.FR104.qty > 0"",
    ""store.FR202.qty > 0"",
    ""store.FR601.qty > 0"",
    ""store.FR606.qty > 0""]],
    ""attributesToRetrieve"": [
	""titre"",
    ""hierarchy.lvl2.code"",
    ""reference_style_couleur"",
	""subtitle"",
	""descriptif_couleur"",
	""regroupement_couleur"",
	""SKU"",
	""price.VEUR"",
    ""images"",
    ""reference_style"",
    ""code_couleur"",
	""store.FR601"",
    ""derniere_collection""
]
}",
   Source = Json.Document(Web.Contents("https://xxx-dsn.algolia.net/1/indexes/xxx/query", [Content=Text.ToBinary(body), Headers=[#"Content-Type"="application/json", #"X-Algolia-API-Key"="xxx", #"X-Algolia-Application-Id"="xxx"]])),
    hits = Source[hits],
    #"Converti en table" = Table.FromList(hits, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
    #"Column1 développé" = Table.ExpandRecordColumn(#"Converti en table", "Column1", {"SKU", "reference_style_couleur", "reference_style", "code_couleur", "regroupement_couleur", "derniere_collection", "titre", "hierarchy", "descriptif_couleur", "subtitle", "objectID", "_highlightResult"}, {"Column1.SKU", "Column1.reference_style_couleur", "Column1.reference_style", "Column1.code_couleur", "Column1.regroupement_couleur", "Column1.derniere_collection", "Column1.titre", "Column1.hierarchy", "Column1.descriptif_couleur", "Column1.subtitle", "Column1.objectID", "Column1._highlightResult"}),
    #"Column1.hierarchy développé" = Table.ExpandRecordColumn(#"Column1 développé", "Column1.hierarchy", {"lvl2"}, {"Column1.hierarchy.lvl2"}),
    #"Column1.hierarchy.lvl2 développé" = Table.ExpandRecordColumn(#"Column1.hierarchy développé", "Column1.hierarchy.lvl2", {"code"}, {"Column1.hierarchy.lvl2.code"}),
    #"Colonnes supprimées" = Table.RemoveColumns(#"Column1.hierarchy.lvl2 développé",{"Column1.objectID"}),
    #"Column1.regroupement_couleur développé" = Table.ExpandRecordColumn(#"Colonnes supprimées", "Column1.regroupement_couleur", {"name"}, {"Column1.regroupement_couleur.name"})
in
    #"Column1.regroupement_couleur développé"

From this query, I received these results:

Hits:List (for the list I can develop for page 0)

nbHits:107722 (total amount of results)

nbPages:107

I would like to retreive the nbPages number and call every pages so I get the results in a single source. Do you know how to handle this?

Many thanks

Salah

[Greg_Deckler]

@Anonymous - If I remember correctly, @ImkeF has done things like this.