Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
jrobertosm
Frequent Visitor

Extract data from an API paginated??

I need to extract data from an API paginated.

The item limit per page is 50.

There are 15,000 items to be extracted.

I can connect with the API and retrieve the first 50 items.

How can I recursively extract all the data? Create a single table with all 15,000 items ...

---

Code:

let
token = Number.ToText(123456),
take = Number.ToText(50),
skip = Number.ToText(0),
url = "https://conectapi.com/v1/authenticate",
body = "{""username"":""myusername@conectapi.com"",""password"":""123456"",""myid"":""123"",""Expires"":""30""}",
Parsed_JSON = Json.Document(body,65001),
BuildQueryString = Uri.BuildQueryString(Parsed_JSON),
Source = Json.Document(Web.Contents(url,[Headers = [#"Content-Type"="application/json"], Content = Text.ToBinary(body) ] )),
#"Convertido em Tabela" = Record.ToTable(Source),
#"Linhas FiltrImageadas" = Table.SelectRows(#"Convertido em Tabela", each ([Name] = "Token")),
#"Personalizado Adicionado" = Table.AddColumn(#"Linhas Filtradas", "URL_Apol_Cia", each "https://conectapi.com/v1/"&[Value]&"/Documents?take="&take&"&skip="&skip),
URL_Docs_Cia = #"Personalizado Adicionado"{0}[URL_Apol_Cia],
OrigemDocsCia = Json.Document(Web.Contents(URL_Apol_Cia,[Timeout=#duration(0,0,30,0)]))
in
OrigemDocsCia

---

Image query result:

g900.png

1 ACCEPTED SOLUTION
jrobertosm
Frequent Visitor

3 REPLIES 3
jrobertosm
Frequent Visitor

Hi;

Grateful for the help.

I solved the situation using:

- Authentication in API Auth, and;

- Function List.Generate;

 

The videos and content from the links below helped a lot:

- https://www.youtube.com/watch?v=Xhr864DlfNE;
- https://www.youtube.com/watch?v=yc78DZnGhbQ;

- https://www.youtube.com/watch?v=mXbjoSg9ldo;

- https://www.youtube.com/watch?v=hHkU8tPhVXs&t=492s;

-http://angryanalyticsblog.azurewebsites.net/index.php/2016/05/16/api-strategies-with-power-bi/;

mahoneypat
Employee
Employee

Please see this video for how to do this.

Power BI - Tales From The Front - REST APIs - YouTube

Pat





Did I answer your question? Mark my post as a solution! Kudos are also appreciated!

To learn more about Power BI, follow me on Twitter or subscribe on YouTube.


@mahoneypa HoosierBI on YouTube


Thank`s man.

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors