Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.
Hi,
Could anybody help me to get the number of pages please? I think my problem is the condition to stop "each Table.IsEmpty(Page)," The call to this page has 1131 pages, and 1132 gives an error. Thanks.
let
GetData =
(PagePath as number) =>
let
Source =
Web.Contents("https://gtr.ukri.org:443/gtr/api/funds?p="&PagePath&"&s=100"
)
in
Source,
Source =
List.Generate(
() =>
[
page = 0,
Page = GetData(page)
],
each Table.IsEmpty(Page),
each
[
page = [page] + 1,
Page = GetData(page)
]
)
in
Source
Solved! Go to Solution.
@Anonymous , you might want to try a complete procedure like this if you insist on List.Generate().
let
GetData = (PagePath as number) =>
Xml.Tables(Web.Contents("https://gtr.ukri.org:443/gtr/api/funds?p="&Text.From(PagePath)&"&s=100")){[Name="fund"]}[Table],
Source = List.Generate(
() => [page = 1, Page = GetData(page)],
each not (try [Page])[HasError],
each [page = [page] + 1, Page = GetData(page)],
each [Page]
),
Dataset = Table.Combine(Source)
in
Dataset
the condition to contiue the loop is a bit tricky,
each not (try [Page])[HasError]
Here's a detailed reference to the syntax of try statement in M language,
https://bengribaudo.com/blog/2020/01/15/4883/power-query-m-primer-part-15-error-handling
Thanks to the great efforts by MS engineers to simplify syntax of DAX! Most beginners are SUCCESSFULLY MISLED to think that they could easily master DAX; but it turns out that the intricacy of the most frequently used RANKX() is still way beyond their comprehension! |
DAX is simple, but NOT EASY! |
@Anonymous , interestingly, I tried the assigned URL by chance, it returns
Thus, page parameter should range from 1 to 1131 rather than from 0.
I think the function should be defined as follows in order to scrape valid tables
GetData = (PagePath as number) =>
let
Source = Xml.Tables(Web.Contents("https://gtr.ukri.org:443/gtr/api/funds?p=" & Text.From(PagePath) & "&s=100")){[Name="fund"]}[Table]
in
Source,
In addition, List.Generate() can be replaced by List.Accumulate().
let
GetData = (PagePath as number) =>
let
Source = Xml.Tables(Web.Contents("https://gtr.ukri.org:443/gtr/api/funds?p=" & Text.From(PagePath) & "&s=100")){[Name="fund"]}[Table]
in
Source,
Source = List.Accumulate({1..1131}, {}, (s,c) => s & {GetData(c)}),
Combination = Table.Combine(Source)
in
Combination
Thanks to the great efforts by MS engineers to simplify syntax of DAX! Most beginners are SUCCESSFULLY MISLED to think that they could easily master DAX; but it turns out that the intricacy of the most frequently used RANKX() is still way beyond their comprehension! |
DAX is simple, but NOT EASY! |
Hi,
Thanks for this. Is that possible to work with List Generate? With List.Acumulate I will have to update {1..1131} as the website updates constantly with new content. That's why I sued ListGenerate, so I get the list of pages. So, I've updated the code with your suggestion, but I think the problem comes with the condition to stop [Page] = Null.
thanks
let
GetData =
(PagePath as number) =>
let
Source = Xml.Tables(
Web.Contents("https://gtr.ukri.org:443/gtr/api/funds?p="&Text.From(PagePath)&"&s=100"),
{[Name="Fund"]}[Table]
)
in
Source,
Source =
List.Generate(
() =>
[
page = 1,
Page = GetData(page)
],
each [Page] = null,
each
[
page = [page] + 1,
Page = GetData(page)
]
)
in
Source
@Anonymous , you might want to try a complete procedure like this if you insist on List.Generate().
let
GetData = (PagePath as number) =>
Xml.Tables(Web.Contents("https://gtr.ukri.org:443/gtr/api/funds?p="&Text.From(PagePath)&"&s=100")){[Name="fund"]}[Table],
Source = List.Generate(
() => [page = 1, Page = GetData(page)],
each not (try [Page])[HasError],
each [page = [page] + 1, Page = GetData(page)],
each [Page]
),
Dataset = Table.Combine(Source)
in
Dataset
the condition to contiue the loop is a bit tricky,
each not (try [Page])[HasError]
Here's a detailed reference to the syntax of try statement in M language,
https://bengribaudo.com/blog/2020/01/15/4883/power-query-m-primer-part-15-error-handling
Thanks to the great efforts by MS engineers to simplify syntax of DAX! Most beginners are SUCCESSFULLY MISLED to think that they could easily master DAX; but it turns out that the intricacy of the most frequently used RANKX() is still way beyond their comprehension! |
DAX is simple, but NOT EASY! |
Thanks. It worked!
However, I changed the website (bold) to grab data from other API (same website but different APIs depending on what info I want to grab). This also happens with other path. Both give "This table is empty".
If this helps, this is the website https://gtr.ukri.org/resources/api.html, and these the endpoints http://gtr.ukri.org/gtr/api/examples
Thanks.
let
GetData = (PagePath as number) =>
Xml.Tables(Web.Contents("https://gtr.ukri.org:443/gtr/api/projects?p="&Text.From(PagePath)&"&s=100")){[Name="fund"]}[Table],
Source = List.Generate(
() => [page = 1, Page = GetData(page)],
each not (try [Page])[HasError],
each [page = [page] + 1, Page = GetData(page)],
each [Page]
),
Dataset = Table.Combine(Source)
in
Dataset
Hi, @Anonymous
As M language is extremely strick on data type, this expression
"https://gtr.ukri.org:443/gtr/api/funds?p="&PagePath&"&s=100"
throws error when a PagePath (as number) is passed to it.
You might want to try
https://gtr.ukri.org:443/gtr/api/funds?p=" & Text.From(PagePath) & "&s=100"
Thanks to the great efforts by MS engineers to simplify syntax of DAX! Most beginners are SUCCESSFULLY MISLED to think that they could easily master DAX; but it turns out that the intricacy of the most frequently used RANKX() is still way beyond their comprehension! |
DAX is simple, but NOT EASY! |
Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City
Check out the April 2024 Power BI update to learn about new features.