Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.

Reply
Anonymous
Not applicable

Count number of records in a table power query

Hi, I have a power query that is extracting the data using the rest API. 

My requirement is to run the query first to extract all the records using the below query:

let
  Pagination = List.Skip(List.Generate( () => [WebCall=[result = {0}], Page = 0, Counter=0], // Start Value
    each List.Count([WebCall][result])>0 or [Counter] =0// Condition under which the next execution will happen
    [Query=[sysparm_offset =Text.From([Counter])]])),
    Page = [Page]+1,
    Counter = [Counter]+1000// internal counter
    ]
    ) ,1),

this will extract all the records from the table. Now I want to run the same query but only want to extract records updated in the last 7 days. What changes I need to make in the query so it should check if there are already records in the table(power query) and

1.if not it will extract the entire dataset 

2. if there are records in the table then it should only extract the updated dataset. Something like this

https://<host>/api/now/table/incident?sysparm_query=sys_created_onONToday%40javascript%3Ags.beginningOfToday()%40javascript%3Ags.endOfToday()%5EORsys_updated_onBETWEENjavascript%3Ags.beginningOfToday()%40javascript%3Ags.beginningOfLast7Days()

 

 

1 ACCEPTED SOLUTION
Mariusz
Community Champion
Community Champion

Hi @Anonymous 

 

Power Query does not have a capability to store the data, so every time you run the script it will pull the data preview egain or it will pull whole data when you load into the model.

You could use dataflows to simulate your scenerio with incremental refresh if your sorce suports it and you are lucky enough to be on premium.

You can think of Power Query as a ETL tool not a storage db.

Please see the below for the ref.

https://docs.microsoft.com/en-us/power-query/power-query-what-is-power-query

 

 

Best Regards,
Mariusz

If this post helps, then please consider Accepting it as the solution.

Please feel free to connect with me.
LinkedIn

 

View solution in original post

1 REPLY 1
Mariusz
Community Champion
Community Champion

Hi @Anonymous 

 

Power Query does not have a capability to store the data, so every time you run the script it will pull the data preview egain or it will pull whole data when you load into the model.

You could use dataflows to simulate your scenerio with incremental refresh if your sorce suports it and you are lucky enough to be on premium.

You can think of Power Query as a ETL tool not a storage db.

Please see the below for the ref.

https://docs.microsoft.com/en-us/power-query/power-query-what-is-power-query

 

 

Best Regards,
Mariusz

If this post helps, then please consider Accepting it as the solution.

Please feel free to connect with me.
LinkedIn

 

Helpful resources

Announcements
Microsoft Fabric Learn Together

Microsoft Fabric Learn Together

Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City

PBI_APRIL_CAROUSEL1

Power BI Monthly Update - April 2024

Check out the April 2024 Power BI update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors