cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Highlighted
Frequent Visitor

Append data from dynamic daily refreshed data

Good afternoon.

 

I am trying to append data from a csv file that is refreshed daily. What I am trying to have is a accumulalive table that will keep all data and only add (append) the new data.

 

Example source data

PRODUCT_TYPE / ID / PRICE

TV / 7 / 1226

.

.

 

As you can see, there is no date, so I am adding a column with the current date to have:

PRODUCT_TYPE / ID / PRICE / DATE

TV / 7 / 1226 / 10/16/2020

.

.

 

What I am trying to acheive is something like this:

PRODUCT_TYPE / ID / PRICE / DATE

WASHER / 8 / 560 / 10/15/2020

GAME CONSOLE / 7 / 10/15/2020

TV / 9 / 840 / 10/14/2020

 

And append today´s records without loosing the previous data.

 

TV / 7 / 1226 / 10/16/2020

.

.

 

Did I explain myself?

Any guidance would be appreciate!

 

Regards

Jean Phillip Mercier

4 REPLIES 4
Highlighted
Super User III
Super User III

Power BI doesn't store data like that.  In your case, I would set up a Flow to name and store your daily csv files (choose a name that incorporates the date so you can use it downstream), and have Power BI combine all the CSVs.  If the csv is emailed to you, you can trigger the flow based on that.  If manual, you could rename it (with date info) and store in a SharePoint or OneDrive location for easy refresh from Power BI.

 

Regards,

Pat

 





Did I answer your question? Please mark my post as a solution! Kudos are also appreciated!

Proud to be a Super User!




Highlighted
Community Support
Community Support

Hi @Jpmercier00 ,

Based on your description, wants to know couple of problems:

  1. Since there is no date column in the source data, how could you add it? In the csv file or in powe query?
  2. Are there some other previous fields in the file or in another file? How could their dates field create?

In addtion, the .csv file will refresh when click the refresh button in power query, not certain what does append a .csv file represent.

 

Best Regards,
Yingjie Li

 

Highlighted

That is how I do It but it is no reliable a practical a the data is about 32,000 records daily and if you miss to do the store and renaming process I loose the entire day. Data is refreshed daily at 12:00.

 

It looks I will help for the SQL guys to to develop a query.

 

Thank you so much!

Highlighted

  1. Since there is no date column in the source data, how could you add it? In the csv file or in powe query?

Data is refrehed every day in a we site at 12:00, so the date is the current date, which I add as a new column.

 

2. Are there some other previous fields in the file or in another file? How could their dates field create?

No, there are no other files. Date can be created as a new column with the Today´s date.

 

Each time you refresh you will dowload the complete .csv that is on the web at that moment that corresponds to the current day data.

 

Thank you!

 

Helpful resources

Announcements
Community Conference

Power Platform Community Conference

Check out the on demand sessions that are available now!

Community Conference

Microsoft Power Platform Communities

Check out the Winners!

secondImage

Power Platform 2020 release wave 2 plan

Features releasing from October 2020 through March 2021

Top Solution Authors
Top Kudoed Authors