Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!
Good afternoon.
I am trying to append data from a csv file that is refreshed daily. What I am trying to have is a accumulalive table that will keep all data and only add (append) the new data.
Example source data
PRODUCT_TYPE / ID / PRICE
TV / 7 / 1226
.
.
As you can see, there is no date, so I am adding a column with the current date to have:
PRODUCT_TYPE / ID / PRICE / DATE
TV / 7 / 1226 / 10/16/2020
.
.
What I am trying to acheive is something like this:
PRODUCT_TYPE / ID / PRICE / DATE
WASHER / 8 / 560 / 10/15/2020
GAME CONSOLE / 7 / 10/15/2020
TV / 9 / 840 / 10/14/2020
And append today´s records without loosing the previous data.
TV / 7 / 1226 / 10/16/2020
.
.
Did I explain myself?
Any guidance would be appreciate!
Regards
Jean Phillip Mercier
Hi @Jpmercier00 ,
Based on your description, wants to know couple of problems:
In addtion, the .csv file will refresh when click the refresh button in power query, not certain what does append a .csv file represent.
Best Regards,
Yingjie Li
Data is refrehed every day in a we site at 12:00, so the date is the current date, which I add as a new column.
2. Are there some other previous fields in the file or in another file? How could their dates field create?
No, there are no other files. Date can be created as a new column with the Today´s date.
Each time you refresh you will dowload the complete .csv that is on the web at that moment that corresponds to the current day data.
Thank you!
Power BI doesn't store data like that. In your case, I would set up a Flow to name and store your daily csv files (choose a name that incorporates the date so you can use it downstream), and have Power BI combine all the CSVs. If the csv is emailed to you, you can trigger the flow based on that. If manual, you could rename it (with date info) and store in a SharePoint or OneDrive location for easy refresh from Power BI.
Regards,
Pat
To learn more about Power BI, follow me on Twitter or subscribe on YouTube.
That is how I do It but it is no reliable a practical a the data is about 32,000 records daily and if you miss to do the store and renaming process I loose the entire day. Data is refreshed daily at 12:00.
It looks I will help for the SQL guys to to develop a query.
Thank you so much!