Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
Anonymous
Not applicable

Appending new CSV to existing file

Hi. I've a data file that consists of Year to date (YTD) transactions.

This file is stored in SharePoint and PowerBI Transforms and Loads this file. I built visualization based on that.

My requirements:

  1. Instead of asking the users to extract each month the YTD transactions, I would like to only get the last month transaction and append it to existing data.
  2. Also, I want to make sure that I use the SAME transformation rules that I used in the initial YTD data.
  3. Any suggestions here?

Note: I tried to do append, but I noticed in append, I need to apply transformation everytime I add a new file. I already completed transformation of data in the YTD file, and I just want to use same transformation on subsequent montly files.

9 REPLIES 9
v-shex-msft
Community Support
Community Support

Hi @Anonymous,

How did your records store? Since I do not so clear about your table structures, can you please share more details about these?

How to Get Your Question Answered Quickly

Regards,
Xiaoxin Sheng

Community Support Team _ Xiaoxin
If this post helps, please consider accept as solution to help other members find it more quickly.
Anonymous
Not applicable

Hi @v-shex-msft 

My data is in CSV file and the headers look as attached.

So both my Year to Date and monthly files have the exact header/layout.

As mentioned, I already built the transformation required to use the YTD data, but now I want to generate monthly incremental files and apply same transformation I did for the YTD data file.

 

 

Data Example.PNG

negi007
Community Champion
Community Champion

@Anonymous if you have all your files in one folder, you can use Get Data->Folder option. This way you can merge all files from the same folder into one table. Every time you put a new file in the folder, on refresh it will automatically pick new file/s and append in the table

negi007_0-1614699453277.png

 




Did I answer your question? Mark my post as a solution!
Appreciate your Kudos



Proud to be a Super User!


Follow me on linkedin

Anonymous
Not applicable

That's a good idea and I tried to implement this, but I'll have to build the transformation logic again. Don't I? I already did the transformation for the YTD file.

Hi @Anonymous,

So you mean your records are stored in different excel sheets with similar structures as your snapshot? (operation date on the top, detail parts are host on the below) 

If that is the case, I'd like to suggest adding a new column below to store the header date, then you can merge these tables and they will be grouped based on the 'backorder date'.

Regards,

Xiaoxin Sheng

Community Support Team _ Xiaoxin
If this post helps, please consider accept as solution to help other members find it more quickly.
PC2790
Community Champion
Community Champion

Hello @Anonymous ,

 

You can make your power query extraction of csv files generic rather so that it picks up all the files with Extension as csv. The below M query will do the trick

 

= Table.SelectRows(#"Previous Step", each [Extension]=".csv")

 

And then to only pick the previous months file, you can setup incremental refresh in Power BI Service.

Please refer this to setup incremental refresh.

 

Edit: Corrected the link 

Anonymous
Not applicable

can you check if the link is correct?

PC2790
Community Champion
Community Champion

Please check now

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.