Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
STEV583
New Member

Schedule upload of daily csv files

Hi All, Just getting my head around Power BI. We have a lot of legacy systems that provide us with daily files. I think I understand that you can manually load and append this data to existing data. However, is it possible to automate this step? Thanks

1 ACCEPTED SOLUTION
v-ljerr-msft
Employee
Employee

@STEV583

 

According to your description, you can put all your csv files into one Folder and use Get Data -> Folder option in Power BI Desktop to load all the csv files and union these files to one table at the same time. Here is a good article about this for your reference.

 

After all files being loaded, if there are any new files added into the Folder, you just need to do a refresh(manual or scheduled) for the dataset in Power BI, then all new added files will be appended to the existing data automatically.

 

Regards

View solution in original post

5 REPLIES 5
STEV583
New Member

Hi Everyone,

 

Thanks so much for all of the advice and replies. I'm looking forward to getting to this stage in the reporting and will post an update to the community with my progress!

v-ljerr-msft
Employee
Employee

@STEV583

 

According to your description, you can put all your csv files into one Folder and use Get Data -> Folder option in Power BI Desktop to load all the csv files and union these files to one table at the same time. Here is a good article about this for your reference.

 

After all files being loaded, if there are any new files added into the Folder, you just need to do a refresh(manual or scheduled) for the dataset in Power BI, then all new added files will be appended to the existing data automatically.

 

Regards

Hi All,

 

Just checking back in to give the community an update. Thanks JerryLi for your post. Very simple and clear solution! Can I just say that I find the ability to do this absolutely mindblowing! As a non IT person I spend my life manipulating one form of data into another and then reporting via massive excel spreadsheets. Being able to do this level of reporting without requiring IT assistance is a game changer!

 

 

I completed the advice posted last night and today the report refreshed at the touch of a button! I still have to transfer the files from the web location to the desktop folder in question but I believe this is possible with scripts (and is outside of the scope of Power BI)

 

Thanks again JerryLi!

If these files come from a web source it's concievable that you could connect Power BI directly to that source rather than downloading the files to a desktop location first. It depends greatly on the specifics of the source though. If it's a service that generates a csv file maybe not, but if it's just an online file location somewhere you might be able to do it.





Did I answer your question? Mark my post as a solution!

Proud to be a Super User!




ankitpatira
Community Champion
Community Champion

@STEV583 Provided you have two csv files, a.csv and b.csv you can import then into power bi desktop -> append file b to a and publish your report to pbi service. Then each time you refresh via power bi service same step of appending file b to a will be applied (obviously you will have gateway setup for the refresh). This is the level of automation you can achieve. Power BI is not ETL tool and so if you're requirement is more complex I would suggest you do it outside of power bi and then use power bi at the end to analyse and visualise data.

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors