Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.
Hello Community,
Looking for your guidance and to learn from you.
I have a web based directory that stores CSV datasets. The datasets hold sales transactions data. A new CSV is automatically added into the directory following each day of new sales. I have attached a screengrab by way of illustrating this. It is secured with username / password, of course.
I want to build a model that can refresh / incorporate the new data as each additional CSV file is added to the directory, merging the new data with the historic data already imported into the Power BI model. I will then be able to do whatever analysis is necessary thereafter.
I only wish to reach out to the directory only to gather new data from any CSV files not prviously loaded into my Power BI model so as not to place un-necessary stress on the system.
Can you advise on the best approach to achieve this?
As always, thank you for your kind assistance and for sharing knowledge.
Much appreciated.
Solved! Go to Solution.
Hi @Anonymous
Do you mean you'd like to automatically fetch new files in the directory into power bi ?
Are the files all in the same format, for example csv (comma separated values)?
Is it important that each file ends up in a separate table?
You can use Get Data - Other - Folder to ask Power Query to grab all files in a specified folder. On each refresh, all data in that folder will be read so when new files are added, they will be read into the model upon refresh. You can also use all methods available in Power Query to shape the data as you see fit. All input will however end up in one table, but with a column containing the file name so it should be a simple matter to extract the date from the file name in that column in order to allow processing of date separated data.
You might refer to this similar thread which appends new excel automatically:
https://community.powerbi.com/t5/Desktop/Get-data-from-folder-and-append/td-p/526274
Hi @Anonymous
Do you mean you'd like to automatically fetch new files in the directory into power bi ?
Are the files all in the same format, for example csv (comma separated values)?
Is it important that each file ends up in a separate table?
You can use Get Data - Other - Folder to ask Power Query to grab all files in a specified folder. On each refresh, all data in that folder will be read so when new files are added, they will be read into the model upon refresh. You can also use all methods available in Power Query to shape the data as you see fit. All input will however end up in one table, but with a column containing the file name so it should be a simple matter to extract the date from the file name in that column in order to allow processing of date separated data.
You might refer to this similar thread which appends new excel automatically:
https://community.powerbi.com/t5/Desktop/Get-data-from-folder-and-append/td-p/526274
@Anonymous , refer if this can help
https://www.howtoexcel.org/power-query/how-to-extract-data-from-multiple-webpages/
Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City
Check out the April 2024 Power BI update to learn about new features.
User | Count |
---|---|
106 | |
104 | |
77 | |
68 | |
61 |
User | Count |
---|---|
148 | |
107 | |
107 | |
83 | |
70 |