Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.
Hi,
An agency I work with has given me a download of the data I need. It's 2,500 (!) CSV files. They've given me a webpage with the links to the files. So imagine that you need to go to www.download.com/thisisthepage.html and then you only see the following:
https://download01-01-2019.csv
https://download02-01-2019.csv
https://download03-01-2019.csv
... and this 2500 times.
I can easily pull a list of all the download link URL's in powerbi by using get data from web, and then drilling a bit deeper into the HTML. But the next step would be to load all those URL's and turn them into one query how can I pull all the data of these links in one query so I can start manipulating it?
Regards
Bas
Solved! Go to Solution.
Hi @Anonymous
once you have all csv-URLs in a column of a table, add another column with the formula: Web.Contents([YourColumnName])
This will retrieve the binary content from the web and create some arrows in the table name the should trigger the "Combine binaries"-procedure. This will create a query that will automatically be connected to a function ... that will automatically be applied to all your files. You can then perform all your transformations in the query and everything you do in there will automagically be applied to the other files as well (through the connected function).
Imke Feldmann (The BIccountant)
If you liked my solution, please give it a thumbs up. And if I did answer your question, please mark this post as a solution. Thanks!
How to integrate M-code into your solution -- How to get your questions answered quickly -- How to provide sample data -- Check out more PBI- learning resources here -- Performance Tipps for M-queries
Most likely need a custom M function. @ImkeF or @edhans can probably assist.
Hi @Anonymous
once you have all csv-URLs in a column of a table, add another column with the formula: Web.Contents([YourColumnName])
This will retrieve the binary content from the web and create some arrows in the table name the should trigger the "Combine binaries"-procedure. This will create a query that will automatically be connected to a function ... that will automatically be applied to all your files. You can then perform all your transformations in the query and everything you do in there will automagically be applied to the other files as well (through the connected function).
Imke Feldmann (The BIccountant)
If you liked my solution, please give it a thumbs up. And if I did answer your question, please mark this post as a solution. Thanks!
How to integrate M-code into your solution -- How to get your questions answered quickly -- How to provide sample data -- Check out more PBI- learning resources here -- Performance Tipps for M-queries
Wow! I just bumped into this post, and I tried it - it is functioning perfectly! Thanks for that!
Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City
Check out the April 2024 Power BI update to learn about new features.
User | Count |
---|---|
114 | |
97 | |
86 | |
70 | |
62 |
User | Count |
---|---|
151 | |
120 | |
103 | |
87 | |
68 |