Earn a 50% discount on the DP-600 certification exam by completing the Fabric 30 Days to Learn It challenge.
I can loop through the output using
files = mssparkutils.fs.ls('Files/orders/')
for file in files:
print(file.name, file.isDir, file.isFile, file.path, file.size)
But how do I send the output to a dataframe instead?
Solved! Go to Solution.
Hi @PeteSpillane ,
You can do this with the following code
from notebookutils import mssparkutils
# Initialise variables
data = []
columns = ["File Name", "Is Dir", "Is File", "File Path", "File Size"]
files = mssparkutils.fs.ls('Files/orders/')
# Add rows to lists
for file in files:
data.append([file.name, file.isDir, file.isFile, file.path, file.size])
# Create a dataframe
dataframe = spark.createDataFrame(data, columns)
# Show data frame
dataframe.show()
Tested my side in Fabric notebook and all seemed to work okay.
Hope it helps,
Kris
Works perfectly. Thanks Kris!
Hi @PeteSpillane ,
You can do this with the following code
from notebookutils import mssparkutils
# Initialise variables
data = []
columns = ["File Name", "Is Dir", "Is File", "File Path", "File Size"]
files = mssparkutils.fs.ls('Files/orders/')
# Add rows to lists
for file in files:
data.append([file.name, file.isDir, file.isFile, file.path, file.size])
# Create a dataframe
dataframe = spark.createDataFrame(data, columns)
# Show data frame
dataframe.show()
Tested my side in Fabric notebook and all seemed to work okay.
Hope it helps,
Kris
Ask questions in Data Engineering, Data Science, Data Warehouse and General Discussion.
Check out the April 2024 Fabric update to learn about new features.
User | Count |
---|---|
13 | |
9 | |
8 | |
4 | |
3 |