Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more.
Get startedGrow your Fabric skills and prepare for the DP-600 certification exam by completing the latest Microsoft Fabric challenge.
Hi all,
I am trying to load parquet data from my ADLS Gen 2 storage. The data is being generated by a Data Flow in ADF/Synapse which stores it in the ADLS Gen 2 Sink with a key partition on 2 columns. As a result, the folder structure looks a bit like this.
MyDataStore/CategoryName=AI/SubCategoryName=Swarm Intelligence/2023-02-27.parquet
However, the data in the parquet file doesn't have the 2 columns used in the key. Now when I try loading this into Power BI using the ADLS Gen 2 connector, I get all the data except the 2 columns that I have partitioned with. I am trying to load the parquet files using the following URL
https://pbixdemo.dfs.core.windows.net/pbixdemo2gen2storage/MyDataStore/ with a SAS token.
I have tried using something like https://pbixdemo.dfs.core.windows.net/pbixdemo2gen2storage/MyDataStore/CategoryName=*/SubCategoryNam... to see if we can infer the CategoryName and SubCategoryName columns, but this threw a 404 error.
Also, after I transform and load the data, I have no way to recompute the partition keys since I only have the Source column as an additional column in the transformed data that houses the name of the parquet file, which won't help in backtracking to the keys.
User | Count |
---|---|
77 | |
77 | |
68 | |
67 | |
49 |
User | Count |
---|---|
108 | |
105 | |
93 | |
83 | |
64 |