Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
Anonymous
Not applicable

Loop through sql server table to read a set of data in each iteration and do unpivoting

Hi PowerBI experts,

 

I am getting data from Excel files (around 400 files from a folder location). These excel files have yyyymm values starts at the cell J28. And i need to unpivot all the 12 yyyymm cell values starting from J28, e,g 202001,202002, .. to 202012.So  So, basically i am skipping first 27 rows from sample excel file and then promote 28th row as column header and then apply unpivoting. All these steps i applied in transform query and defined as a function. Then i call this function as custom function in my main query to read all the excel files in similar manner.

 

Now as a performance tunning step, i have been asked to ETL all these excel files in SSIS and load into sql server tables.

I am bit struggling on data modelling. Can i load all excel files into a single target table and maintain sequence no dynamically while loading such as 1,2,3,.. so on. And then can i read data  from the target table filter by sequence no =1 and then do the unpivoting steps and then loop the same unpivoting steps for sequence no =2 for next excel file, so on. These may be achieved through  paramter in power query as a transofrm query and called in a main query to iterate through all sequence no.

 

I am new to M Language and not able to finding the way. Kindly please help me understand the options available in M Language to achieve this.

 

BR,

Umesh.

1 ACCEPTED SOLUTION
V-pazhen-msft
Community Support
Community Support

@Anonymous 
There is a Query Parameter feature allows you to dynamically change the excel tables based on the sequence you selected. However, I am not sure whether you can bring the parameter together with Pbi dataset back to SQL. You may try export power bi dataset to SQL afterwards: Exporting Power BI Data to SQL Server (biinsight.com).

 

Please refer to my example for using parameter on excel files:

1.  Connect to the folder that has all excel files, OK=>Transform.

V-pazhen-msft_0-1617329537834.png

 

2. Add an index column from 1, right click the index column=>add as new query.

V-pazhen-msft_6-1617329949593.png

3. Create the a parameter based on the new query.

V-pazhen-msft_5-1617329917073.png

 

4. Once parameter is created, set it as filter to the index column. 

V-pazhen-msft_8-1617330159750.png

 

V-pazhen-msft_7-1617330094469.png

5. The final step is to expand the excel table, Close and Apply.

V-pazhen-msft_9-1617330296058.png

 

You then can dynamically change the displayed table data using parameter.

V-pazhen-msft_10-1617330721445.png

 


Paul Zheng _ Community Support Team
If this post helps, please Accept it as the solution to help the other members find it more quickly.

 

View solution in original post

2 REPLIES 2
V-pazhen-msft
Community Support
Community Support

@Anonymous 
There is a Query Parameter feature allows you to dynamically change the excel tables based on the sequence you selected. However, I am not sure whether you can bring the parameter together with Pbi dataset back to SQL. You may try export power bi dataset to SQL afterwards: Exporting Power BI Data to SQL Server (biinsight.com).

 

Please refer to my example for using parameter on excel files:

1.  Connect to the folder that has all excel files, OK=>Transform.

V-pazhen-msft_0-1617329537834.png

 

2. Add an index column from 1, right click the index column=>add as new query.

V-pazhen-msft_6-1617329949593.png

3. Create the a parameter based on the new query.

V-pazhen-msft_5-1617329917073.png

 

4. Once parameter is created, set it as filter to the index column. 

V-pazhen-msft_8-1617330159750.png

 

V-pazhen-msft_7-1617330094469.png

5. The final step is to expand the excel table, Close and Apply.

V-pazhen-msft_9-1617330296058.png

 

You then can dynamically change the displayed table data using parameter.

V-pazhen-msft_10-1617330721445.png

 


Paul Zheng _ Community Support Team
If this post helps, please Accept it as the solution to help the other members find it more quickly.

 

lbendlin
Super User
Super User

You have been asked to do that in SSIS.  Not sure what that has to do with M?

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors