Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.

Reply

Power Query "looping"

Hi all,

 

I am trying to build a custom PQ connector for a data source that provides a number of data tables and related dimensions. Ideally I would like the user to enter the name of the table to retrieve and the connector needs to return a flattened table of the data and its dimensions.

 

I have the following input (with more rows obviously):

 

Data table

 

IDDimension1Dimension2SalesAmount
0Dim1_ADim2_Z50

 

Dimensions table

 

DimensionNameDimensionKeyDimensionValue
Dimension1Dim1_AMyCompany
Dimension 2Dim2_ZMyProduct

 

 What I would like to return is the following:

 

IDDimension1Dimension2SalesAmount
1MyCompanyMyProduct50

 

I know, this is easy enough by just loading the separate table and then creating a relationship between the tables. I do not want to present all the tables to the user, I want them all in one table. Also the number of dimensions will change from table to table (I can query the list of dimensions from the system however). Any given data table might have one or more dimensions that will need to be combined together into the resulting table. Essentially what I would like to build is a dynamic lookup responsive of the number of dimensions of the data table. 

 

In "normal" programming languages I would approach this using something like a foreach, however this obviously does not exist in PQ.

 

Any one that can help me solve this puzzle? Thanks!

7 REPLIES 7
ImkeF
Super User
Super User

Do the dimension-tables have a direct relationship to the main table or are there multiple levels to be expanded/connected (snowflake-structure)?

Does the data come from a SQL-server?

Imke Feldmann (The BIccountant)

If you liked my solution, please give it a thumbs up. And if I did answer your question, please mark this post as a solution. Thanks!

How to integrate M-code into your solution -- How to get your questions answered quickly -- How to provide sample data -- Check out more PBI- learning resources here -- Performance Tipps for M-queries

No, the data does not come from SQL server otherwise I would not have this issue. The tables have a relationship to the main table. However, I need to solve this in PQ, not in PP and I do not know the exact tables beforehand, since it is dependent on a parameter in the PQ

You need an "AllocationTable", that holds the names of the tables and the columns through which they are connected (see in example below).

 

This example flattens all tables that are connected to what you choose in "SelectedTable" from your local AdventureWorks-DB. Watch out: The "SelectedTable" needs to exist in column "FromTable" from "AllocationTable"!

 

let

// Parameters
Database = Sql.Databases("localhost", [CreateNavigationProperties=false]){[Name="AdventureWorksDW2012"]}[Data],
AllocationTable = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("pZPBboMwEET/hXN+oipNFPWQqukt4mC5q8oSXqNlfSBfH0iwsReIkvS4M/NGsAunU1Ea+2Gb2nUAxaY4qhraHyAy7Kj7hK7X+kQuL+aqzbXrQH8KzVmxcdjn3j0RoJ6agiC8gd4qzVuDCvXwJG9aO48cwXHOnTlWKobIDEOiLaShUcQWkHfkfDOBub6cnNelLx+7xEZkZt5y1ICKjJu2PwrCC+QeGQiBrzd5YedzvmVngRL+JghvjS89PHIFiX2R+/V6Ovg4584d2Lps7VGR7lrF/z79ofEbWqhroFcPIfkHFymx8DtHLvm/U2+Nf+oQC/Czh5AVYY4NQRBeVV0A", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [FromTable = _t, FromName = _t, ToTable = _t, ToColumn = _t]),
SelectedTable = "FactInternetSales",
MainTable = "FromTable",
IDMain = "FromName",
LookupTable = "ToTable",
IDLookup = "ToColumn",

// Filter User Selection
    FilterMainTable = Table.SelectRows(AllocationTable, each Record.Field(_, MainTable)=SelectedTable),

// Create row number for iterating
    AddIndex = Table.AddIndexColumn(FilterMainTable, "Index", 0, 1),

// Expanded Columns will be prefixed with name of the table
    ColumnNames = Table.Buffer(Table.AddColumn(AddIndex, "RenameHeaders", each {Table.ColumnNames(Database{[Name=Record.Field(_, LookupTable)]}[Data]),List.Transform(Table.ColumnNames(Database{[Name=Record.Field(_, LookupTable)]}[Data]), (ListItem)=> Record.Field(_, LookupTable) &"."& ListItem)})),

FlattenedTable =

// We only need the last element of the list created
                List.Last(

// Generates a list of nested tabes, looping through the filtered AllocationTable
                List.Generate(()=>    

// StartValue
                [Result = Database{[Name=SelectedTable]}[Data], Counter=-1],

// While-condition
                each [Counter] < Table.RowCount(ColumnNames), 

// Evaluated Expression: Lookup previous Result with new table and expand columns
                each [      Lookup= Table.NestedJoin([Result],{Record.Field(FilterMainTable{Counter}, IDMain)}, Database{[Name=Record.Field(FilterMainTable{Counter}, LookupTable)]}[Data],{Record.Field(FilterMainTable{Counter}, IDLookup)},"LookupTable",JoinKind.LeftOuter),
                            Result= Table.ExpandTableColumn(Lookup, "LookupTable", Record.Field(ColumnNames{Counter}, "RenameHeaders"){0},Record.Field(ColumnNames{Counter}, "RenameHeaders"){1}),
                            Counter = [Counter]+1
        ]
))[Result]

in
    FlattenedTable

Imke Feldmann (The BIccountant)

If you liked my solution, please give it a thumbs up. And if I did answer your question, please mark this post as a solution. Thanks!

How to integrate M-code into your solution -- How to get your questions answered quickly -- How to provide sample data -- Check out more PBI- learning resources here -- Performance Tipps for M-queries

thanks, interesting... do you have sample of the tables involved with data / pictures?

sample is in there: Just click on step "AllocationTable" and the content is there:

Do you need samples for the other tables as well?

 

PBI_SampleData.png

 

 

Imke Feldmann (The BIccountant)

If you liked my solution, please give it a thumbs up. And if I did answer your question, please mark this post as a solution. Thanks!

How to integrate M-code into your solution -- How to get your questions answered quickly -- How to provide sample data -- Check out more PBI- learning resources here -- Performance Tipps for M-queries

Wanted to report back that Imke has helped me resolve this. In the end we needed to do a Unpivot and Pivot operation.

v-chuncz-msft
Community Support
Community Support

@jeroenterheerdt,

 

You may learn to use Table.Join in Advanced Editor.

Community Support Team _ Sam Zha
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Helpful resources

Announcements
Microsoft Fabric Learn Together

Microsoft Fabric Learn Together

Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City

PBI_APRIL_CAROUSEL1

Power BI Monthly Update - April 2024

Check out the April 2024 Power BI update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.