Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.
So I have this report data generated from a system that I would like to wrangle to a good tidy format with power query.
The data set has an information about the export in cell A1.
Then row 2-6 contains some informationd data for the series that in column. The timeseries data below is then divided with the first column beeing the actual timestamp following with the value and the status of that value (e.g. "Corrected" or "Calculated").
I would like to transform this in to a tidy dataset that has each variable as columns and every observation in the rows.
I've attached a spreadsheet with some sample data. In the tab "Base data" the data is the original format. I've added a tab "Wanted output" to show how I would like the data to look.
Since I have multiple reports in this format I would ultimately make this in to a function to import and clean data from a folder structure.
Can't seem to attach a file so here is the original data:
Hourlist 2019-05-14 00:00 - 2019-05-22 23:00 | ||||||
Name | Facility 1 | Facility 2 | Facility 3 | |||
Facility number | ||||||
Alternative Fac. Nr | ||||||
Serie-ID | ABC1 | ABC2 | ABC3 | |||
Time format | Normal time | Normal time | Normal time | |||
Hour | Value | Status | Value | Status | Value | Status |
2019-05-14 00:00 | 0,00 | 0,00 | 0,00 | |||
2019-05-14 01:00 | 0,00 | 0,00 | 0,00 | |||
2019-05-14 02:00 | 0,00 | 0,00 | 0,00 | |||
2019-05-14 03:00 | 0,00 | 0,04 | 0,04 | |||
2019-05-14 04:00 | 0,33 | 0,49 | Corrected | 0,49 | ||
2019-05-14 05:00 | 1,89 | 2,07 | 2,16 | |||
2019-05-14 06:00 | 7,71 | 7,94 | 7,98 | |||
2019-05-14 07:00 | 15,35 | 15,67 | 14,82 | |||
2019-05-14 08:00 | 21,67 | 23,27 | 19,98 | |||
2019-05-14 09:00 | 25,66 | Calculated | 27,51 | 23,51 | ||
2019-05-14 10:00 | 27,32 | Calculated | 29,11 | 13,49 | Corrected | |
2019-05-14 11:00 | 27,07 | 28,59 | 0,00 | Corrected | ||
2019-05-14 12:00 | 25,42 | 26,88 | 0,00 | Corrected | ||
2019-05-14 13:00 | 21,42 | 22,73 | 0,00 | Corrected | ||
2019-05-14 14:00 | 16,47 | 17,44 | 0,00 | Corrected | ||
2019-05-14 15:00 | 9,30 | 10,12 | 0,00 | Corrected | ||
2019-05-14 16:00 | 2,24 | 2,75 | 0,00 | Corrected | ||
2019-05-14 17:00 | 0,80 | 1,13 | 0,00 | Corrected | ||
2019-05-14 18:00 | 0,52 | 0,72 | 0,00 | Corrected | ||
2019-05-14 19:00 | 0,07 | 0,14 | 0,00 | Corrected | ||
2019-05-14 20:00 | 0,00 | 0,00 | 0,00 | Corrected |
Optimal output:
Hour | Value | Status | Name | Facility number | Alternative Fac. Nr | Serie-ID | Time format | Source |
2019-05-14 00:00 | 0,00 | Facility 1 | ABC1 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:00 | |||
2019-05-14 01:00 | 0,00 | Facility 1 | ABC1 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:01 | |||
2019-05-14 02:00 | 0,00 | Facility 1 | ABC1 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:02 | |||
2019-05-14 03:00 | 0,00 | Facility 1 | ABC1 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:03 | |||
2019-05-14 04:00 | 0,33 | Facility 1 | ABC1 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:04 | |||
2019-05-14 05:00 | 1,89 | Facility 1 | ABC1 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:05 | |||
2019-05-14 06:00 | 7,71 | Facility 1 | ABC1 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:06 | |||
2019-05-14 07:00 | 15,35 | Facility 1 | ABC1 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:07 | |||
2019-05-14 08:00 | 21,67 | Facility 1 | ABC1 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:08 | |||
2019-05-14 09:00 | 25,66 | Calculated | Facility 1 | ABC1 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:09 | ||
2019-05-14 10:00 | 27,32 | Calculated | Facility 1 | ABC1 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:10 | ||
2019-05-14 11:00 | 27,07 | Facility 1 | ABC1 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:11 | |||
2019-05-14 12:00 | 25,42 | Facility 1 | ABC1 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:12 | |||
2019-05-14 13:00 | 21,42 | Facility 1 | ABC1 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:13 | |||
2019-05-14 14:00 | 16,47 | Facility 1 | ABC1 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:14 | |||
2019-05-14 15:00 | 9,30 | Facility 1 | ABC1 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:15 | |||
2019-05-14 16:00 | 2,24 | Facility 1 | ABC1 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:16 | |||
2019-05-14 17:00 | 0,80 | Facility 1 | ABC1 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:17 | |||
2019-05-14 18:00 | 0,52 | Facility 1 | ABC1 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:18 | |||
2019-05-14 19:00 | 0,07 | Facility 1 | ABC1 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:19 | |||
2019-05-14 20:00 | 0,00 | Facility 1 | ABC1 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:20 | |||
2019-05-14 00:00 | 0,00 | Facility 2 | ABC2 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:21 | |||
2019-05-14 01:00 | 0,00 | Facility 2 | ABC2 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:22 | |||
2019-05-14 02:00 | 0,00 | Facility 2 | ABC2 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:23 | |||
2019-05-14 03:00 | 0,04 | Facility 2 | ABC2 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:24 | |||
2019-05-14 04:00 | 0,49 | Corrected | Facility 2 | ABC2 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:25 | ||
2019-05-14 05:00 | 2,07 | Facility 2 | ABC2 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:26 | |||
2019-05-14 06:00 | 7,94 | Facility 2 | ABC2 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:27 | |||
2019-05-14 07:00 | 15,67 | Facility 2 | ABC2 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:28 | |||
2019-05-14 08:00 | 23,27 | Facility 2 | ABC2 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:29 | |||
2019-05-14 09:00 | 27,51 | Facility 2 | ABC2 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:30 | |||
2019-05-14 10:00 | 29,11 | Facility 2 | ABC2 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:31 | |||
2019-05-14 11:00 | 28,59 | Facility 2 | ABC2 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:32 | |||
2019-05-14 12:00 | 26,88 | Facility 2 | ABC2 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:33 | |||
2019-05-14 13:00 | 22,73 | Facility 2 | ABC2 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:34 | |||
2019-05-14 14:00 | 17,44 | Facility 2 | ABC2 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:35 | |||
2019-05-14 15:00 | 10,12 | Facility 2 | ABC2 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:36 | |||
2019-05-14 16:00 | 2,75 | Facility 2 | ABC2 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:37 | |||
2019-05-14 17:00 | 1,13 | Facility 2 | ABC2 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:38 | |||
2019-05-14 18:00 | 0,72 | Facility 2 | ABC2 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:39 | |||
2019-05-14 19:00 | 0,14 | Facility 2 | ABC2 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:40 | |||
2019-05-14 20:00 | 0,00 | Facility 2 | ABC2 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:41 | |||
2019-05-14 00:00 | 0,00 | Facility 3 | ABC3 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:42 | |||
2019-05-14 01:00 | 0,00 | Facility 3 | ABC3 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:43 | |||
2019-05-14 02:00 | 0,00 | Facility 3 | ABC3 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:44 | |||
2019-05-14 03:00 | 0,04 | Facility 3 | ABC3 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:45 | |||
2019-05-14 04:00 | 0,49 | Facility 3 | ABC3 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:46 | |||
2019-05-14 05:00 | 2,16 | Facility 3 | ABC3 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:47 | |||
2019-05-14 06:00 | 7,98 | Facility 3 | ABC3 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:48 | |||
2019-05-14 07:00 | 14,82 | Facility 3 | ABC3 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:49 | |||
2019-05-14 08:00 | 19,98 | Facility 3 | ABC3 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:50 | |||
2019-05-14 09:00 | 23,51 | Facility 3 | ABC3 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:51 | |||
2019-05-14 10:00 | 13,49 | Corrected | Facility 3 | ABC3 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:52 | ||
2019-05-14 11:00 | 0,00 | Corrected | Facility 3 | ABC3 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:53 | ||
2019-05-14 12:00 | 0,00 | Corrected | Facility 3 | ABC3 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:54 | ||
2019-05-14 13:00 | 0,00 | Corrected | Facility 3 | ABC3 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:55 | ||
2019-05-14 14:00 | 0,00 | Corrected | Facility 3 | ABC3 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:56 | ||
2019-05-14 15:00 | 0,00 | Corrected | Facility 3 | ABC3 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:57 | ||
2019-05-14 16:00 | 0,00 | Corrected | Facility 3 | ABC3 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:58 | ||
2019-05-14 17:00 | 0,00 | Corrected | Facility 3 | ABC3 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:59 | ||
2019-05-14 18:00 | 0,00 | Corrected | Facility 3 | ABC3 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:60 | ||
2019-05-14 19:00 | 0,00 | Corrected | Facility 3 | ABC3 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:61 | ||
2019-05-14 20:00 | 0,00 | Corrected | Facility 3 | ABC3 | Normal time | Hourlist 2019-05-14 00:00 - 2019-05-22 23:62 |
Solved! Go to Solution.
Here's my M language. I hope this is what you want.
First I duplicated the query
On the first query:
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("nZRNa8MwDIb/SuhZHZbsxPZuXcfYLr107FJ6yDoPAkkLWTLYv5/T+qOkaTGFhLxW9EiyImezma3Kxsxg9lLuqrrq/jK0i+zcQGMDPxq2sImmfd98mtY5Xt6D76LuTLsvu+rXZJZ7yFa3/demrcz87dkaFk9LX5WVFGWs5L1qTPZ9aJuys7bVIOqsq45bu7Vy1xDi9dAPFX2UdT+8WXdl1/8kGAaWGOo5y+coMsYeGbNvGRwf2RU5pvAuiu6i+BQlxnJMCU9xHlyFto/loW3NrjNf0XRJ5ycaQWlHEzAZJBaTVHGiJEg/ABK0iFJNUtLlyoHnztfqwmdDAYomQXUCCaMzcaAA6msZtQNtlmEjy7Le9XV5aglJyDFGc3oUAd3UWGdOFxE0oI+AfNz0cSQMkWJ/FeS+7e6rX+cp7EX4o0YFKJXK89DEyBNInsq7OcMCRGi8BCFSeTdpGrifbmSAlIq7kSMgn9EWn6fS0h8SFZIDJm9deTqP5crkynU41jIgmNo2SvltndHbfw==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [#"Hourlist 2019-05-14 00:00 - 2019-05-22 23:00" = _t, #" " = _t, #" .1" = _t, #" .2" = _t, #" .3" = _t, #"(blank)" = _t, #"(blank).1" = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{{"Hourlist 2019-05-14 00:00 - 2019-05-22 23:00", type text}, {" ", type text}, {" .1", type text}, {" .2", type text}, {" .3", type text}, {"(blank)", type text}, {"(blank).1", type text}}), #"Removed Top Rows" = Table.Skip(#"Changed Type",5), #"Promoted Headers" = Table.PromoteHeaders(#"Removed Top Rows", [PromoteAllScalars=true]), #"Merged Columns" = Table.CombineColumns(#"Promoted Headers",{"Value", "Status"},Combiner.CombineTextByDelimiter(";", QuoteStyle.None),"Facility1"), #"Merged Columns1" = Table.CombineColumns(#"Merged Columns",{"Value_1", "Status_2"},Combiner.CombineTextByDelimiter(";", QuoteStyle.None),"Facility2"), #"Merged Columns2" = Table.CombineColumns(#"Merged Columns1",{"Value_3", "Status_4"},Combiner.CombineTextByDelimiter(";", QuoteStyle.None),"Facility3"), #"Unpivoted Columns" = Table.UnpivotOtherColumns(#"Merged Columns2", {"Hour"}, "Attribute", "Value"), #"Split Column by Delimiter" = Table.SplitColumn(#"Unpivoted Columns", "Value", Splitter.SplitTextByEachDelimiter({";"}, QuoteStyle.Csv, false), {"Value.1", "Value.2"}), #"Sorted Rows" = Table.Sort(#"Split Column by Delimiter",{{"Attribute", Order.Ascending}, {"Hour", Order.Ascending}}), #"Added Index" = Table.AddIndexColumn(#"Sorted Rows", "Index", 0, 1) in #"Added Index"
On the 2nd query:
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("nZRNa8MwDIb/SuhZHZbsxPZuXcfYLr107FJ6yDoPAkkLWTLYv5/T+qOkaTGFhLxW9EiyImezma3Kxsxg9lLuqrrq/jK0i+zcQGMDPxq2sImmfd98mtY5Xt6D76LuTLsvu+rXZJZ7yFa3/demrcz87dkaFk9LX5WVFGWs5L1qTPZ9aJuys7bVIOqsq45bu7Vy1xDi9dAPFX2UdT+8WXdl1/8kGAaWGOo5y+coMsYeGbNvGRwf2RU5pvAuiu6i+BQlxnJMCU9xHlyFto/loW3NrjNf0XRJ5ycaQWlHEzAZJBaTVHGiJEg/ABK0iFJNUtLlyoHnztfqwmdDAYomQXUCCaMzcaAA6msZtQNtlmEjy7Le9XV5aglJyDFGc3oUAd3UWGdOFxE0oI+AfNz0cSQMkWJ/FeS+7e6rX+cp7EX4o0YFKJXK89DEyBNInsq7OcMCRGi8BCFSeTdpGrifbmSAlIq7kSMgn9EWn6fS0h8SFZIDJm9deTqP5crkynU41jIgmNo2SvltndHbfw==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [#"Hourlist 2019-05-14 00:00 - 2019-05-22 23:00" = _t, #" " = _t, #" .1" = _t, #" .2" = _t, #" .3" = _t, #"(blank)" = _t, #"(blank).1" = _t]), #"Demoted Headers" = Table.DemoteHeaders(Source), #"Changed Type" = Table.TransformColumnTypes(#"Demoted Headers",{{"Column1", type text}, {"Column2", type text}, {"Column3", type text}, {"Column4", type text}, {"Column5", type text}, {"Column6", type text}, {"Column7", type text}}), #"Kept First Rows" = Table.FirstN(#"Changed Type",1), #"Removed Other Columns" = Table.SelectColumns(#"Kept First Rows",{"Column1"}), #"Split Column by Delimiter" = Table.SplitColumn(#"Removed Other Columns", "Column1", Splitter.SplitTextByEachDelimiter({":"}, QuoteStyle.Csv, true), {"Column1.1", "Column1.2"}), #"Changed Type1" = Table.TransformColumnTypes(#"Split Column by Delimiter",{{"Column1.1", type text}, {"Column1.2", Int64.Type}}) in #"Changed Type1"
Then I 'merged as new' and on the 3rd query:
let Source = Table.NestedJoin(Table1, {"Hour"}, #"Table1 (2)", {"Column1.1"}, "Table1 (2)", JoinKind.FullOuter), #"Expanded Table1 (2)" = Table.ExpandTableColumn(Source, "Table1 (2)", {"Column1.1"}, {"Table1 (2).Column1.1"}), #"Filled Down" = Table.FillDown(#"Expanded Table1 (2)",{"Table1 (2).Column1.1"}), #"Reordered Columns" = Table.ReorderColumns(#"Filled Down",{"Hour", "Attribute", "Value.1", "Value.2", "Table1 (2).Column1.1", "Index"}), #"Removed Top Rows" = Table.Skip(#"Reordered Columns",1), #"Changed Type" = Table.TransformColumnTypes(#"Removed Top Rows",{{"Index", type text}}), #"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each Text.PadStart(Text.From([Index]),2,"0")), #"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Index"}), #"Merged Columns" = Table.CombineColumns(Table.TransformColumnTypes(#"Removed Columns", {{"Custom", type text}}, "en-GB"),{"Table1 (2).Column1.1", "Custom"},Combiner.CombineTextByDelimiter(":", QuoteStyle.None),"Merged"), #"Reordered Columns1" = Table.ReorderColumns(#"Merged Columns",{"Hour", "Value.1", "Value.2", "Attribute", "Merged"}) in #"Reordered Columns1"
and that should get
There are a few columns missing (they didn't seem to be doing anything). Feel free to add them back in.
Here's my M language. I hope this is what you want.
First I duplicated the query
On the first query:
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("nZRNa8MwDIb/SuhZHZbsxPZuXcfYLr107FJ6yDoPAkkLWTLYv5/T+qOkaTGFhLxW9EiyImezma3Kxsxg9lLuqrrq/jK0i+zcQGMDPxq2sImmfd98mtY5Xt6D76LuTLsvu+rXZJZ7yFa3/demrcz87dkaFk9LX5WVFGWs5L1qTPZ9aJuys7bVIOqsq45bu7Vy1xDi9dAPFX2UdT+8WXdl1/8kGAaWGOo5y+coMsYeGbNvGRwf2RU5pvAuiu6i+BQlxnJMCU9xHlyFto/loW3NrjNf0XRJ5ycaQWlHEzAZJBaTVHGiJEg/ABK0iFJNUtLlyoHnztfqwmdDAYomQXUCCaMzcaAA6msZtQNtlmEjy7Le9XV5aglJyDFGc3oUAd3UWGdOFxE0oI+AfNz0cSQMkWJ/FeS+7e6rX+cp7EX4o0YFKJXK89DEyBNInsq7OcMCRGi8BCFSeTdpGrifbmSAlIq7kSMgn9EWn6fS0h8SFZIDJm9deTqP5crkynU41jIgmNo2SvltndHbfw==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [#"Hourlist 2019-05-14 00:00 - 2019-05-22 23:00" = _t, #" " = _t, #" .1" = _t, #" .2" = _t, #" .3" = _t, #"(blank)" = _t, #"(blank).1" = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{{"Hourlist 2019-05-14 00:00 - 2019-05-22 23:00", type text}, {" ", type text}, {" .1", type text}, {" .2", type text}, {" .3", type text}, {"(blank)", type text}, {"(blank).1", type text}}), #"Removed Top Rows" = Table.Skip(#"Changed Type",5), #"Promoted Headers" = Table.PromoteHeaders(#"Removed Top Rows", [PromoteAllScalars=true]), #"Merged Columns" = Table.CombineColumns(#"Promoted Headers",{"Value", "Status"},Combiner.CombineTextByDelimiter(";", QuoteStyle.None),"Facility1"), #"Merged Columns1" = Table.CombineColumns(#"Merged Columns",{"Value_1", "Status_2"},Combiner.CombineTextByDelimiter(";", QuoteStyle.None),"Facility2"), #"Merged Columns2" = Table.CombineColumns(#"Merged Columns1",{"Value_3", "Status_4"},Combiner.CombineTextByDelimiter(";", QuoteStyle.None),"Facility3"), #"Unpivoted Columns" = Table.UnpivotOtherColumns(#"Merged Columns2", {"Hour"}, "Attribute", "Value"), #"Split Column by Delimiter" = Table.SplitColumn(#"Unpivoted Columns", "Value", Splitter.SplitTextByEachDelimiter({";"}, QuoteStyle.Csv, false), {"Value.1", "Value.2"}), #"Sorted Rows" = Table.Sort(#"Split Column by Delimiter",{{"Attribute", Order.Ascending}, {"Hour", Order.Ascending}}), #"Added Index" = Table.AddIndexColumn(#"Sorted Rows", "Index", 0, 1) in #"Added Index"
On the 2nd query:
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("nZRNa8MwDIb/SuhZHZbsxPZuXcfYLr107FJ6yDoPAkkLWTLYv5/T+qOkaTGFhLxW9EiyImezma3Kxsxg9lLuqrrq/jK0i+zcQGMDPxq2sImmfd98mtY5Xt6D76LuTLsvu+rXZJZ7yFa3/demrcz87dkaFk9LX5WVFGWs5L1qTPZ9aJuys7bVIOqsq45bu7Vy1xDi9dAPFX2UdT+8WXdl1/8kGAaWGOo5y+coMsYeGbNvGRwf2RU5pvAuiu6i+BQlxnJMCU9xHlyFto/loW3NrjNf0XRJ5ycaQWlHEzAZJBaTVHGiJEg/ABK0iFJNUtLlyoHnztfqwmdDAYomQXUCCaMzcaAA6msZtQNtlmEjy7Le9XV5aglJyDFGc3oUAd3UWGdOFxE0oI+AfNz0cSQMkWJ/FeS+7e6rX+cp7EX4o0YFKJXK89DEyBNInsq7OcMCRGi8BCFSeTdpGrifbmSAlIq7kSMgn9EWn6fS0h8SFZIDJm9deTqP5crkynU41jIgmNo2SvltndHbfw==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [#"Hourlist 2019-05-14 00:00 - 2019-05-22 23:00" = _t, #" " = _t, #" .1" = _t, #" .2" = _t, #" .3" = _t, #"(blank)" = _t, #"(blank).1" = _t]), #"Demoted Headers" = Table.DemoteHeaders(Source), #"Changed Type" = Table.TransformColumnTypes(#"Demoted Headers",{{"Column1", type text}, {"Column2", type text}, {"Column3", type text}, {"Column4", type text}, {"Column5", type text}, {"Column6", type text}, {"Column7", type text}}), #"Kept First Rows" = Table.FirstN(#"Changed Type",1), #"Removed Other Columns" = Table.SelectColumns(#"Kept First Rows",{"Column1"}), #"Split Column by Delimiter" = Table.SplitColumn(#"Removed Other Columns", "Column1", Splitter.SplitTextByEachDelimiter({":"}, QuoteStyle.Csv, true), {"Column1.1", "Column1.2"}), #"Changed Type1" = Table.TransformColumnTypes(#"Split Column by Delimiter",{{"Column1.1", type text}, {"Column1.2", Int64.Type}}) in #"Changed Type1"
Then I 'merged as new' and on the 3rd query:
let Source = Table.NestedJoin(Table1, {"Hour"}, #"Table1 (2)", {"Column1.1"}, "Table1 (2)", JoinKind.FullOuter), #"Expanded Table1 (2)" = Table.ExpandTableColumn(Source, "Table1 (2)", {"Column1.1"}, {"Table1 (2).Column1.1"}), #"Filled Down" = Table.FillDown(#"Expanded Table1 (2)",{"Table1 (2).Column1.1"}), #"Reordered Columns" = Table.ReorderColumns(#"Filled Down",{"Hour", "Attribute", "Value.1", "Value.2", "Table1 (2).Column1.1", "Index"}), #"Removed Top Rows" = Table.Skip(#"Reordered Columns",1), #"Changed Type" = Table.TransformColumnTypes(#"Removed Top Rows",{{"Index", type text}}), #"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each Text.PadStart(Text.From([Index]),2,"0")), #"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Index"}), #"Merged Columns" = Table.CombineColumns(Table.TransformColumnTypes(#"Removed Columns", {{"Custom", type text}}, "en-GB"),{"Table1 (2).Column1.1", "Custom"},Combiner.CombineTextByDelimiter(":", QuoteStyle.None),"Merged"), #"Reordered Columns1" = Table.ReorderColumns(#"Merged Columns",{"Hour", "Value.1", "Value.2", "Attribute", "Merged"}) in #"Reordered Columns1"
and that should get
There are a few columns missing (they didn't seem to be doing anything). Feel free to add them back in.
This solves the problem by manually wrangling the set. Optimally I would like to do this to a bunch of different files in a folder with different amount of columns. If you feel like taking a bite at a solution for that, check out my other post:
Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City
Check out the April 2024 Power BI update to learn about new features.