Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.
Hi community,
I am trying to add Index based per document_id, but after specific text string in another column.
My simplified sample data with desired Index Column is this:
document_id | message_text | Desired Index Column |
tKsCu8TQFfBvQAA9qZDS | some message | |
tKsCu8TQFfBvQAA9qZDS | some message | |
tKsCu8TQFfBvQAA9qZDS | some message | |
tKsCu8TQFfBvQAA9qZDS | "connected" | 0 |
tKsCu8TQFfBvQAA9qZDS | some message | 1 |
c8VICY4G3rYUmzcdHMrO | some message | |
c8VICY4G3rYUmzcdHMrO | some message | |
c8VICY4G3rYUmzcdHMrO | some message | |
c8VICY4G3rYUmzcdHMrO | "connected" | 0 |
c8VICY4G3rYUmzcdHMrO | some message | 1 |
c8VICY4G3rYUmzcdHMrO | some message | 2 |
c8VICY4G3rYUmzcdHMrO | some message | 3 |
c8VICY4G3rYUmzcdHMrO | some message | 4 |
c8VICY4G3rYUmzcdHMrO | some message | 5 |
c8VICY4G3rYUmzcdHMrO | some message | 6 |
c8VICY4G3rYUmzcdHMrO | some message | 7 |
c8VICY4G3rYUmzcdHMrO | some message | 8 |
3kJEmSSzqLtnvQ6J3YIo | some message | |
TtnhNTHr9yNzdqCtioFe | some message | |
W4DvaKMXSb4bXxHc7jdw | some message | |
W4DvaKMXSb4bXxHc7jdw | some message | |
W4DvaKMXSb4bXxHc7jdw | some message | |
FX3R0wyA9vN7iy5oFm5P | some message | |
1J0CctLPsKdT4vri1kYB | some message | |
kBJAF409uKjjpvho0XY3 | "connected" | 0 |
kBJAF409uKjjpvho0XY3 | some message | 1 |
kBJAF409uKjjpvho0XY3 | some message | 2 |
kBJAF409uKjjpvho0XY3 | some message | 3 |
kBJAF409uKjjpvho0XY3 | some message | 4 |
kBJAF409uKjjpvho0XY3 | some message | 5 |
kBJAF409uKjjpvho0XY3 | some message | 6 |
I am adding Index with Table.Group and trying use if with Text.Contains, and then Expand, but without success. Anyone can help, please?
#"Grouped Rows" = Table.Group(#"Expanded table1", {"document_id"}, {{"Count", if Table.Contains([message_text], "connected") then each Table.AddIndexColumn(_, "Index",1,1) else ?nothing?, type table [document_id=nullable text, message_text=nullable text,]}}),
#"Expanded Count" = Table.ExpandTableColumn(#"Grouped Rows", "Count", {"message_text", "Index"}, {"message_text", "Index"})
Thanks in advance
Solved! Go to Solution.
You can try this
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("1dJBb4IwGAbgv2I4e8C0TjkCrkNQJoNtdMaDtt0EUjporYNfv8TbMpawzGT6nZ+8eb/kXa8NFUj3ME0i9OroyLat6mUWG0NDCs4GnEm5fWOD0xmbYS99DkdEWTKiGP1VGJk+zV0M70CNH3lLqLes7//RfX3iwspdnQOFf8vjuK0WqtTRjQ/wXHS5RJX7MPFqqwlbWrkqE4h1uWc409tgmcY7uEs/PDLJ6fE08572Lw6l4ME8Nralw0nWjAXi41WXG/mmS9RiJQOaQF1nowI7Xa5wfBtB0zoEef6u98JMMfg2wB9Q37DrcptP", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [document_id = _t, message_text = _t]),
Grouped = Table.Group(Source, {"document_id"}, {{"All", each let t=_, p= List.PositionOf(t[message_text], "connected") in if p<0 then Table.AddColumn(t,"Index",each "") else Table.AddColumn( Table.AddIndexColumn(t,"i", -p,1), "Index", each if [i]<0 then "" else [i]), type table}}),
Expanded = Table.ExpandTableColumn(Grouped, "All", {"message_text", "Index"})
in
Expanded
NewStep=#table(Table.ColumnNames(PreviousStepName)&{"idx"},List.Accumulate(Table.ToRows(PreviousStepName),{},(x,y)=>if x={} then {y&{null}} else let a=List.Last(x) in x&{y&{if y{1}="connected" then 0 else if y{0}=a{0} then a{2}+1 else null}}))
You can try this
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("1dJBb4IwGAbgv2I4e8C0TjkCrkNQJoNtdMaDtt0EUjporYNfv8TbMpawzGT6nZ+8eb/kXa8NFUj3ME0i9OroyLat6mUWG0NDCs4GnEm5fWOD0xmbYS99DkdEWTKiGP1VGJk+zV0M70CNH3lLqLes7//RfX3iwspdnQOFf8vjuK0WqtTRjQ/wXHS5RJX7MPFqqwlbWrkqE4h1uWc409tgmcY7uEs/PDLJ6fE08572Lw6l4ME8Nralw0nWjAXi41WXG/mmS9RiJQOaQF1nowI7Xa5wfBtB0zoEef6u98JMMfg2wB9Q37DrcptP", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [document_id = _t, message_text = _t]),
Grouped = Table.Group(Source, {"document_id"}, {{"All", each let t=_, p= List.PositionOf(t[message_text], "connected") in if p<0 then Table.AddColumn(t,"Index",each "") else Table.AddColumn( Table.AddIndexColumn(t,"i", -p,1), "Index", each if [i]<0 then "" else [i]), type table}}),
Expanded = Table.ExpandTableColumn(Grouped, "All", {"message_text", "Index"})
in
Expanded
Hi @Jakinta ,
thanks for your reply. It works perfectly! But only with example from my post 😞 I am trying to implement it but it show this error
This is my PQ M (shortened Source):
let
Source = Value.NativeQuery(GoogleBigQuery.Database([BillingProject="00000000000"]){[Name="production"]}[Data], "select#(lf)c.document_id,#(lf)timestamp_seconds(cast(json_extract.............shortened................#(lf)order by threat_id, time_of_message asc", null, [EnableFolding=true], let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [document_id = _t, message_text = _t]),
#"Sorted Rows" = Table.Sort(Source,{{"document_id", Order.Ascending}, {"time_of_message", Order.Ascending}}),
#"Added Index" = Table.AddIndexColumn(#"Sorted Rows", "Index", 0, 1, Int64.Type),
#"Replaced Value" = Table.ReplaceValue(#"Added Index","""","",Replacer.ReplaceText,{"message_from_name"}),
#"Merged Queries" = Table.NestedJoin(#"Replaced Value", {"threat_id"}, #"table01", {"threat_id"}, "table01", JoinKind.LeftOuter),
#"Expanded table01" = Table.ExpandTableColumn(#"Merged Queries", "table01", {"price"}, {"table01.price"}),
Grouped = Table.Group(Source, {"document_id"}, {{"All", each let t=_, p= List.PositionOf(t[message_text], "connected") in if p<0 then Table.AddColumn(t,"Index2",each "") else Table.AddColumn( Table.AddIndexColumn(t,"i", -p,1), "Index2", each if [i]<0 then "" else [i]), type table}}),
Expanded = Table.ExpandTableColumn(Grouped, "All", {"message_text", "Index2"})
in
Expanded
with part from you (I have already one Index, co from you is Index2). Can I kindly ask you to look on it?
Thanks
Divous
Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City
Check out the April 2024 Power BI update to learn about new features.
User | Count |
---|---|
100 | |
54 | |
21 | |
12 | |
11 |