Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.
Hi all, I am new to Power BI but fairly experienced with R.
I am training myself by working through the exercises in the Dashboard in a Day course here: https://powerbi.microsoft.com/en-us/diad/
I noticed there is an option in the Query Editor Transform bar to run an R script to transform data instead of/as well as using the GUI.
So I tried to create a very simple new column using the R script:
dataset$double_revenue <- dataset$Revenue*2
This resulted in the error message:
"Formula.Firewall: Query 'Sales' (step 'Run R Script') references other queries or steps, so it may not directly access a data source. Please rebuild this data combination."
The automatically-generated M code for the query is as such (This is my first experience with M so do not know how to debug this):
let
Source = Csv.Document(File.Contents("C:\Users\mydata\Data\USSales\sales.csv"),[Delimiter=",", Columns=5, Encoding=1252, QuoteStyle=QuoteStyle.None]),
#"Promoted Headers" = Table.PromoteHeaders(Source, [PromoteAllScalars=true]),
#"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"ProductID", Int64.Type}, {"Date", type date}, {"Zip", type text}, {"Units", Int64.Type}, {"Revenue", type number}}),
#"Appended Query" = Table.Combine({#"Changed Type", #"International Sales"}),
#"Added Conditional Column" = Table.AddColumn(#"Appended Query", "CountryName", each if [Country] = null then "USA" else [Country]),
#"Removed Columns" = Table.RemoveColumns(#"Added Conditional Column",{"Country"}),
#"Renamed Columns" = Table.RenameColumns(#"Removed Columns",{{"CountryName", "Country"}}),
#"Changed Type1" = Table.TransformColumnTypes(#"Renamed Columns",{{"Country", type text}, {"Revenue", Currency.Type}}),
#"Filtered Rows" = Table.SelectRows(#"Changed Type1", each true),
#"Filtered Rows1" = Table.SelectRows(#"Filtered Rows", each Date.IsInPreviousNYears([Date], 3)),
#"Run R Script" = R.Execute("# 'dataset' holds the input data for this script#(lf)#(lf)dataset$double_revenue <- dataset$Revenue*2",[dataset=#"Filtered Rows1"])
in
#"Run R Script"
I have read the information here for this error message: https://www.excelguru.ca/blog/2015/03/11/power-query-errors-please-rebuild-this-data-combination/ but this is specific to merges and I don't know how to apply it to the issue with this R script.
I am aware that the R script references the query; I wanted to use R code to transform a column in a query that I had already imported and transformed the data for rather than running all of the import and transformation steps in R (in this case I would have used Get Data -> R Script instead).
How do I get this query to work with the R code transform step in it?
@RMel I was having the same problem referenced here and couldnt for the life of me figure it out. Found the following post which highlighted the solution (at least for me).
By changing this setting on the file to ignore privacy levels it worked for me. I am sure there are ramifications of publishing models with this option unchecked but for now I can at leaset continue with using R to perform necessary transformations on existing queries.
https://community.powerbi.com/t5/Desktop/Formula-Firewall-Query-references-other-queries-or-steps/td...
Hi @RMel ,
You could try below Code to see whether it work or not (ds[[1]] is the first column in dataset), this works well when in single dataset
# 'dataset' holds the input data for this script
ds <- dataset
ds$progress <- ds[[1]]*100
dataset <- ds
By the way, I have other error information(can't convert to number when I re-produce your design), I am not sure whether this is caused by query combine or others, and I will test this in my environment, I will inform you as soon as possible.
Best Regards,
Zoe Zhi
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Hi @dax,
I tried this but unfortunately I am still getting the same error!
Thanks!
You need to return a dataframe back to the query. Does this work?
dataset$double_revenue <- dataset$Revenue*2
dataset
If this works for you, please mark it as solution. Kudos are appreciated too. Please let me know if not.
Regards,
Pat
To learn more about Power BI, follow me on Twitter or subscribe on YouTube.
Hi @mahoneypat,
Unfortunately I am still getting the same error message with this R script too.
Thanks!
My bad. It's been a while. You need to send the resulting dataframe to output
# 'dataset' holds the input data for this script
dataset$newcolumn <- dataset$Number * 2
output <- dataset
I did a test query, if you are interested.
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WclTSUTJUitWJVnICsozALGcgy1gpNhYA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [Letter = _t, Number = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Letter", type text}, {"Number", Int64.Type}}),
#"Run R script" = R.Execute("# 'dataset' holds the input data for this script#(lf)dataset$newcolumn <- dataset$Number * 2#(lf)output <- dataset",[dataset=#"Changed Type"]),
#"""output""" = #"Run R script"{[Name="output"]}[Value]
in
#"""output"""
If this works for you, please mark it as solution. Kudos are appreciated too. Please let me know if not.
Regards,
Pat
To learn more about Power BI, follow me on Twitter or subscribe on YouTube.
Hi @mahoneypat,
I tried this and still got the same error. I wondered if this is because I merged this query with another one earlier, so I tried it with a different query that I had not merged previously instead.
This time I received a prompt saying "Information is required about data privacy."
I clicked "Ignore Privacy level checks for this file. Ignoring privacy level checks could expose sensitive or confidential data to an unauthorised person." and clicked "Save" (I wouldn't be able to do this for business data due to security issues but tried it for the purposes of testing.)
Then I received a different error message that starts with (I can't add the whole thing as it exceeds 20,000 characters):
Feedback Type:
Frown (Error)
Error Message:
Object reference not set to an instance of an object.
Stack Trace:
Microsoft.Mashup.Evaluator.Interface.ErrorException: Object reference not set to an instance of an object. ---> Microsoft.Mashup.Evaluator.Interface.ErrorException: Object reference not set to an instance of an object. ---> Microsoft.Mashup.Evaluator.Interface.ErrorException: Object reference not set to an instance of an object. ---> Microsoft.Mashup.Evaluator.Interface.ErrorException: Object reference not set to an instance of an object. ---> Microsoft.Mashup.Evaluator.Interface.ErrorException: Object reference not set to an instance of an object. ---> System.NullReferenceException: Object reference not set to an instance of an object. ---> System.NullReferenceException: Object reference not set to an instance of an object.
at Microsoft.PowerBI.Radio.RConnection.get_ServerVersion()
at Microsoft.Mashup.Engine1.Library.Common.DelegatingDbConnection.get_ServerVersion()
at Microsoft.Mashup.Engine1.Library.Common.DelegatingDbConnection.get_ServerVersion()
at Microsoft.Mashup.Engine1.Library.Common.DbEnvironment.LoadServerVersion()
at Microsoft.Mashup.Engine1.Library.Common.DbEnvironment.LoadCachedServerVersion()
at Microsoft.Mashup.Engine1.Library.Common.DbEnvironment.CreateConnection()
at Microsoft.Mashup.Engine1.Library.Common.DbValueBuilder.ConnectionReader..ctor(DbEnvironment environment, Func`2 readerWrapper)
at Microsoft.Mashup.Engine1.Library.Common.NativeQueryTableValue.Initialize()
at Microsoft.Mashup.Engine1.Library.Common.NativeQueryTableValue.EnsureInitialized()
at Microsoft.Mashup.Engine1.Library.Common.NativeQueryTableValue.GetEnumerator()
at Microsoft.Mashup.Engine1.Runtime.TableValue.get_Item(Int32 index)
at Microsoft.Mashup.Engine1.Runtime.TableValue.get_Item(Value key)
at Microsoft.Mashup.Engine1.Runtime.Library.List.ElementWithListCheckFunctionValue.Invoke(Value collection, Value key)
at Microsoft.Mashup.Engine1.Language.FunctionInvocationInstruction2.Execute(MembersFrame1& frame)
at Microsoft.Mashup.Engine1.Language.DebugInstruction.Execute(MembersFrame1& frame)
at Microsoft.Mashup.Engine1.Language.FieldAccessInstruction.Execute(MembersFrame1& frame)
at Microsoft.Mashup.Engine1.Language.DebugInstruction.Execute(MembersFrame1& frame)
at Microsoft.Mashup.Engine1.Language.InstructionInvocationInstruction1.Execute(MembersFrame1& frame)
at Microsoft.Mashup.Engine1.Language.DebugInstruction.Execute(MembersFrame1& frame)
at Microsoft.Mashup.Engine1.Language.MembersFunctionValue1.Invoke(Value arg0)
at Microsoft.Mashup.Engine1.Language.RecordInstruction.RuntimeRecordValue.Force(Int32 index)
at Microsoft.Mashup.Engine1.Language.RecordInstruction.RuntimeRecordValue.get_Item(Int32 index)
at Microsoft.Mashup.Engine1.Runtime.RecordValue.get_Item(String field)
at Microsoft.Mashup.Engine1.Language.FieldAccessInstruction.Execute(MembersFrame1& frame)
at Microsoft.Mashup.Engine1.Language.InstructionInvocationInstruction1.Execute(MembersFrame1& frame)
at Microsoft.Mashup.Engine1.Language.DebugInstruction.Execute(MembersFrame1& frame)
at Microsoft.Mashup.Engine1.Language.Instruction.Execute(MembersFrame1& frame, Instruction[] instructions)
at Microsoft.Mashup.Engine1.Language.InstructionInvocationInstruction.Execute(MembersFrame1& frame)
at Microsoft.Mashup.Engine1.Language.DebugInstruction.Execute(MembersFrame1& frame)
at Microsoft.Mashup.Engine1.Language.MembersFunctionValue1.Invoke(Value arg0)
at Microsoft.Mashup.Engine1.Language.RecordInstruction.RuntimeRecordValue.Force(Int32 index)
at Microsoft.Mashup.Engine1.Language.RecordInstruction.RuntimeRecordValue.get_Item(Int32 index)
at Microsoft.Mashup.Engine1.Runtime.RecordValue.get_Item(String field)
at Microsoft.Mashup.Engine1.Language.FieldAccessInstruction.Execute(Value frame)
at Microsoft.Mashup.Engine1.Language.IfInstruction.Execute(Value frame)
at Microsoft.Mashup.Engine1.Language.DebugInstruction.Execute(Value frame)
at Microsoft.Mashup.Engine1.Language.RuntimeFunctionValue1.Invoke(Value arg0)
at Microsoft.Mashup.Engine1.Language.RecordInstruction.RuntimeRecordValue.Force(Int32 index)
at Microsoft.Mashup.Engine1.Language.RecordInstruction.RuntimeRecordValue.get_Item(Int32 index)
at Microsoft.Mashup.Engine1.Runtime.RecordValue.get_Item(String field)
at Microsoft.Mashup.Engine1.Language.FieldAccessInstruction.Execute(MembersFrame2& frame)
at Microsoft.Mashup.Engine1.Language.MembersFunctionValue2.Invoke(Value arg0, Value arg1)
at Microsoft.Mashup.Engine1.Language.InstructionInvocationInstruction2.Execute(MembersFrame2& frame)
at Microsoft.Mashup.Engine1.Language.FunctionInvocationInstruction2.Execute(MembersFrame2& frame)
at Microsoft.Mashup.Engine1.Language.MembersFunctionValue2.Invoke(Value arg0, Value arg1)
at Microsoft.Mashup.Engine1.Runtime.FunctionValue.MetaTypeFunctionValue.Invoke(Value arg0, Value arg1)
at Microsoft.Mashup.Engine1.Language.InstructionInvocationInstruction2.Execute(MembersFrame2& frame)
at Microsoft.Mashup.Engine1.Language.DebugInstruction.Execute(MembersFrame2& frame)
at Microsoft.Mashup.Engine1.Language.MembersFunctionValue2.Invoke(Value arg0, Value arg1)
at Microsoft.Mashup.Engine1.Language.InstructionInvocationInstruction2.Execute(MembersFrame2& frame)
at Microsoft.Mashup.Engine1.Language.FunctionInvocationInstruction2.Execute(MembersFrame2& frame)
at Microsoft.Mashup.Engine1.Language.MembersFunctionValue2.Invoke(Value arg0, Value arg1)
at Microsoft.Mashup.Engine1.Runtime.FunctionValue.Invoke(Value[] args)
at Microsoft.Mashup.Engine1.Runtime.FunctionValue.MetaTypeFunctionValue.Invoke(Value[] args)
at Microsoft.Mashup.Engine1.Runtime.Extensibility.HostRelinkingFunctionValue.ExtensionFunctionValue.InvokeN(Value[] args)
at Microsoft.Mashup.Engine1.Runtime.NativeFunctionValueN.Invoke(Value[] args)
at Microsoft.Mashup.Engine1.Runtime.NativeFunctionValueN.Invoke(Value arg0, Value arg1)
at Microsoft.Mashup.Engine1.Language.InstructionInvocationInstruction2.Execute(MembersFrame1& frame)
at Microsoft.Mashup.Engine1.Language.DebugInstruction.Execute(MembersFrame1& frame)
at Microsoft.Mashup.Engine1.Language.FunctionInvocationInstruction2.Execute(MembersFrame1& frame)
at Microsoft.Mashup.Engine1.Language.DebugInstruction.Execute(MembersFrame1& frame)
at Microsoft.Mashup.Engine1.Language.InstructionInvocationInstruction2.Execute(MembersFrame1& frame)
at Microsoft.Mashup.Engine1.Language.DebugInstruction.Execute(MembersFrame1& frame)
at Microsoft.Mashup.Engine1.Language.MembersFunctionValue1.Invoke(Value arg0)
at Microsoft.Mashup.Engine1.Language.RecordInstruction.RuntimeRecordValue.Force(Int32 index)
at Microsoft.Mashup.Engine1.Language.RecordInstruction.RuntimeRecordValue.get_Item(Int32 index)
at Microsoft.Mashup.Engine1.Runtime.RecordValue.get_Item(String field)
at Microsoft.Mashup.Engine1.Language.FieldAccessInstruction.Execute(MembersFrame1& frame)
at Microsoft.Mashup.Engine1.Language.MembersFunctionValue1.Invoke(Value arg0)
at Microsoft.Mashup.Engine1.Language.RecordInstruction.RuntimeRecordValue.Force(Int32 index)
at Microsoft.Mashup.Engine1.Language.RecordInstruction.RuntimeRecordValue.get_Item(Int32 index)
at Microsoft.Mashup.Engine1.Runtime.RecordValue.get_Item(Value key)
at Microsoft.Mashup.Engine1.Runtime.Library.Linker.BindFunctionValue.TypedInvoke(RecordValue environment, Value section, TextValue name)
at Microsoft.Mashup.Engine1.Runtime.NativeFunctionValue3`4.Invoke(Value arg0, Value arg1, Value arg2)
at Microsoft.Mashup.Engine1.Runtime.FunctionValue.Invoke(Value[] args)
at Microsoft.Mashup.Engine1.Language.InstructionInvocationInstruction.Execute(MembersFrame0& frame)
at Microsoft.Mashup.Engine1.Language.MembersFunctionValue0.Invoke()
at Microsoft.Mashup.Engine1.Language.ListInstruction.RuntimeListValue.Force(Int32 index)
at Microsoft.Mashup.Engine1.Language.ListInstruction.RuntimeListValue.get_Item(Int32 index)
at Microsoft.Mashup.Engine1.Runtime.ListValue.get_Item(Value key)
at Microsoft.Mashup.Engine1.Runtime.Library.List.ElementWithListCheckFunctionValue.Invoke(Value collection, Value key)
at Microsoft.Mashup.Engine1.Language.FunctionInvocationInstruction2.Execute(MembersFrame1& frame)
at Microsoft.Mashup.Engine1.Language.MembersFunctionValue1.Invoke(Value arg0)
at Microsoft.Mashup.Engine1.Language.RecordInstruction.RuntimeRecordValue.Force(Int32 index)
at Microsoft.Mashup.Engine1.Language.RecordInstruction.RuntimeRecordValue.get_Item(Int32 index)
at Microsoft.Mashup.Engine1.Runtime.RecordValue.get_Item(String field)
at Microsoft.Mashup.Engine1.Language.FieldAccessInstruction.Execute(MembersFrame1& frame)
at Microsoft.Mashup.Engine1.Language.FunctionInvocationInstruction2.Execute(MembersFrame1& frame)
at Microsoft.Mashup.Engine1.Language.Instruction.ExecuteCondition(MembersFrame1& frame)
at Microsoft.Mashup.Engine1.Language.DebugInstruction.ExecuteCondition(MembersFrame1& frame)
at Microsoft.Mashup.Engine1.Language.IfInstruction.Execute(MembersFrame1& frame)
at Microsoft.Mashup.Engine1.Language.DebugInstruction.Execute(MembersFrame1& frame)
at Microsoft.Mashup.Engine1.Language.MembersFunctionValue1.Invoke(Value arg0)
at Microsoft.Mashup.Engine1.Language.RecordInstruction.RuntimeRecordValue.Force(Int32 index)
at Microsoft.Mashup.Engine1.Language.RecordInstruction.RuntimeRecordValue.get_Item(Int32 index)
at Microsoft.Mashup.Engine1.Runtime.RecordValue.get_Item(String field)
at Microsoft.Mashup.Engine1.Language.FieldAccessInstruction.Execute(MembersFrame0& frame)
at Microsoft.Mashup.Engine1.Language.MembersFunctionValue0.Invoke()
at Microsoft.Mashup.Engine1.Runtime.FunctionValue.Invoke(Value[] args)
at Microsoft.Mashup.Engine1.Engine.Microsoft.Mashup.Engine.Interface.IEngine.Invoke(IValue function, IValue[] arguments)
at Microsoft.Mashup.Evaluator.SimpleDocumentEvaluator.BeginGetResult(DocumentEvaluationParameters parameters, Action`1 callback)
--- End of inner exception stack trace ---
Any ideas as to what this means?
Please see this post.
http://officeusers.blogspot.com/2018/06/using-r-in-power-bi-object-reference.html
Are you using 32 bit R Studio?
Regards
Pat
To learn more about Power BI, follow me on Twitter or subscribe on YouTube.
Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City
Check out the April 2024 Power BI update to learn about new features.
User | Count |
---|---|
111 | |
94 | |
80 | |
68 | |
59 |
User | Count |
---|---|
150 | |
119 | |
104 | |
87 | |
67 |