Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.

Reply
DavidPratten
Frequent Visitor

Gathering data from an API without triggering "references other queries or steps"

Hi,


I've read though a dozen articles on this and haven't found this scenario yet.  Your assistance, or pointers to prior solutions, will be appreciated.  Thanks in advance.  

 

I'm gathering data from a single data source (it happens to be Atlassian Jira Cloud REST API). The API frequently requires the caller to provide data to the API in order to get data.  How is this use case supported?

 

The obvious solution doesn't work.  The following code triggers the "references other queries or steps, so it may not directly access a data source." error. 

 

// Fetch the Fix Versions for a list of projects.  
// The projects are fetched by the #"Projects" query from the same data source.

let
Next = Table.AddColumn(#"Projects", "Versions", each Json.Document(Web.Contents("...atlassian.net",
[RelativePath = "/rest/api/3/project/"&Number.ToText([project.id])&"/versions",
Headers= [#"Content-Type"="application/json"]]
)))
in
Next

where #"Projects" is a reference to a query which directly access the same "...atlassian.net" data source

 

Background

  • The API doesn't provide for requesting a list of all Fix Versions without providing a project id, so that is not an option.
  • I'm preparing a PowerBI dataset for uploading to the Online Service and so the "Ignore the Privacy levels and potentially improve performance" must remain Off.  See Understand Power BI Desktop privacy levels - Power BI | Microsoft Docs
  • The data source is set to Privacy "Organisational".
  • And, if relevant, here is the source of the #"Projects" Query:
let
readPath = "/rest/api/3/project/search",

// This is how you handle paginated rest api's in Jira! Simples.
fetchbatch = (thisStartAt) =>
let
data = Json.Document(Web.Contents("https://...atlassian.net/",
[RelativePath = readPath,
Query = [startAt=Number.ToText(thisStartAt)],
Headers= [
#"Content-Type"="application/json"]]
)),
res = if not Record.Field(data,"isLast") then List.Combine({data[values],@fetchbatch(Record.Field(data,"maxResults")+Record.Field(data,"startAt"))}) else data[values]
in
res,
#"Converted to Table" = Table.FromList(fetchbatch(0), Splitter.SplitByNothing(), null, null, ExtraValues.Error),

// expand project fields
#"Expanded Column1" = Table.ExpandRecordColumn(#"Converted to Table", "Column1", {"id", "key", "name", "projectCategory"}, {"project.id", "project.key", "project.name", "project.projectCategory"}),
// expand category field
#"Expanded projectCategory" = Table.ExpandRecordColumn(#"Expanded Column1", "project.projectCategory", {"name"}, {"project.projectCategory.name"}),
// type project id to integer
#"Changed Type" = Table.TransformColumnTypes(#"Expanded projectCategory",{{"project.id", Int64.Type}})
in
#"Changed Type"

 

David

 

4 REPLIES 4
DavidPratten
Frequent Visitor

Hi @v-easonf-msft 

 

Thanks for your response.  Unfortunately they do not answer my question above.

 

Article 1. Feedback wanted: Behind the Scenes of the Data Privacy Firewall  - doesn't explain how to rewrite my queries above (from the same data source) to be successful on the PowerBI Service.

 

Article 2. The work around approach in Power Query Errors: Please Rebuild This Data Combination  doesn't work with the PowerBI Service.

 

I would value code examples that show how to join data from compatible privacy sources when using the PowerBI Service.

 

Thanks


David

 

 

 

 

DavidPratten
Frequent Visitor

@v-easonf-msft thanks for the pointers. I had read both these articles in my prior research.

 

While these articles do explain the mechanism that prevents compatible sources (even same source) from being joined, that is not my question.

 

My question is this.  In what way does the PowerBI Service support the use case of joining tables from compatible sources?   

 

Code examples to show how this use case is supported would be helpful.

 

Puzzled.

 

David

DavidPratten
Frequent Visitor

Addendum. Just to reference the official docs on this.  Why not do both?  (https://docs.microsoft.com/en-us/power-query/dataprivacyfirewall#why-not-do-both)  Gives a justification for the limitation but sadly doesn't explain how to join data from single data source with itself.

Hi, @DavidPratten 

Please check if the below threads could help.

Power Query Errors: Please Rebuild This Data Combination 

Feedback wanted: Behind the Scenes of the Data Privacy Firewall 

 

Best Regards,
Community Support Team _ Eason

Helpful resources

Announcements
Microsoft Fabric Learn Together

Microsoft Fabric Learn Together

Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City

PBI_APRIL_CAROUSEL1

Power BI Monthly Update - April 2024

Check out the April 2024 Power BI update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.