Power BI Dataflow to Azure Data Lake Gen2 Setup

[b5lurker]

Excited to see all of the new Power BI Dataflow capabilities especially with using Azure Data Lake Gen2 as the storage location for those Dataflows.

But, I have been trying to get it setup using the Microsoft provided step-by-step guide on: https://docs.microsoft.com/en-us/power-bi/service-dataflows-connect-azure-data-lake-storage-gen2. The specific steps that I have been unable to complete are in the 'Grant Power BI permissions to the file system' section steps #7 and #8. I have only tried it using Azure Storage Explorer v1.6 since that is the method shown in the documentation.

When I try to Add the Object IDs for the Power BI Service and Power Query Online as shown in the screenshots I only get an error saying that it cannot find those User IDs/Groups. I have attached the full error message captured in ASE to see if that might help track it down. I did this with my elevated permissions Azure account and also had our admin grant me full Admin rights over our tenant and could not get it working.

Has anyone been able to complete the steps to grant the Power BI Service and Power Query Online applications access to the powerbi blob container in their Azure Data Lake Store Gen2? I even re-created the ADLSg2 yesterday since I had it created in the original private preview. Without these steps I have not been able to complete the process in the Power BI Admin screen to connect Power BI up to the Data Lake for Dataflows.

Thanks,

Steve

Below is the error message from ASE when trying to Add the Power BI Service Object ID in the Manage Access dialog:

FROM CHILD PROCESS 24608: {"id":"MessagePassingHostProxy151","messageType":"FunctionResponse","response":{"type":"error","error":"{\n  \"message\": \"{\\\"message\\\":\\\"HTTP ERROR 404: Not Found\\\",\\\"response\\\":{\\\"statusCode\\\":404,\\\"body\\\":\\\"{\\\\\\\"odata.error\\\\\\\":{\\\\\\\"code\\\\\\\":\\\\\\\"Request_ResourceNotFound\\\\\\\",\\\\\\\"message\\\\\\\":{\\\\\\\"lang\\\\\\\":\\\\\\\"en\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"Resource 'ee4bcd73-1e25-4154-befe-f8180e3f14d5' does not exist or one of its queried reference-property objects are not present.\\\\\\\"},\\\\\\\"requestId\\\\\\\":\\\\\\\"724657f5-7d0c-4896-9230-5226b96ff9b6\\\\\\\",\\\\\\\"date\\\\\\\":\\\\\\\"2018-12-11T16:34:51\\\\\\\"}}\\\",\\\"headers\\\":{\\\"cache-control\\\":\\\"no-cache\\\",\\\"pragma\\\":\\\"no-cache\\\",\\\"content-type\\\":\\\"application/json; odata=minimalmetadata; streaming=true; charset=utf-8\\\",\\\"expires\\\":\\\"-1\\\",\\\"server\\\":\\\"Microsoft-IIS/10.0\\\",\\\"ocp-aad-diagnostics-server-name\\\":\\\"QAcXmVYUoJaUSrw2ftJmfZfMjTyUjtSkz5dJvbTYsQ0=\\\",\\\"request-id\\\":\\\"724657f5-7d0c-4896-9230-5226b96ff9b6\\\",\\\"client-request-id\\\":\\\"587c72a6-407f-4024-95b6-fbc710401123\\\",\\\"x-ms-dirapi-data-contract-version\\\":\\\"1.6\\\",\\\"ocp-aad-session-key\\\":\\\"hqhNfIg59I6k-OSo-mWlGPFJUAeZDoSngtOC2dH4v4szDJot9OmJGoysNRoVpgZKj_T_ctHrILdmnh-x_E_1VTEIkhcExHCXJk-BPGZKyTzxWQqqAJYh7r3biDvlbfg1.tMDQP1ZfH1teuhHoVGVcULDVzDSD4PiGem3E2On_fFM\\\",\\\"dataserviceversion\\\":\\\"3.0;\\\",\\\"strict-transport-security\\\":\\\"max-age=31536000; includeSubDomains\\\",\\\"access-control-allow-origin\\\":\\\"*\\\",\\\"x-aspnet-version\\\":\\\"4.0.30319\\\",\\\"x-powered-by\\\":\\\"ASP.NET\\\",\\\"duration\\\":\\\"430184\\\",\\\"date\\\":\\\"Tue, 11 Dec 2018 16:34:51 GMT\\\",\\\"connection\\\":\\\"close\\\",\\\"content-length\\\":\\\"294\\\"},\\\"request\\\":{\\\"uri\\\":{\\\"protocol\\\":\\\"https:\\\",\\\"slashes\\\":true,\\\"auth\\\":null,\\\"host\\\":\\\"graph.windows.net\\\",\\\"port\\\":443,\\\"hostname\\\":\\\"graph.windows.net\\\",\\\"hash\\\":null,\\\"search\\\":\\\"?api-version=1.6\\\",\\\"query\\\":\\\"api-version=1.6\\\",\\\"pathname\\\":\\\"/cb2bab3d-7d90-44ea-9e31-531011b1213d/users/ee4bcd73-1e25-4154-befe-f8180e3f14d5\\\",\\\"path\\\":\\\"/cb2bab3d-7d90-44ea-9e31-531011b1213d/users/ee4bcd73-1e25-4154-befe-f8180e3f14d5?api-version=1.6\\\",\\\"href\\\":\\\"https://graph.windows.net/cb2bab3d-7d90-44ea-9e31-531011b1213d/users/ee4bcd73-1e25-4154-befe-f8180e3f14d5?api-version=1.6\\\"},\\\"method\\\":\\\"get\\\",\\\"headers\\\":{\\\"Content-Type\\\":\\\"application/json\\\",\\\"Authorization\\\":\\\"Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6IndVTG1ZZnNxZFF1V3RWXy1oeFZ0REpKWk00USIsImtpZCI6IndVTG1ZZnNxZFF1V3RWXy1oeFZ0REpKWk00USJ9.eyJhdWQiOiJodHRwczovL2dyYXBoLndpbmRvd3MubmV0LyIsImlzcyI6Imh0dHBzOi8vc3RzLndpbmRvd3MubmV0L2NiMmJhYjNkLTdkOTAtNDRlYS05ZTMxLTUzMTAxMWIxMjEzZC8iLCJpYXQiOjE1NDQ1NDU3OTEsIm5iZiI6MTU0NDU0NTc5MSwiZXhwIjoxNTQ0NTQ5NjkxLCJhY3IiOiIxIiwiYWlvIjoiQVZRQXEvOEpBQUFBTExXL2dQcVV2ZmhjODM1MGtrYXhKdVFsMGpUcU00N3dwTG14WGo3b05oZGJHRTRYTjZXdk01VFJYanhuWm9jdzh2Z2I3emZDWXR6ZWw5NFo4WHRsd0RKZ3l0LzFqSlFWbWZzNGlVZm44STQ9IiwiYW1yIjpbInB3ZCIsIm1mYSJdLCJhcHBpZCI6IjA0YjA3Nzk1LThkZGItNDYxYS1iYmVlLTAyZjllMWJmN2I0NiIsImFwcGlkYWNyIjoiMCIsImZhbWlseV9uYW1lIjoiV2FrZSIsImdpdmVuX25hbWUiOiIhU3RldmUiLCJpcGFkZHIiOiIyMDYuODAuMjA5LjE5NiIsIm5hbWUiOiIhU3RldmUgV2FrZSIsIm9pZCI6ImZjOTAxZTNkLWE0MGQtNDgzOC05MDZlLWE5Y2UwYTk0ZTk0MyIsIm9ucHJlbV9zaWQiOiJTLTEtNS0yMS0yMTQ3MjQyNzI2LTM2MzMxNTgwLTE4ODQ0MTQ0NC0xNzI1NTkiLCJwdWlkIjoiMTAwMzdGRkVBODFDNTIyNyIsInNjcCI6IjYyZTkwMzk0LTY5ZjUtNDIzNy05MTkwLTAxMjE3NzE0NWUxMCIsInN1YiI6Ilo0TjlINTQ4alBPNUlGSVE0dS1FU01OdllHRE02WndNVEs3dm80LVljSzQiLCJ0ZW5hbnRfcmVnaW9uX3Njb3BlIjoiTkEiLCJ0aWQiOiJjYjJiYWIzZC03ZDkwLTQ0ZWEtOWUzMS01MzEwMTFiMTIxM2QiLCJ1bmlxdWVfbmFtZSI6IiFzd2FrZUBicnduY2FsZC5jb20iLCJ1cG4iOiIhc3dha2VAYnJ3bmNhbGQuY29tIiwidXRpIjoiOGVCcThEa0o4VVNrT05RQ2dZa2tBQSIsInZlciI6IjEuMCJ9.DVQ1-dpeAfEPX9Lf62JuVtheaAiTNECZpTfoihlL93TKFzEJW-N8alEwDLAfkGTwvj92GRgi7MThS-HqWGfAla4C0ntKxuzq8NhTeGEuACNXVUDNO5mtfYi1W2jJDJirduMSOgdNkkBqCnFvVMqM49cUAYEaPOD-Wn-Fb-U_L3K-hOaZ2EL2_aFLXpm5w-JJs9MPQXcoOSR1rT1x5L9x7At0hIn-A97RzzabFTAMfoolAVDymjLrT2It5jf1qn1vZ1aGXBZPLH_uF7_Aq_v_gTPECuuj-B2rJV3FokQHHCYu9iNFl7lt95ZP_lv_Q9nH01QNp-xjxyHfTrub7iCYCA\\\",\\\"content-length\\\":0}}},\\\"body\\\":\\\"{\\\\\\\"odata.error\\\\\\\":{\\\\\\\"code\\\\\\\":\\\\\\\"Request_ResourceNotFound\\\\\\\",\\\\\\\"message\\\\\\\":{\\\\\\\"lang\\\\\\\":\\\\\\\"en\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"Resource 'ee4bcd73-1e25-4154-befe-f8180e3f14d5' does not exist or one of its queried reference-property objects are not present.\\\\\\\"},\\\\\\\"requestId\\\\\\\":\\\\\\\"724657f5-7d0c-4896-9230-5226b96ff9b6\\\\\\\",\\\\\\\"date\\\\\\\":\\\\\\\"2018-12-11T16:34:51\\\\\\\"}}\\\"}\"\n}"}}

[joshbrownwsna]

Hi Everyone,

Posting in case anyone has linked their account before they have the ACLs in place as I have. I can answer the questions I posed above.

As I noted above I had linked our Powr BI account to our gen 2 storage account before I found this thread. However, since we had not enabled any of our workspaces to use the data lake storage account, our exiisting Dataflows have continued to write to the Power BI storage.

Said another way using Dataflows with Data Lake storage takes 5 steps:

1. Create a Gen2 Storage account
2. Configure the storage account so that the Powe BI service (power query, power bi service, power bi premium) can read and write to the storage account. This is done in Azure and the storage explorer (once it’s updated).
3. Use the admin panel in Power Bi web app to Enable/link your Power BI instance to the storage account.
4. Use the admin panel in Power BI to provide workspace admins the ability to use datalake storage for their Dataflows.
5. Use the Power BI web app to enable the use of Datalake storage for dataflows in for each workspace

Unless all of these steps are done the data flow will continue to use the internal power bi storage.

Hope that’s helpful to others. This is all laid out in the different pieces of documentation, however, it’s not as clear if you start out from the Powe BI admin panel. If I’m in correct please let me know!

Josh

[Anonymous]

Same issue here when I try grant permissions to Apps. Following https://community.powerbi.com/t5/Service/Power-BI-Dataflow-to-Azure-Data-Lake-Gen2-Setup/m-p/583564/....

It seems Azure Storage Explorer cant grant permissions to Apps ObjectId. The ids showed in the Storage Explorer screenshot are not the same as in the previous App ObjectId Screenshot. 

[healthEteam]

I was coming here to post about the exact same problem.

I cannot get past that step either.

It seems that those users (object ids?) seem to already need to be populated in the list because anything I put into the line as far as new user and hit "Add", it fails to find.

Hopefully someone has gotten past this and can provide some insight.

Seems like just a step missing in the documentation.

I see @MatthewRoche is providing feedback on some of the datflows posts.  Maybe he can help us.

[MatthewRoche]

Hi @healthEteam - Unfortunately I'm still working throigh the same process. I'll help when I'm able, but it will likely be a day or three...

[healthEteam]

Great - thanks for responding.
Everything I know about dataflows thus far have been from your presentations.
So figured if anyone would know it would likely be you.
Glad to know you are still working through it as well.

[MatthewRoche]

Thank you!

(Blushing furiously)

One of the benefits of working at Microsoft means that I get to play with all the cool toys...

...but to test this feature end to end I need to set up my own Power BI tenant, and that isn't fitting into my week so far.  😞