cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
b5lurker
Advocate I
Advocate I

Power BI Dataflow to Azure Data Lake Gen2 Setup

Excited to see all of the new Power BI Dataflow capabilities especially with using Azure Data Lake Gen2 as the storage location for those Dataflows.

 

But, I have been trying to get it setup using the Microsoft provided step-by-step guide on: https://docs.microsoft.com/en-us/power-bi/service-dataflows-connect-azure-data-lake-storage-gen2. The specific steps that I have been unable to complete are in the 'Grant Power BI permissions to the file system' section steps #7 and #8. I have only tried it using Azure Storage Explorer v1.6 since that is the method shown in the documentation.

 

When I try to Add the Object IDs for the Power BI Service and Power Query Online as shown in the screenshots I only get an error saying that it cannot find those User IDs/Groups. I have attached the full error message captured in ASE to see if that might help track it down. I did this with my elevated permissions Azure account and also had our admin grant me full Admin rights over our tenant and could not get it working.

 

Has anyone been able to complete the steps to grant the Power BI Service and Power Query Online applications access to the powerbi blob container in their Azure Data Lake Store Gen2? I even re-created the ADLSg2 yesterday since I had it created in the original private preview. Without these steps I have not been able to complete the process in the Power BI Admin screen to connect Power BI up to the Data Lake for Dataflows.

 

Thanks,

Steve

 

Below is the error message from ASE when trying to Add the Power BI Service Object ID in the Manage Access dialog:

FROM CHILD PROCESS 24608: {"id":"MessagePassingHostProxy151","messageType":"FunctionResponse","response":{"type":"error","error":"{\n  \"message\": \"{\\\"message\\\":\\\"HTTP ERROR 404: Not Found\\\",\\\"response\\\":{\\\"statusCode\\\":404,\\\"body\\\":\\\"{\\\\\\\"odata.error\\\\\\\":{\\\\\\\"code\\\\\\\":\\\\\\\"Request_ResourceNotFound\\\\\\\",\\\\\\\"message\\\\\\\":{\\\\\\\"lang\\\\\\\":\\\\\\\"en\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"Resource 'ee4bcd73-1e25-4154-befe-f8180e3f14d5' does not exist or one of its queried reference-property objects are not present.\\\\\\\"},\\\\\\\"requestId\\\\\\\":\\\\\\\"724657f5-7d0c-4896-9230-5226b96ff9b6\\\\\\\",\\\\\\\"date\\\\\\\":\\\\\\\"2018-12-11T16:34:51\\\\\\\"}}\\\",\\\"headers\\\":{\\\"cache-control\\\":\\\"no-cache\\\",\\\"pragma\\\":\\\"no-cache\\\",\\\"content-type\\\":\\\"application/json; odata=minimalmetadata; streaming=true; charset=utf-8\\\",\\\"expires\\\":\\\"-1\\\",\\\"server\\\":\\\"Microsoft-IIS/10.0\\\",\\\"ocp-aad-diagnostics-server-name\\\":\\\"QAcXmVYUoJaUSrw2ftJmfZfMjTyUjtSkz5dJvbTYsQ0=\\\",\\\"request-id\\\":\\\"724657f5-7d0c-4896-9230-5226b96ff9b6\\\",\\\"client-request-id\\\":\\\"587c72a6-407f-4024-95b6-fbc710401123\\\",\\\"x-ms-dirapi-data-contract-version\\\":\\\"1.6\\\",\\\"ocp-aad-session-key\\\":\\\"hqhNfIg59I6k-OSo-mWlGPFJUAeZDoSngtOC2dH4v4szDJot9OmJGoysNRoVpgZKj_T_ctHrILdmnh-x_E_1VTEIkhcExHCXJk-BPGZKyTzxWQqqAJYh7r3biDvlbfg1.tMDQP1ZfH1teuhHoVGVcULDVzDSD4PiGem3E2On_fFM\\\",\\\"dataserviceversion\\\":\\\"3.0;\\\",\\\"strict-transport-security\\\":\\\"max-age=31536000; includeSubDomains\\\",\\\"access-control-allow-origin\\\":\\\"*\\\",\\\"x-aspnet-version\\\":\\\"4.0.30319\\\",\\\"x-powered-by\\\":\\\"ASP.NET\\\",\\\"duration\\\":\\\"430184\\\",\\\"date\\\":\\\"Tue, 11 Dec 2018 16:34:51 GMT\\\",\\\"connection\\\":\\\"close\\\",\\\"content-length\\\":\\\"294\\\"},\\\"request\\\":{\\\"uri\\\":{\\\"protocol\\\":\\\"https:\\\",\\\"slashes\\\":true,\\\"auth\\\":null,\\\"host\\\":\\\"graph.windows.net\\\",\\\"port\\\":443,\\\"hostname\\\":\\\"graph.windows.net\\\",\\\"hash\\\":null,\\\"search\\\":\\\"?api-version=1.6\\\",\\\"query\\\":\\\"api-version=1.6\\\",\\\"pathname\\\":\\\"/cb2bab3d-7d90-44ea-9e31-531011b1213d/users/ee4bcd73-1e25-4154-befe-f8180e3f14d5\\\",\\\"path\\\":\\\"/cb2bab3d-7d90-44ea-9e31-531011b1213d/users/ee4bcd73-1e25-4154-befe-f8180e3f14d5?api-version=1.6\\\",\\\"href\\\":\\\"https://graph.windows.net/cb2bab3d-7d90-44ea-9e31-531011b1213d/users/ee4bcd73-1e25-4154-befe-f8180e3f14d5?api-version=1.6\\\"},\\\"method\\\":\\\"get\\\",\\\"headers\\\":{\\\"Content-Type\\\":\\\"application/json\\\",\\\"Authorization\\\":\\\"Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6IndVTG1ZZnNxZFF1V3RWXy1oeFZ0REpKWk00USIsImtpZCI6IndVTG1ZZnNxZFF1V3RWXy1oeFZ0REpKWk00USJ9.eyJhdWQiOiJodHRwczovL2dyYXBoLndpbmRvd3MubmV0LyIsImlzcyI6Imh0dHBzOi8vc3RzLndpbmRvd3MubmV0L2NiMmJhYjNkLTdkOTAtNDRlYS05ZTMxLTUzMTAxMWIxMjEzZC8iLCJpYXQiOjE1NDQ1NDU3OTEsIm5iZiI6MTU0NDU0NTc5MSwiZXhwIjoxNTQ0NTQ5NjkxLCJhY3IiOiIxIiwiYWlvIjoiQVZRQXEvOEpBQUFBTExXL2dQcVV2ZmhjODM1MGtrYXhKdVFsMGpUcU00N3dwTG14WGo3b05oZGJHRTRYTjZXdk01VFJYanhuWm9jdzh2Z2I3emZDWXR6ZWw5NFo4WHRsd0RKZ3l0LzFqSlFWbWZzNGlVZm44STQ9IiwiYW1yIjpbInB3ZCIsIm1mYSJdLCJhcHBpZCI6IjA0YjA3Nzk1LThkZGItNDYxYS1iYmVlLTAyZjllMWJmN2I0NiIsImFwcGlkYWNyIjoiMCIsImZhbWlseV9uYW1lIjoiV2FrZSIsImdpdmVuX25hbWUiOiIhU3RldmUiLCJpcGFkZHIiOiIyMDYuODAuMjA5LjE5NiIsIm5hbWUiOiIhU3RldmUgV2FrZSIsIm9pZCI6ImZjOTAxZTNkLWE0MGQtNDgzOC05MDZlLWE5Y2UwYTk0ZTk0MyIsIm9ucHJlbV9zaWQiOiJTLTEtNS0yMS0yMTQ3MjQyNzI2LTM2MzMxNTgwLTE4ODQ0MTQ0NC0xNzI1NTkiLCJwdWlkIjoiMTAwMzdGRkVBODFDNTIyNyIsInNjcCI6IjYyZTkwMzk0LTY5ZjUtNDIzNy05MTkwLTAxMjE3NzE0NWUxMCIsInN1YiI6Ilo0TjlINTQ4alBPNUlGSVE0dS1FU01OdllHRE02WndNVEs3dm80LVljSzQiLCJ0ZW5hbnRfcmVnaW9uX3Njb3BlIjoiTkEiLCJ0aWQiOiJjYjJiYWIzZC03ZDkwLTQ0ZWEtOWUzMS01MzEwMTFiMTIxM2QiLCJ1bmlxdWVfbmFtZSI6IiFzd2FrZUBicnduY2FsZC5jb20iLCJ1cG4iOiIhc3dha2VAYnJ3bmNhbGQuY29tIiwidXRpIjoiOGVCcThEa0o4VVNrT05RQ2dZa2tBQSIsInZlciI6IjEuMCJ9.DVQ1-dpeAfEPX9Lf62JuVtheaAiTNECZpTfoihlL93TKFzEJW-N8alEwDLAfkGTwvj92GRgi7MThS-HqWGfAla4C0ntKxuzq8NhTeGEuACNXVUDNO5mtfYi1W2jJDJirduMSOgdNkkBqCnFvVMqM49cUAYEaPOD-Wn-Fb-U_L3K-hOaZ2EL2_aFLXpm5w-JJs9MPQXcoOSR1rT1x5L9x7At0hIn-A97RzzabFTAMfoolAVDymjLrT2It5jf1qn1vZ1aGXBZPLH_uF7_Aq_v_gTPECuuj-B2rJV3FokQHHCYu9iNFl7lt95ZP_lv_Q9nH01QNp-xjxyHfTrub7iCYCA\\\",\\\"content-length\\\":0}}},\\\"body\\\":\\\"{\\\\\\\"odata.error\\\\\\\":{\\\\\\\"code\\\\\\\":\\\\\\\"Request_ResourceNotFound\\\\\\\",\\\\\\\"message\\\\\\\":{\\\\\\\"lang\\\\\\\":\\\\\\\"en\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"Resource 'ee4bcd73-1e25-4154-befe-f8180e3f14d5' does not exist or one of its queried reference-property objects are not present.\\\\\\\"},\\\\\\\"requestId\\\\\\\":\\\\\\\"724657f5-7d0c-4896-9230-5226b96ff9b6\\\\\\\",\\\\\\\"date\\\\\\\":\\\\\\\"2018-12-11T16:34:51\\\\\\\"}}\\\"}\"\n}"}}
1 ACCEPTED SOLUTION

Hi,
 
There is a temporary issue with settings ACLs through Microsoft Azure Storage Explorer, until this is resolved, as a temporary mitigation you can try using this powershel script:https://setacls.blob.core.windows.net/scriptfolder/setacls.ps1
 
In order to run the script, your global tenant admin needs to have "Storage Blob Data Contributor (Preview)" role on your storage account.
 
Instructions on how to run this script:
    
1. Login as global tenant administrator into Azure portal and open PowerShell (Cloud Shell) in your browser
2. Choose Azure CLI 2.0 (type ‘az’)
3. Upload the script (there is a button for it)
4. Run the script. For example:  /home/admin/setacls -storageaccountname your_alds_gen2_storage_account_name

 

The script will ask you to sign in, please follow the instructions.
 
Thanks,
Michael

 

 

Hi,

 

There is no more need in this workaround, please use ASE 1.6.2 which is availably for update and download.

 

Thanks,

Michael

View solution in original post

68 REPLIES 68

Hi orashare92, From looking at your traces it seems that Power BI Service miss required Reader role on storage as explained here: https://docs.microsoft.com/en-us/power-bi/service-dataflows-connect-azure-data-lake-storage-gen2#gra... Can you please make sure Reader role is set correctly and update us if you continue to experience problems?

Hi

 

You're right. After many tests I created new storage account and i miss this step.

But even after adding Reader role to it the integration still doesn't work.

 

Something went wrong
Error while connecting your Data Lake Storage account
Please try again later or contact support. If you contact support, please provide these details.
Activity ID: 8c6c3ac3-ac67-4c4f-9433-0cf9ed8238c1
Request ID: b0ead637-031a-07ee-cb79-88ab611e177b
Correlation ID: b168ddcd-3e75-1a5e-8fd1-0fdb13396aa0
Time: Mon Jan 07 2019 10:05:00 GMT+0100 (Środkowoeuropejski czas stand.)
Version: 13.0.7840.225
Cluster URI: https://wabi-europe-north-b-redirect.analysis.windows.net

 

Regards

Maciek

I'm still seeing failures due to missing Reader role, can you wait some time and try again and if that doesn't work remove the role and re-assign - it might be issue with role propagation in AAD. Please collect the errors messages you encounter along the way and share if my suggestion doesn't work.

Hi,

I've opted in for Insider Builds in ASE 1.6.1, set up Manage Access as required from the tutorial and assigned Power BI Service a Reader role for the storage account, but I still receive the following error when trying to connect the ADLS Storage from Power BI Service

 

Something went wrong
Error while connecting your Data Lake Storage account
Please try again later or contact support. If you contact support, please provide these details.
Activity ID: 0d0cb805-f647-4f2c-b60b-fcbba8ce312e
Request ID: 6cc12836-acaa-33cc-d0da-efc259f45004
Correlation ID: da689ea8-6789-520a-078b-f62f4827f184
Time: Tue Jan 08 2019 13:46:21 GMT+0100 (Central European Standard Time)
Version: 13.0.7840.225
Cluster URI: https://wabi-north-europe-redirect.analysis.windows.net

 

Both my storage account and Power BI tenant are in region North Europe.

Can you please help me?

Hi Picci, can you please try again with 1.6.2 insider build? that is the build that includes the needed fix.

 

Thanks. 

Hi @sagivh, thanks for your support!

Can you please send me the downloand link? Check for Updates has no effect currently in my desktop application for ASE.

 

Thanks,

Elisa

Picci, please note that 'Check for Updates' might take some time before a notification bar will appears notifying you a new version is available with an option to install it. If you click on update and nothing happens after few minutes, can you please try the "Roll Back To Public Build" option and then check for updates? Please let us know if it doesn't work for you.

@yehong updating ASE to 1.6.2 I was able to connect Power BI Service to my ADL Store!!!! Too happy for that!

Thank you and @sagivh so much!

 

 

So, now it's working.

Thanks for your help.

 

Regards

Maciek

I'm happy to hear that.

 

If you can tell, can you please share which of the methods you've tried? Did you get it after waiting some time? Or was it after the removal/add things started to work?

 

This can help other forum readers, in addition, we would like to communicate this back to storage team as part of our efforts to improve the experience.

First, I set access to powerbi folder on ADLS2 storage account using Storage explorer (1.6.1) and then i tried to enable connection PowerBI with ADLS2 without success.

Next i run script from Michael7 on the same storage account but nothing changed.

 

So I decided today to create new storage account and executing on them script from Michael7, then after correcting my mistake with missing  Read role (for PowerBI service) and waiting some time (around 30 minutes), connection from ADLS2 to PowerBI was created with success.

 

Regards

Maciek

My issue solved! Thank you yehong!

Hi All,

Azure Storage Explorer version 1.6.2 is now available as an insider build, and I can confirm it has fixed the issue of setting ACLs as confirmed in its release notes: "In 1.6.1, entities added to ADLS Gen2 ACLs by ObjectId which were not users were always added as groups. Now, only groups are added as groups, and entities such as Enterprise Applications and Service Principals are added as users."

 

Here is what you can do to obtain it:

1. In Azure Storage Explorer, click on the Help menu, then Opt in to insider builds.

2. In Azure Storage Explorer, click on the Help menu, then Check for updates.

 

After a few moments, a notification bar will appear notifying you a new version is available with an option to install it. Once installation is complete, you can follow the instructions in Power BI documentation to complete the setup: https://docs.microsoft.com/en-us/power-bi/service-dataflows-connect-azure-data-lake-storage-gen2

 

Please let us know if you encounter any issues with setting up storage using Azure Storage Explorer 1.6.2.

Hi,

 

There is a temporary issue with settings ACLs through Microsoft Azure Storage Explorer, until this is resolved, as a temporary mitigation you can try using this powershel script: https://setacls.blob.core.windows.net/scriptfolder/setacls.ps1

 

In order to run the script, your global tenant admin needs to have "Storage Blob Data Contributor (Preview)" role on your storage account.

 

Instructions on how to run this script:

     

  1. Login as global tenant administrator into Azure portal and open PowerShell (Cloud Shell) in your browser
  2. Choose Azure CLI 2.0 (type ‘az’)
  3. Upload the script (there is a button for it)
  4. Run the script. For example:  /home/admin/setacls -storageaccountname your_alds_gen2_storage_account_name
    The script will ask you to sign in, please follow the instructions.

 

Thanks,

Michael

 

Hi,

 

There is no more need in this workaround, please use ASE 1.6.2 which is availably for update and download.

 

In order to update:

ase.png

 

For download: https://go.microsoft.com/fwlink/?LinkId=708343

 

Thanks,

Michael

b5lurker
Advocate I
Advocate I

Some progress today!

 

I was able to get the 1.6.1 release of ASE installed today by Opting In on Insider Builds and checking for updates. Once that was setup I was able to complete the ACL changes for the Power BI Service and Power Query Online applications!

 

Now I'm stuck on the connection in the Power BI Admin panel where I have correctly filled in my Subscription ID, Resource Group Name and Data Lake Storage Account and it just shows a "Something went wrong" error:

 

Something went wrong
Error while connecting your Data Lake Storage account
Please try again later or contact support. If you contact support, please provide these details.
Activity ID: 3dfbcee0-9cdd-4137-b626-31d40d8aa15a
Request ID: 4f8a2a8e-a28d-ee1d-39b2-f81c9e5048e0
Correlation ID: d56c5d02-bd2a-1afd-4dc9-29f46a15d640
Time: Fri Dec 14 2018 14:02:35 GMT-0700 (Mountain Standard Time)
Version: 13.0.7683.181
Cluster URI: https://wabi-west-us-redirect.analysis.windows.net

 

Not sure what is causing this error as there no other details provided. I do know that there is some text in the documentation about having your ADLSg2 setup in the same region as your Power BI tenant. Our Power BI tenant is in West US as you can see from the link shown in the error above. I was unable to setup an ADLSg2 storage account in West US, so the one that I'm trying to point to is in West US 2. Not sure if that is part of the problem or not, but it is the only thing that stands out for our setup.

 

At least we are a few steps closer now. 🙂

 

Happy Friday!

Steve

 

 

Hi all,

 

I experience the same issue as described by Steve, got this working until step 2, but then got the following error, when I pressed 'continue'.

 

Something went wrong

Error while connecting your Data Lake Storage account

Please try again later or contact support. If you contact support, please provide these details.

Activity ID: f53bf51e-984e-4706-9667-49ab5ef8c6a5

Request ID: 12daf846-e59f-7254-a62a-3ee0bfa3e305

Correlation ID: 7bb1b282-ece4-856c-0c7e-8b8fdb9097f3

Time: Thu Dec 20 2018 14:26:34 GMT+0100 (West-Europa (standaardtijd))

Version: 13.0.7683.188

Cluster URI: https://wabi-west-europe-b-primary-redirect.analysis.windows.net

 

When I look at the 'request breakdown'-graph on the dashboard of my datalake, I don't see errors of 'ClientOtherError' at the time that I tried to connect to the lake via PowerBI. (as described by xuanalytics)

Hi lhanegraaf,

Both you and Steve experience different issue from what xuanalytics experience.

Looking at your session traces you both fail to attach external storage since you miss Global Administrator role as required here: https://docs.microsoft.com/en-us/power-bi/service-dataflows-connect-azure-data-lake-storage-gen2.

Can you try again with a user that has Global Administrator role and update us?

Hi Yehong,

 

Allright clear!

 

I managed to try it with Global Administrator rights (on the same account), but still have little success:

 

Something went wrong
Error while connecting your Data Lake Storage account
Please try again later or contact support. If you contact support, please provide these details.
Activity ID: 2aa13457-2360-47d0-8bda-0ff8e6cb0428
Request ID: 74af588b-63d5-f31f-6874-42cf4fac39d9
Correlation ID: fb58b233-080f-9204-0875-2a61b82356ae
Time: Thu Dec 20 2018 15:29:33 GMT+0100 (West-Europa (standaardtijd))
Version: 13.0.7683.188
Cluster URI: https://wabi-west-europe-b-primary-redirect.analysis.windows.net

 

It shows me the 'almost done' message before displaying the error. (This wasn't the case before, but not sure whether that makes a difference?)

 

Thanks!

Same here - updated the Azure Storage Explorer, was able to assign all accesses to the object ids as directed.

 

Still get this same message from Power Bi:

 

Something went wrong
Error while connecting your Data Lake Storage account
Please try again later or contact support. If you contact support, please provide these details.
Activity ID06c5229a-4491-4e95-af0c-2f467c35767f
Request ID07ed9dbc-a104-0953-e079-576d524036ff
Correlation ID5954e326-7c05-59e7-3bd7-6b9bc5eab410
TimeThu Dec 20 2018 09:33:21 GMT-0600 (Central Standard Time)
Version13.0.7683.188
 
Has anyone actually got this to work yet?

Helpful resources

Announcements
Power BI December 2021 Update_carousel 768x460.jpg

Check it Out!

Click here to read more about the December 2021 Updates!

User Group Leader Meeting January 768x460.png

Calling all User Group Leaders!

Don't miss the User Group Leader meetings on January, 24th & 25th, 2022.

Jan 2022 Dev Camp 768x460 copy.png

Power BI Dev Camp- January 27th, 2022

Mark your calendars and join us for our next Power BI Dev Camp!