cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
b5lurker
Advocate II
Advocate II

Power BI Dataflow to Azure Data Lake Gen2 Setup

Excited to see all of the new Power BI Dataflow capabilities especially with using Azure Data Lake Gen2 as the storage location for those Dataflows.

 

But, I have been trying to get it setup using the Microsoft provided step-by-step guide on: https://docs.microsoft.com/en-us/power-bi/service-dataflows-connect-azure-data-lake-storage-gen2. The specific steps that I have been unable to complete are in the 'Grant Power BI permissions to the file system' section steps #7 and #8. I have only tried it using Azure Storage Explorer v1.6 since that is the method shown in the documentation.

 

When I try to Add the Object IDs for the Power BI Service and Power Query Online as shown in the screenshots I only get an error saying that it cannot find those User IDs/Groups. I have attached the full error message captured in ASE to see if that might help track it down. I did this with my elevated permissions Azure account and also had our admin grant me full Admin rights over our tenant and could not get it working.

 

Has anyone been able to complete the steps to grant the Power BI Service and Power Query Online applications access to the powerbi blob container in their Azure Data Lake Store Gen2? I even re-created the ADLSg2 yesterday since I had it created in the original private preview. Without these steps I have not been able to complete the process in the Power BI Admin screen to connect Power BI up to the Data Lake for Dataflows.

 

Thanks,

Steve

 

Below is the error message from ASE when trying to Add the Power BI Service Object ID in the Manage Access dialog:

FROM CHILD PROCESS 24608: {"id":"MessagePassingHostProxy151","messageType":"FunctionResponse","response":{"type":"error","error":"{\n  \"message\": \"{\\\"message\\\":\\\"HTTP ERROR 404: Not Found\\\",\\\"response\\\":{\\\"statusCode\\\":404,\\\"body\\\":\\\"{\\\\\\\"odata.error\\\\\\\":{\\\\\\\"code\\\\\\\":\\\\\\\"Request_ResourceNotFound\\\\\\\",\\\\\\\"message\\\\\\\":{\\\\\\\"lang\\\\\\\":\\\\\\\"en\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"Resource 'ee4bcd73-1e25-4154-befe-f8180e3f14d5' does not exist or one of its queried reference-property objects are not present.\\\\\\\"},\\\\\\\"requestId\\\\\\\":\\\\\\\"724657f5-7d0c-4896-9230-5226b96ff9b6\\\\\\\",\\\\\\\"date\\\\\\\":\\\\\\\"2018-12-11T16:34:51\\\\\\\"}}\\\",\\\"headers\\\":{\\\"cache-control\\\":\\\"no-cache\\\",\\\"pragma\\\":\\\"no-cache\\\",\\\"content-type\\\":\\\"application/json; odata=minimalmetadata; streaming=true; charset=utf-8\\\",\\\"expires\\\":\\\"-1\\\",\\\"server\\\":\\\"Microsoft-IIS/10.0\\\",\\\"ocp-aad-diagnostics-server-name\\\":\\\"QAcXmVYUoJaUSrw2ftJmfZfMjTyUjtSkz5dJvbTYsQ0=\\\",\\\"request-id\\\":\\\"724657f5-7d0c-4896-9230-5226b96ff9b6\\\",\\\"client-request-id\\\":\\\"587c72a6-407f-4024-95b6-fbc710401123\\\",\\\"x-ms-dirapi-data-contract-version\\\":\\\"1.6\\\",\\\"ocp-aad-session-key\\\":\\\"hqhNfIg59I6k-OSo-mWlGPFJUAeZDoSngtOC2dH4v4szDJot9OmJGoysNRoVpgZKj_T_ctHrILdmnh-x_E_1VTEIkhcExHCXJk-BPGZKyTzxWQqqAJYh7r3biDvlbfg1.tMDQP1ZfH1teuhHoVGVcULDVzDSD4PiGem3E2On_fFM\\\",\\\"dataserviceversion\\\":\\\"3.0;\\\",\\\"strict-transport-security\\\":\\\"max-age=31536000; includeSubDomains\\\",\\\"access-control-allow-origin\\\":\\\"*\\\",\\\"x-aspnet-version\\\":\\\"4.0.30319\\\",\\\"x-powered-by\\\":\\\"ASP.NET\\\",\\\"duration\\\":\\\"430184\\\",\\\"date\\\":\\\"Tue, 11 Dec 2018 16:34:51 GMT\\\",\\\"connection\\\":\\\"close\\\",\\\"content-length\\\":\\\"294\\\"},\\\"request\\\":{\\\"uri\\\":{\\\"protocol\\\":\\\"https:\\\",\\\"slashes\\\":true,\\\"auth\\\":null,\\\"host\\\":\\\"graph.windows.net\\\",\\\"port\\\":443,\\\"hostname\\\":\\\"graph.windows.net\\\",\\\"hash\\\":null,\\\"search\\\":\\\"?api-version=1.6\\\",\\\"query\\\":\\\"api-version=1.6\\\",\\\"pathname\\\":\\\"/cb2bab3d-7d90-44ea-9e31-531011b1213d/users/ee4bcd73-1e25-4154-befe-f8180e3f14d5\\\",\\\"path\\\":\\\"/cb2bab3d-7d90-44ea-9e31-531011b1213d/users/ee4bcd73-1e25-4154-befe-f8180e3f14d5?api-version=1.6\\\",\\\"href\\\":\\\"https://graph.windows.net/cb2bab3d-7d90-44ea-9e31-531011b1213d/users/ee4bcd73-1e25-4154-befe-f8180e3f14d5?api-version=1.6\\\"},\\\"method\\\":\\\"get\\\",\\\"headers\\\":{\\\"Content-Type\\\":\\\"application/json\\\",\\\"Authorization\\\":\\\"Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6IndVTG1ZZnNxZFF1V3RWXy1oeFZ0REpKWk00USIsImtpZCI6IndVTG1ZZnNxZFF1V3RWXy1oeFZ0REpKWk00USJ9.eyJhdWQiOiJodHRwczovL2dyYXBoLndpbmRvd3MubmV0LyIsImlzcyI6Imh0dHBzOi8vc3RzLndpbmRvd3MubmV0L2NiMmJhYjNkLTdkOTAtNDRlYS05ZTMxLTUzMTAxMWIxMjEzZC8iLCJpYXQiOjE1NDQ1NDU3OTEsIm5iZiI6MTU0NDU0NTc5MSwiZXhwIjoxNTQ0NTQ5NjkxLCJhY3IiOiIxIiwiYWlvIjoiQVZRQXEvOEpBQUFBTExXL2dQcVV2ZmhjODM1MGtrYXhKdVFsMGpUcU00N3dwTG14WGo3b05oZGJHRTRYTjZXdk01VFJYanhuWm9jdzh2Z2I3emZDWXR6ZWw5NFo4WHRsd0RKZ3l0LzFqSlFWbWZzNGlVZm44STQ9IiwiYW1yIjpbInB3ZCIsIm1mYSJdLCJhcHBpZCI6IjA0YjA3Nzk1LThkZGItNDYxYS1iYmVlLTAyZjllMWJmN2I0NiIsImFwcGlkYWNyIjoiMCIsImZhbWlseV9uYW1lIjoiV2FrZSIsImdpdmVuX25hbWUiOiIhU3RldmUiLCJpcGFkZHIiOiIyMDYuODAuMjA5LjE5NiIsIm5hbWUiOiIhU3RldmUgV2FrZSIsIm9pZCI6ImZjOTAxZTNkLWE0MGQtNDgzOC05MDZlLWE5Y2UwYTk0ZTk0MyIsIm9ucHJlbV9zaWQiOiJTLTEtNS0yMS0yMTQ3MjQyNzI2LTM2MzMxNTgwLTE4ODQ0MTQ0NC0xNzI1NTkiLCJwdWlkIjoiMTAwMzdGRkVBODFDNTIyNyIsInNjcCI6IjYyZTkwMzk0LTY5ZjUtNDIzNy05MTkwLTAxMjE3NzE0NWUxMCIsInN1YiI6Ilo0TjlINTQ4alBPNUlGSVE0dS1FU01OdllHRE02WndNVEs3dm80LVljSzQiLCJ0ZW5hbnRfcmVnaW9uX3Njb3BlIjoiTkEiLCJ0aWQiOiJjYjJiYWIzZC03ZDkwLTQ0ZWEtOWUzMS01MzEwMTFiMTIxM2QiLCJ1bmlxdWVfbmFtZSI6IiFzd2FrZUBicnduY2FsZC5jb20iLCJ1cG4iOiIhc3dha2VAYnJ3bmNhbGQuY29tIiwidXRpIjoiOGVCcThEa0o4VVNrT05RQ2dZa2tBQSIsInZlciI6IjEuMCJ9.DVQ1-dpeAfEPX9Lf62JuVtheaAiTNECZpTfoihlL93TKFzEJW-N8alEwDLAfkGTwvj92GRgi7MThS-HqWGfAla4C0ntKxuzq8NhTeGEuACNXVUDNO5mtfYi1W2jJDJirduMSOgdNkkBqCnFvVMqM49cUAYEaPOD-Wn-Fb-U_L3K-hOaZ2EL2_aFLXpm5w-JJs9MPQXcoOSR1rT1x5L9x7At0hIn-A97RzzabFTAMfoolAVDymjLrT2It5jf1qn1vZ1aGXBZPLH_uF7_Aq_v_gTPECuuj-B2rJV3FokQHHCYu9iNFl7lt95ZP_lv_Q9nH01QNp-xjxyHfTrub7iCYCA\\\",\\\"content-length\\\":0}}},\\\"body\\\":\\\"{\\\\\\\"odata.error\\\\\\\":{\\\\\\\"code\\\\\\\":\\\\\\\"Request_ResourceNotFound\\\\\\\",\\\\\\\"message\\\\\\\":{\\\\\\\"lang\\\\\\\":\\\\\\\"en\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"Resource 'ee4bcd73-1e25-4154-befe-f8180e3f14d5' does not exist or one of its queried reference-property objects are not present.\\\\\\\"},\\\\\\\"requestId\\\\\\\":\\\\\\\"724657f5-7d0c-4896-9230-5226b96ff9b6\\\\\\\",\\\\\\\"date\\\\\\\":\\\\\\\"2018-12-11T16:34:51\\\\\\\"}}\\\"}\"\n}"}}
1 ACCEPTED SOLUTION

Hi,
 
There is a temporary issue with settings ACLs through Microsoft Azure Storage Explorer, until this is resolved, as a temporary mitigation you can try using this powershel script:https://setacls.blob.core.windows.net/scriptfolder/setacls.ps1
 
In order to run the script, your global tenant admin needs to have "Storage Blob Data Contributor (Preview)" role on your storage account.
 
Instructions on how to run this script:
    
1. Login as global tenant administrator into Azure portal and open PowerShell (Cloud Shell) in your browser
2. Choose Azure CLI 2.0 (type ‘az’)
3. Upload the script (there is a button for it)
4. Run the script. For example:  /home/admin/setacls -storageaccountname your_alds_gen2_storage_account_name

 

The script will ask you to sign in, please follow the instructions.
 
Thanks,
Michael

 

 

Hi,

 

There is no more need in this workaround, please use ASE 1.6.2 which is availably for update and download.

 

Thanks,

Michael

View solution in original post

68 REPLIES 68

Hi healthEteam,

 

From looking at your traces it seems that that storage account is not located at the same region of your tenant. From the setup manual, "The storage account must be created in the same region as your Power BI tenant. To determine where you Power BI tenant is located, see where is my Power BI tenant located."

Can you please tell if placing the storage in your tenant's storage solves the issue?

Both region are West US set since the beginning. Thanks.

 Storage location.PNGPBI location.PNG

xuanalytics, please note that my comment about incorrect region was directed to healthEteam.

 

We're still waiting for a response from Azure Storage team for your issue (you managed to attached storage but fail to create dataflows on top of it). Will update you once we reiceve their response.

Hi Eveyone,

 

I tried again this morning just to make sure I am still getting the same error. I am and am in the same boat as xuanaltyics.  Pasting it below incase its helpful for the storage team.  Any help figuring this out is greatly appreciated! Thank you everyone for your time!

 

Something went wrong
Unable to save the dataflow.
Please try again later or contact support. If you contact support, please provide these details.
Activity ID: f617af12-4384-4fb6-a8e8-0a84bdfe0977
Request ID: 74bb234b-89de-4bf3-0fe9-2d062ed3d3e0
Correlation ID: 1b76189c-c3ae-ffd6-0ab6-1b5f7ad05176
Status code: 500
Time: Thu Dec 20 2018 09:23:56 GMT-0800 (Pacific Standard Time)
Version: 13.0.7683.188
Cluster URI: https://wabi-west-us-redirect.analysis.windows.net

 

Josh

General note -- we are working on improving the error handling in this scenario.

 

For your specific issue, it seems from traces that you didn't configure the filesystem correctly (didn't create 'powerbi' file system? didn't set correct permissions on root?) - filesystem setup starts here: https://docs.microsoft.com/en-us/power-bi/service-dataflows-connect-azure-data-lake-storage-gen2#cre...

 

Can you let us know this solved your issue?

Hi Yehong,

 

Thanks for your help, but it didn't succeed unfortunately.

I already set it up the way that is described in the article you linked. However I tried setting it up once again, but without success.

I was able to get my login setup with the Global Admin role and after I signed out of Power BI and signed back in with the credentials that have been attached to the Global Admin it sat on the Almost Done message for quite a bit longer then it has before, but still ended up with the "Something went wrong" message. Below are the error details:

 

Something went wrong
Error while connecting your Data Lake Storage account
Please try again later or contact support. If you contact support, please provide these details.
Activity ID: a2de8002-2fa2-45f1-bc6f-50d8403a4b9d
Request ID: c895f191-8ad8-79a9-bc46-27e9c6495712
Correlation ID: 14645bcf-3862-8786-24de-b0d03a3f5946
Time: Thu Dec 20 2018 08:32:25 GMT-0700 (Mountain Standard Time)
Version: 13.0.7683.188
Cluster URI: https://wabi-west-us-redirect.analysis.windows.net

 

As far as I know I have the powerbi blob container and security all setup for the Power BI Service, Power BI Premium and Power Query Online application Object IDs, so not sure what else is missing to get this working. I have also setup the ADLSg2 resource in the West US region, so it is in the same region as our Power BI tenant now.

 

Thanks,

Steve

Hi Steve, from looking at your traces it seems that you're hitting the same issue as lhanegraaf. We're waiting for response from Azure Storage team and will update you.

Hi Everyone,

 

I was also able to add the ACLs using the internal build. In addition, since my account was already connected, I was able to work through all the steps in the configuration document. 

 

I also created a new workspace and was able to configure the workspace to use data lake storage. 

 

Please note, this option is only available in workspaces created to view the new experience.  If your workspace has been created via an O365 group, you will not be able to configure it to use data lake storage.

 

Unfortunately, when I try to save a workflow I get the following error:

 

Something went wrong
Unable to save the dataflow.
Please try again later or contact support. If you contact support, please provide these details.
Activity ID: 6cf62757-8eed-4c8f-b89a-19e737873a62
Request ID: 4ebf59ad-5980-ed22-5432-3f2915ef5446
Correlation ID: 1f1bbc99-c76b-8c1b-54af-aaa270b10656
Status code: 500
Time: Fri Dec 14 2018 22:07:33 GMT-0800 (Pacific Standard Time)
Version: 13.0.7683.181
Cluster URI: https://wabi-west-us-redirect.analysis.windows.net

 

So, unfortunately, no dice ... yet. But progress!

 

Thanks to everyone for the help.  This has been a great thread!

 

*** Update 12/16/18 ***

 

Hi Everyone,

 

Double checked my config tonight against the instructions laid out in https://docs.microsoft.com/en-us/power-bi/service-dataflows-connect-azure-data-lake-storage-gen2.  Noted my config appears correct, but still getting the 500 error.  Any help in review would be appreciated!

 

Josh

Hi Josh,

 

I am getting the exact same error as you got. I configured everything according to the Microsoft document. But I am not able to save any data flow in the new app workspace which Data Lake Storage enabled. I have then checked teh activity log of the storage account and I found everytime I am trying to save it 

 

Activity.PNG

Hi xuanalytics - are you seeing the failures in PowerBI portal? can you please share the "Something went wrong" message contents?

Hi yehong, yes, I got it from PowerBI portal and it is the same error as Josh got. Here is the information.

Something went wrong
Unable to save the dataflow.
Please try again later or contact support. If you contact support, please provide these details.
Activity ID: d33a532d-45eb-4d65-9b7a-f7041b3cd1a8
Request ID: 9135fad1-a6f5-c6f9-e204-165cfd0e48ba
Correlation ID: c7c3a7ba-233c-8e8f-1139-28bbb6935973
Status code: 500
Time: Thu Dec 20 2018 07:55:02 GMT-0500 (Eastern Standard Time)
Version: 13.0.7683.188
Cluster URI: https://wabi-us-west2-redirect.analysis.windows.net

I see Authorization errors on storage (the spikes you've shared before), since you confirmed that you've set up the storage according to documentation, I'll be reaching out to storage team to understand the issue.

BhaveshPatel
Community Champion
Community Champion

Hi All,

 

I am having similar issues connecting my Azure Data Lake account to Power BI Dataflows. I have called Microsoft Support to get this resolved. I wait till next week before the fix is in place.

 

I find this thread really helpful.

 

 

Thanks,

Bhavesh

Thanks & Regards,
Bhavesh

Love the Self Service BI.
Please use the 'Mark as answer' link to mark a post that answers your question. If you find a reply helpful, please remember to give Kudos.
bensack
Microsoft
Microsoft

Hi Steve, 

 

 

I am sorry for the trouble setting ACLs on the storage account per the documentation. The error you are seeing is a bug in Azure Storage Explorer that unfortunatly prevents setting ACLs for anything but people. It is being fixed, should be available soon, and we are working with the Storage Explorer team on an exact ETA.

 

Until it is fixed, the only option for setting ACLs is via REST API call as documented here:

https://docs.microsoft.com/en-us/rest/api/storageservices/datalakestoragegen2/filesystem/setproperti...

but unfortunatly its not trivial, even with the above documentation.

 

 

 

 

We are working to expedite the fix and also find an alternative path until such a fix is available, I hope to have an update soon.

Thank you,

Ben

 

 

 

I would like to update that a new version with the ACL issue fix should be available next week. Until then, another option is to opt in for insider builds of Azure Storage Explorer. In Azure Storage Explorer, click on the Help menu, then Opt in to insider builds. a build should be available via that route sooner.

 

 

Hi All, 

 

Azure Storage Explorer version 1.6.1 is now available as an insider build, and I can confirm it has fixed the issue of setting ACLs.

 

Here is what you can do to obtain it:

 

1.  In Azure Storage Explorer, click on the Help menu, then Opt in to insider builds.

2.  In Azure Storage Explorer, click on the Help menu, then Check for updates.

After a few moments, a notification bar will appear notifying you a new version is available with an option to install it.

 

Once installation is complete, you can follow the instructions in Power BI documentation to complete the setup:

https://docs.microsoft.com/en-us/power-bi/service-dataflows-connect-azure-data-lake-storage-gen2

 

Thank you,

Ben

Thanks Ben -

 

If you would, can you please update this thread when this update has been released so we will know when we can go retreive it.

 

Thanks again

Hi Everyone,

Just FYI I ran into the same issue today. However I’ve got one small wrinkle. While I was not able to update the ACLs, I was able to get past the screen in Power BI. Now the service is linked I to my storage account, but does not appear to be writing files to the data lake. FYI.

Josh

Hi Josh, 

 

 

The setup of the storage account must happen as documented - the ACLs are critical for the feature to work and the storage account should not be connected to Power BI without them. I recommend you do not try to use the feature until ACLs are set on the filesystem. 

 

The fix from Azure Storage Explorer should be available next week, but as mentioned above you can opt in for insider builds and get it sooner. 

 

Please let me know if you have questions or if you run into any trouble after setting the ACLs.

 

Thanks,

Ben

 

 

 

Hi Ben,

 

Thank you for your reply.  Unfortunately, I linked my power bi account before finding this post. While we have linked the power bi account, we have not configured any of our app workspaces to use the linked storage account. Will using dataflows like this cause us issues or will our flows continue to populate the Power BI storage?

 

Is there any way in Power BI to unlink the account?  There does not seem to be a setting to do so in the Power BI interface.

 

Josh

Helpful resources

Announcements
Carousel_PBI_Wave1

2023 Release Wave 1 Plans

Power BI release plans for 2023 release wave 1 describes all new features releasing from April 2023 through September 2023.

Power BI Summit Carousel 2

Global Power BI Training

Make sure you register today for the Power BI Summit 2023. Don't miss all of the great sessions and speakers!

BizApps LATAM 2023

Business Application LATAM Summit 2023

Join the biggest FREE Business Applications Event in LATAM this February.

Power Platform Bootcamp

Global Power Platform Bootcamp

In this bootcamp we will deep-dive into Microsoft’s Power Platform stack with hands-on sessions and labs, delivered to you by experts and community leaders.

Top Solution Authors
Top Kudoed Authors