Deployment pipelines copies credentials entered in production step to test step and vice versa

[KrisD]

Hi!

I have yet to find anyone else who has encountered the same problem. 

When I publish a dataset from TEST to PROD it brings the TEST credentials with it to PROD, forcing me to change to PROD credentials for the dataset.

BUT, when I do that the credentials get copied to the TEST dataset which then creashes at refresh since the credentials are wrong. 

If I then change to TEST credentials the dataset is fine. But that in turn copies the credentials to PROD dataset. 

This can also occur at complete random, my PROD dataset can refresh properly for weeks and then suddenly on its own change credentials and fail at refresh which occured during my vacation. 

This problem pertains to mainly a specific dataset, the rest function properly. 

Has anyone experienced something like this?

[MT1178 · Accepted solution]

KrisD, we found a way through this issue by ensuring you leverage separate logical warehouse names in Snowflake. It seems it is the connection string + warehouse name that is cached. By introducing unique Snowflake warehouse names for DEV/TEST/PROD, they are never overwritten. 

Hope this helps others in the community. 

[MT1178]

We are having the same issue @KrisD - we are using Power BI against Snowflake, and it appears we are not able to have unique usernames and passwords for DEV/TEST/PROD, due to this overwriting or caching issue.

Did you find a resolution in the end?

Rgds

[KrisD]

Hi!

As a temporary solution I have myself taken ownership of all our PROD datasets and someone else owns all TEST datasets, just to have the refreshes come through. 

We are however investigating the proper solution to this, I'll return with updates when I have more! 

[v-yetao1-msft]

Hi @KrisD 

I am not particularly able to understand the situation you are talking about. What is the specific dataset you are referring to? During deployment, the data source credentials will not be copied. Could this be the kind of situation you are talking about?

Best Regards

Community Support Team _ Ailsa Tao

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[KrisD]

I first deploy (IT Service DS in this case) from desktop to the dev pipeline. From there I publish to test where I have to change data connection credentials in "Dataset --> Settings --> Data connection" to the test credentials. From there I publish to the prod pipeline and change the credentials to the prod credentials, and here is where the problem occurs. The prod credentials will automatically get copied to the test pipeline crashing any refresh in the test pipeline. If I then change the credentials to test credentials they get copied to the prod pipeline crashing any refresh in that pipeline.
I am automatically forced to have the same credentials for both the test AND the prod pipeline. Sometimes even the dev pipeline automatically changes credentials from the test and prod pipeline.  

[MT1178 · Accepted solution]

KrisD, we found a way through this issue by ensuring you leverage separate logical warehouse names in Snowflake. It seems it is the connection string + warehouse name that is cached. By introducing unique Snowflake warehouse names for DEV/TEST/PROD, they are never overwritten. 

Hope this helps others in the community. 

[KrisD · Accepted solution]

Thanks for your reply!

We chose to use different service accounts for test and prod which we deploy as dataset owners through DevOps pipelines, so this also is a solution.