Dataflow and Snapshot

[silcambro]

Hello,

I have a Power BI workspace and I set the Storage to my Azure Data Lake.

I created a Datafow and as expeced the Data are saved in my ADLS in CDM format.

What is strange, at least for me, is that every time  I refresh the dataflow a new CSV file is created and the previous file

is not overwritten. Essentially a snapshot is created and added to the folder. Is there a way i can limit the number of snapshots created or my ADLS will grow to infinite size.

Thank you so much

[otravers]

Azure Data Lake has lifecycle management options:

https://learn.microsoft.com/en-us/azure/storage/blobs/lifecycle-management-overview

[nqzhhzmn]

This is by design so Microsoft can sell more storage space.  There is no way to limit the space usage.

[silcambro]

Hello,

thank you so much for your reply.

My source are CSV files in Azure Data Lake.

And as you Know when Data flow Runs it will generate a CDM Folder,

those are the file in one of the CDM Snapshot Folders after three refreshes:

orderheader.csv@snapshot=2020-12-04T22:07:00.6513096Z

orderheader.csv@snapshot=2020-12-09T18:34:44.7746893Z

orderheader.csv@snapshot=2020-12-24T21:25:08.5191499Z

and this is the Image

Dataflow Name : Orders

Query: orderheader

It is the normal behavior didn't set anything in particular.

Any ideas, if this is by design?

Thank you so much

[v-rzhou-msft]

Hi @silcambro 

Could you tell me what kind of datasource did you use to build your dataflow?

I think your issue may be that when you refresh your dataflow, your file in Azure Data Lake will create a new file instead of overwritting the previous file. 

Does the new file have the same name as the previous one?

Please show me a screenshot of your issue.

You may try to delete the previous, and check whether the dataflow will work well.

The blogs as below may help you.

Blog: Dataflows in Power BI

Best Regards,

Rico Zhou

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.