What is the best way not to have duplicate data and to be productive when importing data

[matheus_peppers]

Hi guys, what's up?

I would like to take a doubt a little less technical, as I believe this is more like good practice.

I created a report where the database, was exported from Facebook Ads, where I pulled the values from the maximum period.

I did the data retrieval through the folder path, instead of choosing the file.

However, I am afraid, and even here is my doubt.

In this report, I pulled the values from a maximum date period, and for example, if next week, I also pull the updated report, but from the maximum period as well, instead of just this last week.

Do I run the risk of having duplicate data? (When I import the base into Power BI, I always Transform the Data and choose to combine the tables, but it is a doubt that sticks in my mind).

And another thing, instead of importing the database by folder path so I don't need to mess with the Excel file, but only download and put it there in the folder, is it not better if I choose to import by Excel file itself, instead of folder, and every day update the data in the file with the new data downloaded?

These are just questions I have about good practices to:

1) Not run the risk of having duplicate data in my final report...

2) And to optimize my time as much as possible, without having to manually update the data in the base spreadsheet, copying and pasting the updated data.

If someone can help me with this question, I will be extremely grateful.

[Adel]

and just another point, why are you loading the files through import or through file path, why not use use a power bi connector, set it up with a future date and you will get the data you need with no headaches or wasted time.

[Greg_Deckler]

@matheus_peppers So, if you are using import mode and not using incremental refresh, Power BI basically truncates the tables in the data model and reloads the data in its entirety. Thus, in your situation you should not run into duplicate rows in this circumstance. You can make absolutely certain of this by doing a "Remove duplicates" step in Power Query Editor.