Data Refresh

[Anonymous]

Hello all,

I am new to Power BI and I'm trying to better understand how the data refresh function works. I am currently working on a report that shows the stock inventory history. I want my report to refresh every day so that the data and information is always up to date. 

My data source includes a CSV file that I manually export to my computer and manually import into Power BI. To get up to date information on the stock inventory, I have to manually export the CSV file everyday. This file does not update on its own. 

The only way I can think of doing this, is manually exporting the CSV file and then changing the name of the file every time. 

I've watched some tutorial videos, and they mention using a gateway connection and so on. But again, I'm new to Power BI and I'm just not sure how that is possible, when the file containing the stock inventory data isn't created until an export is made. (I hope this makes sense)

Is a scheduled data resfresh even possible, since everything is being done manually?

[ray_aramburo]

There are 2 stages of refresh as I see it:

1) If you are working locally (this means only through Power BI Desktop) no matter how times your data source is refreshed you'll always need to manually refresh from Power BI Desktop to get the latest data.

2) If/When you publish your report to the Power BI Service, you can enable a scheduled refresh. Since you are working with csv files, I would recommend you to host them in a OneDrive/SharePoint/Cloud location so you can avoid using any gateway and then use the SharePoint folder or Web connection to access them. 

Now, you mention "the file containing the stock inventory data isn't created until an export is made." You'll need to identify what's exactly the origin of the data that builds the csv, is it a database?, ERP?, existing report?. Once identified, instead of building csv files you'll aim to connect to that data directly. 

[Anonymous]

Regarding your first point, I am working locally in the Power BI Desktop. I am not as concerned about getting the latest data in the desktop as I am about having the latest data on the Power BI Web Service. 

I've already published my Product Stock Inventory report to the web service and have enabled a schedule refresh. When hosting the CSV file in OneDrive, would I still need to manually export the CSV file, and change the name to match the original, to my OneDrive to get the latest data? 

Do you have any other clearer intructions on how to complete this?

[ray_aramburo]

So, if I understood correctly, your issue relies that the data source needs to be manually updated on daily basis, and because of this, it has a different name every day. Is that correct?

If that's the case, could you specify from where do you need to extract the csv files?

[Anonymous]

Yes our product stock inventory is hosted on a website and the data from that can then be exported into a CSV file. Meaning that I have to manually export a CSV file everyday to get the latest data. And yes these CSV files all have a different name, with the date and timestamp that it was created in the name. 

[ray_aramburo]

Okay, I see 2 options, one a short term workaround, and the other one as a long term solution:

1) Short-term: When you export the file, save it with a specific naming convention on a cloud-based location (Sharepoint/OneDrive/Google Drive). You will always name it this way on a daily basis. Since it will have the same name, the last file would be replaced/overwritten. Your Power BI report will have no issue because it will always read the file with the same name.

You can reduce some manual steps here if you develop some small automation in Power Automate to do the file saving process, but this solution will always inherently be manual because it requires for you to extract data manually.

2) The long-term solution is you finding a way to connect to the web page (or the database/data sources) behind it. The nature of this will depend on the architecture of the web page. By setting up a direct connection you avoid exporting manually the data. As to how to achieve this, my answer is, it depends on how is the data behind the web page structured/located.

[Anonymous]

As far as the short term solution, once I export it and store my file on a cloud-based platform, all the data in my report should be updated in both the deskop and online web service or just one location? And when I export that new file, will the measures I previously created still be available to me? Or do I need to create new measures since it is technically a new table?

[ray_aramburo]

You will need to manually refresh from the desktop (at least once) and publish the report with the updated reference. You don't need to create any new measures or edit them as long as you keep the same table structure (column names and number of columns). Then you can schedule your refresh. 

Independently of the type of solution, when working with Power BI Desktop you will always need to manually refresh the data from Power BI, but that's only required when you need to develop or update the report. Else, the Power BI Service will run the refresh for you automatically on your scheduled periods. 

[Anonymous]

I've moved my files into my OneDrive and my reports have been refreshing on the desktop side. However I am still strugglign to get them to refresh on the web service. I haven't been able to schedule a refresh because my data source credentials are not being received. 

I've tried editing and configuring the credentials but it say it is failing to update. 

Is there a way to fix this?

[ray_aramburo]

Which authentication method did you select? Should be OAuth2 Organizational. If you still struggle, DM me and we can see how to connect to address your issue 🙂