I am not sure if this is the question but this how I'm doing it, assuming for example my Data Source is SQL Server Database:
I have a report named Report_Dev that is based on a Data Source named Source_Dev.
In order to create a production version of this report I open it in PowerBI Desktop and then from Edit Queries in the Home menu I select Data Source Settings.
I select Change Source and specify the production server (my production server has the same db schema but only the data is differrent) and database I.e., my Data Source is now Source_Prod. Then I save the PowerBI Desktop file as Report_Prod and publish to a workspace that I have named Prod_Workspace.
Yes, that's the problem and that's how we have ended up doing it. However, for highly complex reports that connect to several different data sources, that's a real pain. If you are publishing to a Power BI Report Server, you also have to set up the connections there as well, and the refresh schedule. Not really joined up, but I appreciate it's a difficult nut to crack!