Easiest way is to think of it like this:
While you are in Power BI Desktop, you have some locally stored credentials that are used to make the connection. If you save your PBIX file and pass it to someone else, the credentials are not stored in the PBIX file itself, but in the local cache of your Power BI Desktop application. This new person would need to enter their own credentials to access the DirectQuery data, or run a refresh (Import).
When you publish to the Power BI Service, again no credentials are sent along with the file. Since the cloud is outside your network, unless your data source is also in the cloud, it has no direct connection to your data source.
An On-Premise Data Gateway acts as a secure bridge. It gets installed on hardware inside your network and you make a specific connection to the Office 365 tenant. Under "Manage Gateways" in the Power BI Service, you can add Data Sources, their credintials, and which users can access this data source. This is now avaiable to be used by any project that contains that data source and is being configured by someone on the access list. (small note, a project that wants to use a gateway needs to ensure that every data source is within the gateway, or is a cloud data source).
When you go into the "Schedule Refresh" of your Power BI Dataset on the Service, you must select a gateway. If its configured correctly you will see it appear in the gateway listing. Whenever this refreshes (import) or connects (direct query), it will talk only to the gateway. The gateway itself will handle the authentication and passing the data between the cloud and your network. The report itself doesn't need any credentials.
The credentials while in the cloud are all managed via "Manage Gateways" and work irrespective of how you publish a project/dataset into the Power BI Service.
