Thank you, @ericOnline! Yes, I might have to try the Advanced Editor. I find the UI is not as simple to use as it looks, when it comes to particular cases like this.
Silly me, actually @ericOnline's solution above worked for me as well. It seems there is currently no direct way to use Azure AD as authentication via the UI. I opted to use API key authentication, but did it the wrong way. So i modified my api to have a custom request header passing the API key, and applied the steps above in my Power BI report.