Please test again. If the issue still occurs, please create a support ticket to get help.
Best Regards,Qiuyun Yu
The key to this was power bi support being able to send me the actual error message from their backend system. It said I hadn't assigned enough memory to the workload.
The answer is in this document https://docs.microsoft.com/en-us/power-bi/service-admin-premium-workloads
I needed to assign more memory, for example for an A2 embedded pricing tier needed 100% maximum memory assigned.