Power BI + BigQuery = GARBAGE. Refreshes Do Not Work.

[danzrust]

I've been using PBI online service with BigQuery for a very long time. Since about Q4 2019, it's becoming super unreliable.

I used to link directly to views in BigQuery, than the datasets started to fail to refresh more than to succeed. 

We were told to use DataFlows is middleman to get BQ data and then to link PBI to Dataflows. Not even this is working. DataFlows are failing 50% of the time.

If you are planning to build complex models on the top of BQ data, don't do it in Power BI. You'll go crazy.

This is my DataFlow refresh history:

[danzrust]

Okay, so once again, I am having issues refreshing BigQuery datasets after making no changes. This is now happening in multiple Power BI datasets which have been reliably updating for months.

I am project owner + Big Admin.

Yet, I keep getting errors such as:
Error: DataSource.Error: ODBC: ERROR [42000] [Microsoft][BigQuery] (100) Error interacting with REST API: Access Denied: Project project_id: "sys-50980974576927539066589731";gaia_id: 460742005487

This pops sometimes right after I start refresh of a dataset or sometimes during refreshes of data flows. 

Funny thing #1: I have no clue what this "sys-50980974576927539066589731" project is. I see these wierd names in Power BI along with correct project name, but this "sys-" is definitely not my project ID.

Funny thing #2: In my dataflow, I am pulling data from X BigQuery tables under the same Google Cloud project. 7 flows refreshes OK and then boom, 8th flow won't go through. This is what I see in the dataflow fail logs:

It does not even make sense to fail just on 1 table when the others go through OK.

Then I am also getting errors such as: The last refresh attempt failed because of an internal service error. This is usually a transient issue. If you try again later and still see this message, contact support. 

So transitient that I get it 10 times in row. 

Or this one:  There was an error when processing the data in the dataset.

And of course, when I refresh datasets from desktop with the same account credentails, it works fine. 

I would be good if someone from Microsoft connects with someone from Google and finally make the BigQuery connector in Power BI reliable. 

[v-alq-msft]

Hi, @danzrust 

The screenshot shows 'schedual refresh' and 'refresh now' both have problems when refreshing. I wonder if there are some error messaages. Do you have this issue for other entities? Please check if you have installed the latest version on-premised gateway and if the gateway is online.

Best Regards

Allan

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[danzrust]

HI Allan,

I don't need any gateway. I'm pulling the data just from BigQuery so I just login in the browser and then the data source is available for dataflow... It's the inconsistency and 50% unreliability which is killing me. 

Today, I tried to manually run the dataflow 4 times. 4 fails. Scheduled ones worked (3x). 

This is one of the errors from today: Error: We're sorry an error occurred during evaluation. Request ID: 02355ac1-2c3e-67e3-72eb-070d69d59de7 Activity ID: 20479c7e-53f6-4230-9520-729c2fdb2359 DZ 

[Greg_Deckler]

You could check the Issues forum here:

https://community.powerbi.com/t5/Issues/idb-p/Issues

And if it is not there, then you could post it.

If you have Pro account you could try to open a support ticket. If you have a Pro account it is free. Go to https://support.powerbi.com. Scroll down and click "CREATE SUPPORT TICKET".