Push to Dataset table - The remote server returned an error: (404) not found error in powerBI

[sarogersz]

I have built an app that pushes data into Power BI datasets.  I have been pushing to one dataset just fine.  I added another, using the same code pattern and helper routines for teeing up Power BI GET and POST requests.  In this new case, I have the 404 problem when pushing.

In the problem case, I have no problem creating the dataset, and verifying that it is correct (e.g. has all the right columns, names etc.) using powerbi.com.  I further verified that I am getting the table name right using GET Tables:

      https://api.powerbi.com/v1.0/myorg/datasets/5a2b6767-67dd-48ef-bab4-1d3ad7299f7d/tables

From the json returned I see that the table "DataHeatResults" is the one table in the dataset, and this is correct.

So, next I am ready to push data into this DataHeatResults table, and I'm using the following as the URL for the push:

      https://api.powerbi.com/v1.0/myorg/datasets/5a2b6767-67dd-48ef-bab4-1d3ad7299f7d/tables/DataHeatResu...

This URL is not found in Power BI, i.e. the 404 error.

It's weird to be stuck on this since I've verified that the URL seems correct (and everything matches the pattern of the Push URL I'm using for the dataset that actually works solidly).

But if anyone can help me with ideas for tracking this down I'd be grateful.  As described above I verified that the table name is correct.  Clearly this verifies that the Dataset ID is correct,  What else might trigger this type of error?

[sarogersz · Accepted solution]

These are great suggestions, and I have done all of them in the course of events before and after receiving this feedback.

From the feedback I have received and the results of many hours of debugging, my next step is to do a detailed check of the datasets in my code to make sure they all align perfectly.  Here are snippets of representatives of the data descriptor elements I employ in this solution:

1) -----------------------------------------------------------------------------------------------------

JSON dataset schemas consumed by powerbi.com to create and push data in (via GET and POST):

{
"name": "@DatasetName",
"defaultMode": "PushStreaming",
"tables": [
{
"name": "DataHeatResults",
"columns": [
{
"name": "Record_ID",
"dataType": "Int64"
},
{
"name": "Date_Time",
"dataType": "Datetime"
},
{
"name": "DryerID",
"dataType": "string"
}, .......................etc.

2) -----------------------------------------------------------------------------------------------------

Model Classes used in C# code to work on assembling and marshalling specific data rows:

using System;

namespace PushDemo.Models
{
public class DataHeatResults
{
public int Record_ID_ { get; set; }
public DateTime Date_Time { get; set; }
public string DryerID { get; set; }
public string RunID { get; set; }

............................etc.

-----------------------------------------------------------------------------------------------------

I have been advised that all of these must be in perfect alignment from a data type, order of declaration etc. perspective or 404 errors can result.  It does not matter that all data elements in the data sources are unused - nothing can be omitted.  Also, even though the data types associated with Powerbi.com do not perfectly align with those in C#, they must be mapped perfectly or you can get a 404.

This is not intuitive to me, but my next step is to do another pass of fine-detail code analysis to ensure that I have the best possible matching of what is being sent within the powerbi.com context and the C# context.  With this advice I will take a hard look at problems that might arising from (for example) Int64 in powerbi.com not being compatible with int in C#.

Because of the inordinate amount of time I've spent dealing with this house of cards I will report back so others may be spared being stuck like this, where everything looks perfect, the URI's used to create the powerbi.com dataset schema are proved to be identical with those being used to do the POST / push, and casual assumptions that data types between powerbi.com, JSON and C# are compatible.  All dataset IDs have been checked, the datasets created in powerbi.com are identical in every way I can see (looking at a report I create to see the datasets that way).

The only anomoly in evidence is a 404 at the moment of truth where data is pushed into a fresh dataset just created by the same code.

Adding to this frustration is the fact that I am using the same code to create a different dataset, and there is no problem pushing into that dataset.

Go figure!

As always, it will turn into some simple mistake somewhere.  But this is a lot to go through.........  maybe some kind of more explanatory error message than a 404 might save thousands of hours of wasted debugging time across the world!  

If anyone has an idea for how to do this more simply, I'd be happy to refactor and take a different approach....

Thanks, Stephen

[v-ljerr-msft]

Hi @sarogersz,

Could you recheck the steps below to troubleshooting this issue?

• Set some breakpoints in the sample code and see if the dataset is actually created.
• If the dataset is created, and is found by the call that lists all datasets, check for the id, as there can be multiple datasets with the same name, id is unique.
• Finally, make sure that id is used by the POST to push rows into the dataset.

In addition, if the issue persists, I would suggest you create a support ticket on Power BI Support page for better assistance. 

Regards

[sarogersz · Accepted solution]

These are great suggestions, and I have done all of them in the course of events before and after receiving this feedback.

From the feedback I have received and the results of many hours of debugging, my next step is to do a detailed check of the datasets in my code to make sure they all align perfectly.  Here are snippets of representatives of the data descriptor elements I employ in this solution:

1) -----------------------------------------------------------------------------------------------------

JSON dataset schemas consumed by powerbi.com to create and push data in (via GET and POST):

{
"name": "@DatasetName",
"defaultMode": "PushStreaming",
"tables": [
{
"name": "DataHeatResults",
"columns": [
{
"name": "Record_ID",
"dataType": "Int64"
},
{
"name": "Date_Time",
"dataType": "Datetime"
},
{
"name": "DryerID",
"dataType": "string"
}, .......................etc.

2) -----------------------------------------------------------------------------------------------------

Model Classes used in C# code to work on assembling and marshalling specific data rows:

using System;

namespace PushDemo.Models
{
public class DataHeatResults
{
public int Record_ID_ { get; set; }
public DateTime Date_Time { get; set; }
public string DryerID { get; set; }
public string RunID { get; set; }

............................etc.

-----------------------------------------------------------------------------------------------------

I have been advised that all of these must be in perfect alignment from a data type, order of declaration etc. perspective or 404 errors can result.  It does not matter that all data elements in the data sources are unused - nothing can be omitted.  Also, even though the data types associated with Powerbi.com do not perfectly align with those in C#, they must be mapped perfectly or you can get a 404.

This is not intuitive to me, but my next step is to do another pass of fine-detail code analysis to ensure that I have the best possible matching of what is being sent within the powerbi.com context and the C# context.  With this advice I will take a hard look at problems that might arising from (for example) Int64 in powerbi.com not being compatible with int in C#.

Because of the inordinate amount of time I've spent dealing with this house of cards I will report back so others may be spared being stuck like this, where everything looks perfect, the URI's used to create the powerbi.com dataset schema are proved to be identical with those being used to do the POST / push, and casual assumptions that data types between powerbi.com, JSON and C# are compatible.  All dataset IDs have been checked, the datasets created in powerbi.com are identical in every way I can see (looking at a report I create to see the datasets that way).

The only anomoly in evidence is a 404 at the moment of truth where data is pushed into a fresh dataset just created by the same code.

Adding to this frustration is the fact that I am using the same code to create a different dataset, and there is no problem pushing into that dataset.

Go figure!

As always, it will turn into some simple mistake somewhere.  But this is a lot to go through.........  maybe some kind of more explanatory error message than a 404 might save thousands of hours of wasted debugging time across the world!  

If anyone has an idea for how to do this more simply, I'd be happy to refactor and take a different approach....

Thanks, Stephen

[sarogersz · Accepted solution]

To close the loop:

I finally found a one-character typo in a field called Record_Id_ where the trailing underscore was left off.  Add that character to the Json  file (misspelled as Record_Id) and poof the 404 error is gone.

The lesson here is that in crafting a solution for push, it is possible to have no problem at all successfully creating a model remotely using the API, and then when you follow up with the push post, to error out with a 404.  This error leads you down the path of looking for URL formation problems, but in reality it has nothing to do with your URL.  It is an internal error in the service somewhere that is caused by a data model field mismatch (or data type mismatch possibly as well - I found that error before the above error was found).

I am refactoring my code to eliminate the manual process of creating jsons and object classes in code, and this particular issue shouldn't happen again.

I'll update this again with how I eliminate the manual process - stay tuned!

Stephen

[sonoilconte]

I also faced this error while trying out the Power BI API for streaming datasets. To me, 404 is a misleading error for a mismatch between JSON and the schema defined in Power BI, and it sent my brain down the wrong troubleshooting path. 400 Bad Request would be more acurate.