cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Highlighted
Helper I
Helper I

push data into Power BI Dataset from asp.net web api

Hi,

 

I am getting my data for power bi report from asp.net web api as json. As i am fetching 50-60k records, it takes time to get all data and if there is network traffic, request timeout also happens sometimes.

So my question is:

 

Is there a way to populate dataset as a stream means instead of getting all the data at once. can i fetch some data pushing it to dataset and so on? It is not a live streaming scenario. 

 

Any pointers will be very appreciated. 

 

Thanks.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted
Community Support
Community Support

Re: push data into Power BI Dataset from asp.net web api

Hi @nasirali,

 

If you push data to streaming api, it should be streaming dataset.(streaming dataset not cache data, it can only use with specific tiles)

 

If you mean publish data to rest api and create dataset who stored in power bi service. It should be a normal dataset.

I'm not so sure for timeout issue which you mentioned, but I think you can write a loop function to split these data to suitable size/interval and push them.

 

For power bi rest api limitation, please refer to below link:

Power BI REST API limitations

To POST Rows

  • 75 max columns
  • 75 max tables
  • 10,000 max rows per single POST rows request
  • 1,000,000 rows added per hour per dataset
  • 5 max pending POST rows requests per dataset
  • 120 POST rows requests per minute per dataset
  • If table has 250,000 or more rows, 120 POST rows requests per hour per dataset
  • 200,000 max rows stored per table in FIFO dataset
  • 5,000,000 max rows stored per table in ‘none retention policy’ dataset
  • 4,000 characters per value for string column in POST rows operation

 

Regards,

Xiaoxin Sheng

Community Support Team _ Xiaoxin
If this post helps, please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

4 REPLIES 4
Highlighted
Community Support
Community Support

Re: push data into Power BI Dataset from asp.net web api

Hi @nasirali,

 

What type of power bi you used ? Desktop or service? If you mean desktop, current it not support get streaming data.

If you mean power bi service, you can simply setting an application to push your data to streaming dateset api.

Real-time streaming in Power BI

Push Data into Power BI Streaming Datasets with C#

 

But current power bi not support to use steaming dateseting on normal visuals.

 

Regards,

Xiaoxin Sheng

Community Support Team _ Xiaoxin
If this post helps, please consider Accept it as the solution to help the other members find it more quickly.
Highlighted
Helper I
Helper I

Re: push data into Power BI Dataset from asp.net web api

Hi @v-shex-msft

 

Thank you for reply and links. I am using power bi desktop for development of reports then publish to service for embedding in application. I have a report which is getting data from webapi as json and data is being updated with refresh schedule. Everything is working fine.

 

Only problem is, web api has a request timeout. And if data is large enough it will timeout, So i was thinking if there is a way to push data as i read it. It will be efficient and i will not have this problem. Something like in the second the link you provided. Will it work with normal dataset too or just streaming dataset? 

 

 

Regards,

nasir ali

Highlighted
Community Support
Community Support

Re: push data into Power BI Dataset from asp.net web api

Hi @nasirali,

 

If you push data to streaming api, it should be streaming dataset.(streaming dataset not cache data, it can only use with specific tiles)

 

If you mean publish data to rest api and create dataset who stored in power bi service. It should be a normal dataset.

I'm not so sure for timeout issue which you mentioned, but I think you can write a loop function to split these data to suitable size/interval and push them.

 

For power bi rest api limitation, please refer to below link:

Power BI REST API limitations

To POST Rows

  • 75 max columns
  • 75 max tables
  • 10,000 max rows per single POST rows request
  • 1,000,000 rows added per hour per dataset
  • 5 max pending POST rows requests per dataset
  • 120 POST rows requests per minute per dataset
  • If table has 250,000 or more rows, 120 POST rows requests per hour per dataset
  • 200,000 max rows stored per table in FIFO dataset
  • 5,000,000 max rows stored per table in ‘none retention policy’ dataset
  • 4,000 characters per value for string column in POST rows operation

 

Regards,

Xiaoxin Sheng

Community Support Team _ Xiaoxin
If this post helps, please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

Highlighted
Frequent Visitor

Re: push data into Power BI Dataset from asp.net web api

For none retention policy dataset , what is the behaviour if table reaches the max rows of 5,000,000 as specified below.

I believe it is not FIFO and there is no delete individual rows APIs available. If we need to push new data to it, how do we do it ? Is flush all the rows and start again the only option ?

 

Thanks,

Kiran

Helpful resources

Announcements
August 2020 Community Challenge: Can You Solve These?

August 2020 Community Challenge: Can You Solve These?

We're excited to announce our first cross-community 'Can You Solve These?' challenge!

Community Blog

Community Blog

Visit our Community Blog for articles, guides, and information created by fellow community members.

Upcoming Events

Upcoming Events

Wondering what events you could join or have an event to promote yourself? Check out our Upcoming Events.

Get Ready for Power BI Dev Camp

Get Ready for Power BI Dev Camp

We are thrilled to announce we will begin running a monthly webinar series named Power BI Dev Camp.

Top Kudoed Authors