cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
devssolution
Frequent Visitor

Rest API with large data

Hello Friends,

 

I want to create power bi dashboard using Mongo DB, but I think right now Mongo DB conector is not avilable for power bi. 

 

So I have create Rest API and connect power bi using "Web Connector".  Now I need your suggestion , How can I handle (approx or more than 10 Million) data in Rest API. What is best way in Power Bi.

 

 

Regards,

Sandeep.

4 REPLIES 4
v-sihou-msft
Microsoft
Microsoft

@devssolution

 

In this scenario, since you are connecting Mongo DB, I suggest you use MongoDB ODBC Driver to connect it in Power BI Desktop instead of using REST API. Then publish it onto Power BI Service for creating dashboard. For more details, please see: MongoDB ODBC Driver for Data Integration with Power BI.

 

Regards,

Thanks For Your Reply,

 

Can we use Elasticsearch Rest API insted of Mongo DB?

 

I need to handle big data. which is best way for it?

 

 

hello devssolution, did you solve your problem with api?

If you did it, could you  share the way of large data transfer?

I have implement Paiging in My Rest API and using Looping in Power Bi M Query.


 

let
    name="^^^^^^^^^^^^^^^^^^",
   url="http://*******************",
   searchSize=5000,
   GetJson = Web.Contents(url,[Query=[name=name,searchSize=Text.From(searchSize),searchFrom="0",pageId="ALL",scrollTime="5m"]]),
   FormatAsJson = Json.Document(GetJson),
   pageId= FormatAsJson[pageId],
   pageNumber = Int32.From(FormatAsJson[count]/searchSize)+1,
 FnGetOnePage =
  (url1,pageId1,page) as record =>
   let
    Source = Json.Document(Web.Contents(url,[Query=[name=name,,searchSize=Text.From(searchSize),searchFrom=Text.From(page),pageId=pageId1,scrollTime="5m"]])),
    pageId2= try Source[pageId] otherwise pageId1,
    data = try Source[message] otherwise null,
    res = [Data=data, PageId= pageId2]
   in
    res,
 GeneratedList =
  List.Generate(
   ()=>[i=0, res = FnGetOnePage(url,"ALL",i)],
   each [i]<pageNumber and [res][Data]<>null,
   each [i=[i]+1, res = FnGetOnePage(url,[res][PageId],i)],
   each [res][Data]),
    #"Converted to Table" = Table.FromList(GeneratedList, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
   #"Expanded Column1" = Table.ExpandListColumn(#"Converted to Table", "Column1")
 in
    #"Expanded Column1"

 

Helpful resources

Announcements
PBI User Groups

Welcome to the User Group Public Preview

Check out new user group experience and if you are a leader please create your group!

MBAS Attendee Badge

Claim Your Badge & Digital Swag!

Check out how to claim yours today!

secondImage

Are You Ready?

Test your skills now with the Cloud Skills Challenge.

Top Solution Authors