cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
devssolution
Frequent Visitor

Rest API with large data

Hello Friends,

 

I want to create power bi dashboard using Mongo DB, but I think right now Mongo DB conector is not avilable for power bi. 

 

So I have create Rest API and connect power bi using "Web Connector".  Now I need your suggestion , How can I handle (approx or more than 10 Million) data in Rest API. What is best way in Power Bi.

 

 

Regards,

Sandeep.

4 REPLIES 4
v-sihou-msft
Microsoft
Microsoft

@devssolution

 

In this scenario, since you are connecting Mongo DB, I suggest you use MongoDB ODBC Driver to connect it in Power BI Desktop instead of using REST API. Then publish it onto Power BI Service for creating dashboard. For more details, please see: MongoDB ODBC Driver for Data Integration with Power BI.

 

Regards,

Thanks For Your Reply,

 

Can we use Elasticsearch Rest API insted of Mongo DB?

 

I need to handle big data. which is best way for it?

 

 

hello devssolution, did you solve your problem with api?

If you did it, could you  share the way of large data transfer?

I have implement Paiging in My Rest API and using Looping in Power Bi M Query.


 

let
    name="^^^^^^^^^^^^^^^^^^",
   url="http://*******************",
   searchSize=5000,
   GetJson = Web.Contents(url,[Query=[name=name,searchSize=Text.From(searchSize),searchFrom="0",pageId="ALL",scrollTime="5m"]]),
   FormatAsJson = Json.Document(GetJson),
   pageId= FormatAsJson[pageId],
   pageNumber = Int32.From(FormatAsJson[count]/searchSize)+1,
 FnGetOnePage =
  (url1,pageId1,page) as record =>
   let
    Source = Json.Document(Web.Contents(url,[Query=[name=name,,searchSize=Text.From(searchSize),searchFrom=Text.From(page),pageId=pageId1,scrollTime="5m"]])),
    pageId2= try Source[pageId] otherwise pageId1,
    data = try Source[message] otherwise null,
    res = [Data=data, PageId= pageId2]
   in
    res,
 GeneratedList =
  List.Generate(
   ()=>[i=0, res = FnGetOnePage(url,"ALL",i)],
   each [i]<pageNumber and [res][Data]<>null,
   each [i=[i]+1, res = FnGetOnePage(url,[res][PageId],i)],
   each [res][Data]),
    #"Converted to Table" = Table.FromList(GeneratedList, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
   #"Expanded Column1" = Table.ExpandListColumn(#"Converted to Table", "Column1")
 in
    #"Expanded Column1"

 

Helpful resources

Announcements
MBAS on Demand

2021 Release Wave 2 Plan

Power Platform release plan for the 2021 release wave 2 describes all new features releasing from October 2021 through March 2022.

July 2021 Update 768x460.png

Check it out!

Click here to read more about the July 2021 Updates

Power Query PA Forum 768x460.png

Check it out!

Did you know that you can visit the Power Query Forum in Power BI and now Power Apps

Urdu Hindi D365 Bootcamp 768x460.png

Urdu Hindi D365 Bootcamp

Dont miss our very own April Dunnam’s The Developer Guide to the Galaxy! Find out what the Power Platform has to offer for the traditional developer.

Top Solution Authors
Top Kudoed Authors