Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
devssolution
Frequent Visitor

Rest API with large data

Hello Friends,

 

I want to create power bi dashboard using Mongo DB, but I think right now Mongo DB conector is not avilable for power bi. 

 

So I have create Rest API and connect power bi using "Web Connector".  Now I need your suggestion , How can I handle (approx or more than 10 Million) data in Rest API. What is best way in Power Bi.

 

 

Regards,

Sandeep.

4 REPLIES 4
v-sihou-msft
Employee
Employee

@devssolution

 

In this scenario, since you are connecting Mongo DB, I suggest you use MongoDB ODBC Driver to connect it in Power BI Desktop instead of using REST API. Then publish it onto Power BI Service for creating dashboard. For more details, please see: MongoDB ODBC Driver for Data Integration with Power BI.

 

Regards,

Thanks For Your Reply,

 

Can we use Elasticsearch Rest API insted of Mongo DB?

 

I need to handle big data. which is best way for it?

 

 

hello devssolution, did you solve your problem with api?

If you did it, could you  share the way of large data transfer?

I have implement Paiging in My Rest API and using Looping in Power Bi M Query.


 

let
    name="^^^^^^^^^^^^^^^^^^",
   url="http://*******************",
   searchSize=5000,
   GetJson = Web.Contents(url,[Query=[name=name,searchSize=Text.From(searchSize),searchFrom="0",pageId="ALL",scrollTime="5m"]]),
   FormatAsJson = Json.Document(GetJson),
   pageId= FormatAsJson[pageId],
   pageNumber = Int32.From(FormatAsJson[count]/searchSize)+1,
 FnGetOnePage =
  (url1,pageId1,page) as record =>
   let
    Source = Json.Document(Web.Contents(url,[Query=[name=name,,searchSize=Text.From(searchSize),searchFrom=Text.From(page),pageId=pageId1,scrollTime="5m"]])),
    pageId2= try Source[pageId] otherwise pageId1,
    data = try Source[message] otherwise null,
    res = [Data=data, PageId= pageId2]
   in
    res,
 GeneratedList =
  List.Generate(
   ()=>[i=0, res = FnGetOnePage(url,"ALL",i)],
   each [i]<pageNumber and [res][Data]<>null,
   each [i=[i]+1, res = FnGetOnePage(url,[res][PageId],i)],
   each [res][Data]),
    #"Converted to Table" = Table.FromList(GeneratedList, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
   #"Expanded Column1" = Table.ExpandListColumn(#"Converted to Table", "Column1")
 in
    #"Expanded Column1"

 

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.