Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.
We are experiencing huge delays when loading big datasets from our FHIR Server.
I am talking about 1 Mio+ records, which take close to a day to fully load.
The main problem seems to be the way how the FHIR Connector is loading the data:
- Small sets with small offset (about 200 for each request)
- No utilization of FHIR export feature
Is there any way to tweak the connector e.g. by overwriting the default offset size?
Our server would have no problems to serve sth. like 25k records with each request.
Any update on this issue?
The problem is still existing.
I honestly don't understand why the PBI FHIR Connector does not ship with an option to set the page size. Queryfolding is supported which is much harder to implement.
Providing a parameter to set the page size should be easy to be tackled by the team, would love to see it on the roadmap.
Hi @DataInsights ,
thanks for the advice. It's pretty nice explained, but only works for local datasets.
However the problem with FHIR is that it takes the most time to load all datapoints 1by1 (buckets of 100-500) from the server instead of incresing the offset size. So you loose a lot of time with establishing unnecessary requests.
A colleague alerted me to this interesting technique, and got a 10x improvement in his query. Let me know if it works for you.
Proud to be a Super User!
Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City
Check out the April 2024 Power BI update to learn about new features.
User | Count |
---|---|
110 | |
95 | |
76 | |
65 | |
51 |
User | Count |
---|---|
146 | |
109 | |
106 | |
88 | |
61 |