Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn a 50% discount on the DP-600 certification exam by completing the Fabric 30 Days to Learn It challenge.

Reply
razieh1990
Helper I
Helper I

live data stream from oracle to powerbi

Hello 

I have an inquiry and would be grateful for assistance. To avoid potential technical issues or causing Oracle to crash, I cannot establish a direct query connection. Nevertheless, I require a real-time data stream for my report, given the substantial daily data volume. Do you have any recommendations for creating a push dataset and maintaining a live data stream without causing any technical disruptions to the Oracle database?

1 ACCEPTED SOLUTION
123abc
Community Champion
Community Champion

Certainly! If you cannot establish a direct query connection from Power BI to Oracle due to technical limitations or concerns about performance impact, you can consider alternative approaches to achieve a live data stream in Power BI.

Here's a high-level approach you can consider:

  1. Change Data Capture (CDC): Implement CDC on your Oracle database. CDC captures changes made to the database and stores them in a separate change table. This allows you to track inserts, updates, and deletes in near real-time.

  2. Data Integration Platform: Use a data integration platform like Apache Kafka, Apache NiFi, or AWS Kinesis to stream data from Oracle to Power BI. These platforms provide reliable, scalable, and real-time data streaming capabilities.

  3. Event-Driven Architecture: Design an event-driven architecture where changes in the Oracle database trigger events that are captured and processed by an intermediary system. These events can then be pushed to Power BI for visualization.

  4. Custom ETL Process: Develop a custom Extract, Transform, Load (ETL) process that periodically extracts data from Oracle, transforms it into a suitable format, and loads it into Power BI. You can schedule this process to run at regular intervals to keep your Power BI dataset updated.

  5. Change Data Capture Tools: Explore third-party change data capture tools that are compatible with Oracle databases. These tools are specifically designed to capture database changes efficiently and can integrate with Power BI for real-time reporting.

  6. API Integration: If Oracle provides an API for accessing data, you can integrate Power BI with the Oracle API to fetch real-time data. This approach requires the Oracle API to support the necessary data retrieval operations.

  7. Hybrid Approach: Consider a hybrid approach where you combine near real-time streaming with periodic data refreshes. This allows you to balance the need for real-time insights with the limitations of your Oracle database environment.

Before implementing any of these approaches, it's essential to assess the impact on your Oracle database performance, network bandwidth, and overall system stability. You should also consider security requirements and compliance regulations when streaming data from Oracle to Power BI.

Choose the approach that best fits your technical requirements, resource constraints, and scalability needs. Additionally, consult with your IT and database administration teams to ensure that the chosen solution aligns with your organization's policies and guidelines.

 

If this post helps, then please consider Accepting it as the solution to help the other members find it more quickly.

 

In case there is still a problem, please feel free and explain your issue in detail, It will be my pleasure to assist you in any way I can.

View solution in original post

3 REPLIES 3
razieh1990
Helper I
Helper I

Hello 

 

I have an inquiry and would be grateful for assistance. To avoid potential technical issues or causing Oracle to crash, I cannot establish a direct query connection. Nevertheless, I require a real-time data stream for my report, given the substantial daily data volume. Do you have any recommendations for creating a push dataset and maintaining a live data stream without causing any technical disruptions to the Oracle database?

Hello 

I have an inquiry and would be grateful for assistance. To avoid potential technical issues or causing Oracle to crash, I cannot establish a direct query connection. Nevertheless, I require a real-time data stream for my report, given the substantial daily data volume. Do you have any recommendations for creating a push dataset and maintaining a live data stream without causing any technical disruptions to the Oracle database?

123abc
Community Champion
Community Champion

Certainly! If you cannot establish a direct query connection from Power BI to Oracle due to technical limitations or concerns about performance impact, you can consider alternative approaches to achieve a live data stream in Power BI.

Here's a high-level approach you can consider:

  1. Change Data Capture (CDC): Implement CDC on your Oracle database. CDC captures changes made to the database and stores them in a separate change table. This allows you to track inserts, updates, and deletes in near real-time.

  2. Data Integration Platform: Use a data integration platform like Apache Kafka, Apache NiFi, or AWS Kinesis to stream data from Oracle to Power BI. These platforms provide reliable, scalable, and real-time data streaming capabilities.

  3. Event-Driven Architecture: Design an event-driven architecture where changes in the Oracle database trigger events that are captured and processed by an intermediary system. These events can then be pushed to Power BI for visualization.

  4. Custom ETL Process: Develop a custom Extract, Transform, Load (ETL) process that periodically extracts data from Oracle, transforms it into a suitable format, and loads it into Power BI. You can schedule this process to run at regular intervals to keep your Power BI dataset updated.

  5. Change Data Capture Tools: Explore third-party change data capture tools that are compatible with Oracle databases. These tools are specifically designed to capture database changes efficiently and can integrate with Power BI for real-time reporting.

  6. API Integration: If Oracle provides an API for accessing data, you can integrate Power BI with the Oracle API to fetch real-time data. This approach requires the Oracle API to support the necessary data retrieval operations.

  7. Hybrid Approach: Consider a hybrid approach where you combine near real-time streaming with periodic data refreshes. This allows you to balance the need for real-time insights with the limitations of your Oracle database environment.

Before implementing any of these approaches, it's essential to assess the impact on your Oracle database performance, network bandwidth, and overall system stability. You should also consider security requirements and compliance regulations when streaming data from Oracle to Power BI.

Choose the approach that best fits your technical requirements, resource constraints, and scalability needs. Additionally, consult with your IT and database administration teams to ensure that the chosen solution aligns with your organization's policies and guidelines.

 

If this post helps, then please consider Accepting it as the solution to help the other members find it more quickly.

 

In case there is still a problem, please feel free and explain your issue in detail, It will be my pleasure to assist you in any way I can.

Helpful resources

Announcements
LearnSurvey

Fabric certifications survey

Certification feedback opportunity for the community.

PBI_APRIL_CAROUSEL1

Power BI Monthly Update - April 2024

Check out the April 2024 Power BI update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors