RTR of a database transaction to a replica table may include receiving replication and transaction commit log entries (representing a database transaction). The replication log entry has a row-ID value, and the row at the replica table has a row-ID value. The replication log entry may be dispatched to a parallel log replayer and the associated transaction commit log entry to a transaction commit log replayer. The row-ID values may be compared, and the replication log entry is replayed at the parallel log replayer based on the comparison. The database transaction may then be committed to the replica table by replaying the associated transaction commit log entry at the transaction log replayer, wherein the database transaction is associated with row-level parallel replay having transactional consistency and DDL replication and reconstruction of a DDL statement at the replica system is associated with one or multiple metadata update log entries.
Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A system for Real-time Table Replication (“RTR”) of a database transaction to a replica table, comprising: a computer memory; and at least one computer processor coupled to the memory and configured to: receive a replication log entry and an associated transaction commit log entry, the replication log entry and the associated transaction commit log entry together representing a database transaction to be replayed to a row at a replica table, the replication log entry having a row-ID value and the row at the replica table having a row-ID value, dispatch the replication log entry to a parallel log replayer and the associated transaction commit log entry to a transaction commit log replayer, compare the row-ID value of the replication log entry to the row-ID value of the row at the replica table, replay the replication log entry at the parallel log replayer based on the comparison, and commit the database transaction to the replica table by replaying the associated transaction commit log entry at the transaction log replayer, wherein the database transaction is associated with row-level parallel replay having transactional consistency and Data Dictionary Language (“DDL”) replication and reconstruction of a DDL statement at the replica system is associated with one or multiple metadata update log entries, and further wherein push-based and early log shipping reduce propagation delay between a source system and a replica system.
This invention relates to real-time database replication systems, specifically for efficiently replicating database transactions to a replica table while maintaining transactional consistency and supporting Data Dictionary Language (DDL) operations. The system addresses the challenge of minimizing propagation delay between a source database and its replica while ensuring accurate and parallelized replication of transactions. The system includes a computer memory and at least one processor configured to process replication log entries and associated transaction commit log entries. These entries collectively represent a database transaction to be replayed at a replica table. Each replication log entry contains a row-ID value, which is compared to the row-ID of the corresponding row in the replica table. The system dispatches replication log entries to a parallel log replayer and transaction commit log entries to a transaction commit log replayer. The parallel log replayer replays the log entry based on the row-ID comparison, while the transaction commit log replayer commits the transaction to the replica table. The system supports row-level parallel replay with transactional consistency, ensuring that replicated transactions are applied in the correct order. It also handles DDL replication by reconstructing DDL statements from one or multiple metadata update log entries. To reduce propagation delay, the system employs push-based and early log shipping mechanisms, which expedite the transfer of log entries from the source system to the replica system. This approach enhances replication efficiency and minimizes latency in distributed database environments.
2. The system of claim 1 , wherein multiple replication object granularities are supported, including at least one of: (i) a set of tables, (ii) a table, (iii) a sub-table, (iv) one or more columns, and (v) one or more partitions.
This invention relates to a data replication system that supports multiple granularities for replicating database objects. The system addresses the challenge of efficiently replicating data at different levels of granularity in database environments, where traditional replication methods often lack flexibility in selecting the specific data subsets to replicate. The system enables replication of entire databases, individual tables, sub-tables, specific columns, or database partitions, allowing users to tailor replication to their specific needs. This granularity control is particularly useful in scenarios requiring partial replication, such as when only certain tables or columns need to be synchronized across systems. The system dynamically adjusts replication operations based on the selected granularity, ensuring consistency and minimizing unnecessary data transfer. By supporting these various replication levels, the system provides a more efficient and adaptable solution for database synchronization compared to prior art methods that only support coarse-grained replication. The invention is applicable in distributed database systems, cloud environments, and any scenario requiring selective data replication.
3. The system of claim 1 , wherein replication having a topology from multiple distinct remote source systems is supported as N-to-1 replication.
This invention relates to data replication systems that support N-to-1 replication topologies, where multiple distinct remote source systems replicate data to a single target system. The system enables efficient and scalable data synchronization from multiple independent sources to a centralized destination, addressing challenges in distributed data management where data must be consolidated from geographically dispersed or organizationally separate systems. The replication process ensures data consistency, minimizes conflicts, and maintains performance across heterogeneous source systems. The system may include mechanisms for conflict resolution, data transformation, and real-time or batch-based synchronization to handle varying data formats and update frequencies. By supporting N-to-1 replication, the system simplifies data integration in scenarios such as multi-region deployments, mergers, or federated data environments where centralized access to distributed data is required. The invention may also include features for monitoring replication status, error handling, and ensuring data integrity during the transfer process. This approach reduces the complexity of managing multiple independent replication streams while maintaining reliability and scalability.
4. The system of claim 1 , wherein replication having a topology from to multiple distinct remote replica systems is supported as 1-to-N replication.
A distributed data replication system supports 1-to-N replication, enabling a primary system to replicate data to multiple distinct remote replica systems in a 1-to-N topology. The system ensures data consistency and availability across geographically distributed locations by maintaining synchronized copies of data in multiple remote systems. This approach improves fault tolerance and disaster recovery capabilities by allowing data to be replicated to multiple independent sites, reducing the risk of data loss due to single-point failures. The replication process involves transferring data changes from the primary system to each remote replica system, ensuring that all replicas remain up-to-date with the latest modifications. The system may include mechanisms for conflict resolution, network latency handling, and bandwidth optimization to maintain efficient and reliable replication. This topology is particularly useful in enterprise environments where high availability and data redundancy are critical requirements. The system may also support configurable replication policies, allowing administrators to define which data sets are replicated and the frequency of updates. By distributing data across multiple remote systems, the system enhances scalability and performance, enabling faster access to data for geographically dispersed users. The replication process may be implemented using synchronous or asynchronous methods, depending on the specific requirements of the deployment.
5. The system of claim 1 , wherein replication having a topology with a replica table being a source of another replica table is supported as chain replication.
This invention relates to data replication systems, specifically addressing the challenge of efficiently managing and replicating data across distributed systems while maintaining consistency and minimizing latency. The system supports chain replication, a topology where a replica table serves as the source for another replica table, enabling scalable and fault-tolerant data distribution. In chain replication, data is propagated sequentially through a series of nodes, where each node acts as both a consumer and a producer, ensuring ordered and reliable replication. This approach reduces network overhead and improves fault tolerance by allowing data to be replicated even if intermediate nodes fail. The system dynamically manages replication paths, ensuring that data integrity is maintained across the chain. Additionally, the system may include mechanisms for conflict resolution, load balancing, and performance optimization to handle high-throughput environments. The invention is particularly useful in distributed databases, cloud storage systems, and large-scale data processing applications where data consistency and availability are critical. By supporting chain replication, the system provides a robust solution for maintaining synchronized data across geographically dispersed locations while minimizing latency and ensuring fault tolerance.
6. The system of claim 1 , wherein in-memory log replication does not rely on a store-and-forward mechanism.
The system relates to in-memory log replication in distributed computing environments, addressing the inefficiency and latency issues associated with traditional store-and-forward replication mechanisms. In conventional systems, log entries are typically stored temporarily before being forwarded to replicas, introducing delays and increasing resource overhead. This system eliminates the need for intermediate storage by enabling direct, real-time replication of log entries between nodes without relying on a store-and-forward approach. The system ensures data consistency and fault tolerance by maintaining synchronized in-memory logs across distributed nodes, allowing for immediate propagation of updates. This method reduces latency, minimizes storage requirements, and improves overall system performance by streamlining the replication process. The system is particularly useful in high-throughput environments where low-latency data synchronization is critical, such as financial transactions, real-time analytics, and distributed databases. By avoiding the overhead of temporary storage, the system enhances scalability and reliability while maintaining data integrity across distributed systems.
7. The system of claim 1 , wherein there is a separate transaction domain between the source system and the replica system.
This invention relates to data replication systems, specifically addressing the challenge of maintaining data consistency and integrity between a source system and a replica system while ensuring transactional isolation. The system includes a separate transaction domain positioned between the source system and the replica system to manage data replication. This transaction domain acts as an intermediary layer that processes transactions from the source system before propagating them to the replica system. The transaction domain ensures that transactions are applied in a controlled manner, preventing conflicts and maintaining consistency between the source and replica systems. It may include mechanisms for conflict resolution, transaction validation, and synchronization to handle concurrent operations and ensure data integrity. The system is designed to support high availability and fault tolerance by isolating transaction processing from the source and replica systems, reducing the risk of data corruption or inconsistencies during replication. The transaction domain may also include logging and monitoring capabilities to track transaction status and performance, enabling administrators to diagnose and resolve issues efficiently. This approach improves reliability and scalability in distributed data environments where maintaining synchronized replicas is critical.
8. The system of claim 1 , wherein there is a separate metadata domain between the source system and the replica system.
A system for managing data replication includes a separate metadata domain positioned between a source system and a replica system. The metadata domain acts as an intermediary layer that facilitates the transfer and synchronization of data between the two systems. The source system generates or stores original data, while the replica system maintains a copy of that data for redundancy, backup, or distributed access purposes. The metadata domain handles the metadata associated with the data transfer, such as tracking changes, ensuring consistency, and managing replication policies. This separation of the metadata domain from the source and replica systems allows for more efficient and reliable data replication, as it centralizes metadata operations and reduces the load on the source system. The system may also include mechanisms for conflict resolution, data validation, and performance optimization within the metadata domain. By isolating the metadata domain, the system improves scalability, fault tolerance, and maintainability of the replication process.
9. The system of claim 1 , wherein there are different software binary versions between the source system and the replica system.
This invention relates to a system for managing software deployments across distributed systems, specifically addressing the challenge of maintaining consistency between a source system and a replica system when different software binary versions are present. The system ensures that updates or changes made to the source system are accurately replicated to the replica system, even when the two systems operate with different versions of the same software. The core functionality involves detecting discrepancies between the binary versions, identifying compatible update mechanisms, and applying changes in a way that preserves system integrity. The system may include components for version comparison, conflict resolution, and automated synchronization to handle version mismatches without manual intervention. This approach is particularly useful in environments where systems must remain operational while undergoing updates, such as in cloud computing, distributed databases, or enterprise applications. The invention aims to reduce downtime and minimize errors during software updates by dynamically adapting to version differences between systems.
10. A computer-implemented method for real-time table replication of a database transaction to a replica table, comprising: receiving, by at least one processor, a replication log entry and an associated transaction commit log entry, the replication log entry and the associated transaction commit log entry together representing a database transaction to be replayed to a row at a replica table, the replication log entry having a row-ID value and the row at the replica table having a row-ID value; dispatching, by the at least one processor, the replication log entry to a parallel log replayer and the associated transaction commit log entry to a transaction commit log replayer; comparing, by the at least one processor, the row-ID value of the replication log entry to the row-ID value of the row at the replica table; replaying, by the at least one processor, the replication log entry at the parallel log replayer based on the comparison; and committing, by the at least one processor, the database transaction to the replica table by replaying the associated transaction commit log entry at the transaction log replayer, wherein the database transaction is associated with row-level parallel replay having transactional consistency and Data Dictionary Language (“DDL”) replication and reconstruction of a DDL statement at the replica system is associated with one or multiple metadata update log entries, and further wherein push-based and early log shipping reduce propagation delay between a source system and a replica system.
This invention relates to real-time database replication, specifically a method for efficiently replicating database transactions to a replica table while maintaining transactional consistency and supporting Data Dictionary Language (DDL) operations. The problem addressed is the delay and complexity in replicating database transactions, particularly in distributed systems where consistency and low-latency propagation are critical. The method involves receiving a replication log entry and its associated transaction commit log entry, which together represent a database transaction to be replayed. The replication log entry includes a row-ID value, which is compared to the row-ID of the corresponding row in the replica table. This comparison ensures accurate replay of the transaction. The replication log entry is dispatched to a parallel log replayer, while the transaction commit log entry is sent to a transaction commit log replayer. The system supports row-level parallel replay, allowing multiple transactions to be processed concurrently while maintaining consistency. DDL operations are handled by reconstructing DDL statements from one or multiple metadata update log entries, ensuring schema changes are properly replicated. The method also incorporates push-based and early log shipping techniques to minimize propagation delay between the source and replica systems, improving real-time synchronization. This approach enhances performance in distributed database environments where low-latency replication is essential.
11. The method of claim 10 , wherein multiple replication object granularities are supported, including at least one of: (i) a set of tables, (ii) a table, (iii) a sub-table, (iv) one or more columns, and (v) one or more partitions.
This invention relates to database replication systems, specifically methods for supporting multiple replication object granularities in a distributed database environment. The problem addressed is the need for flexible and efficient data replication across distributed systems, where different applications or use cases may require replication at varying levels of granularity, such as entire databases, subsets of tables, individual columns, or specific partitions. The method enables replication of database objects at different levels of granularity, including a set of tables, a single table, a sub-table (a portion of a table), one or more columns within a table, and one or more partitions of a table. This flexibility allows for optimized replication strategies based on specific requirements, such as minimizing network overhead, ensuring data consistency, or supporting partial updates. The system dynamically selects the appropriate replication granularity based on factors like data size, update frequency, and application needs, ensuring efficient and scalable data distribution across distributed systems. The method also supports conflict resolution and synchronization mechanisms to maintain data integrity during replication. This approach improves performance and reduces resource consumption compared to traditional replication methods that rely on fixed granularity levels.
12. The method of claim 10 , wherein replication having a topology from multiple distinct remote source systems is supported as N-to-1 replication.
This invention relates to data replication systems, specifically addressing the challenge of efficiently replicating data from multiple distinct remote source systems into a single target system. The problem solved is the complexity and inefficiency of traditional replication methods when handling multiple source systems, which often require separate replication processes or intermediate consolidation steps. The method supports N-to-1 replication, where data from multiple distinct remote source systems is replicated to a single target system. Each remote source system operates independently, and the method ensures that data from all sources is accurately and consistently replicated to the target system without conflicts or data loss. The replication process includes mechanisms to handle differences in data formats, schemas, or protocols between the source systems and the target system, ensuring seamless integration. The method also includes conflict resolution strategies to manage situations where the same data is updated in multiple source systems, ensuring data integrity in the target system. Additionally, the method may include monitoring and logging features to track replication status, performance, and errors across all source systems. The system is designed to scale efficiently, allowing for the addition or removal of source systems without disrupting ongoing replication processes. This approach simplifies data consolidation from multiple sources, reducing operational overhead and improving data availability.
13. The method of claim 10 , wherein replication having a topology from to multiple distinct remote replica systems is supported as 1-to-N replication.
This invention relates to data replication systems, specifically addressing the challenge of efficiently replicating data from a single source to multiple distinct remote systems in a 1-to-N topology. The method enables a primary system to replicate data to multiple remote replica systems simultaneously, ensuring data consistency and availability across distributed environments. The replication process involves establishing connections between the primary system and each remote replica, synchronizing data changes in real-time or near real-time, and managing conflicts or errors that may arise during replication. The system supports various replication modes, including synchronous and asynchronous replication, to accommodate different performance and reliability requirements. Additionally, the method includes mechanisms for monitoring replication status, detecting failures, and recovering from errors to maintain data integrity. The invention is particularly useful in distributed computing environments, cloud storage systems, and enterprise applications where data must be replicated across multiple locations for redundancy, disaster recovery, or performance optimization. The 1-to-N replication topology allows for scalable and efficient data distribution, reducing latency and improving accessibility for end-users.
14. The method of claim 10 , wherein replication having a topology with a replica table being a source of another replica table is supported as chain replication.
A system and method for data replication in distributed databases supports chain replication, where a replica table serves as the source for another replica table. This approach enables efficient data propagation across multiple nodes in a distributed system, reducing latency and improving consistency. The method involves configuring a primary data source to replicate data to a first replica table, which then acts as a source for a second replica table, forming a chain. Each replication step ensures data integrity and consistency between the source and subsequent replicas. The system monitors replication status, detects failures, and automatically recovers from errors to maintain continuous data availability. This chain replication topology is particularly useful in large-scale distributed databases where direct replication from a single source to multiple replicas may be inefficient or impractical. The method optimizes network bandwidth usage and minimizes replication delays by leveraging intermediate replicas as sources for downstream replicas. The system also supports dynamic adjustments to the replication chain based on network conditions or system performance requirements. This approach enhances scalability and reliability in distributed data environments.
15. The method of claim 10 , wherein in-memory log replication does not rely on a store-and-forward mechanism.
A system and method for in-memory log replication in distributed computing environments addresses the challenge of ensuring data consistency and reliability across multiple nodes without relying on traditional store-and-forward mechanisms. In distributed systems, maintaining synchronized logs across nodes is critical for fault tolerance and data integrity, but conventional approaches often introduce latency and complexity due to intermediate storage and forwarding steps. This invention eliminates the need for such mechanisms by enabling direct, real-time replication of log entries between nodes using in-memory techniques. The method involves capturing log entries generated by applications or system processes and transmitting them directly to one or more replica nodes without intermediate storage. Each replica node processes the log entries in real time, ensuring immediate consistency across the distributed system. The approach reduces latency, minimizes resource overhead, and enhances fault tolerance by avoiding dependencies on external storage systems or network buffers. Additionally, the method supports configurable replication policies, allowing for customization based on performance and reliability requirements. By operating entirely in memory, the system achieves high throughput and low-latency replication, making it suitable for high-performance computing, financial transactions, and real-time data processing applications. The invention ensures that log entries are replicated reliably and efficiently, even in the presence of network partitions or node failures, by leveraging in-memory replication protocols that prioritize immediate synchronization over intermediate storage.
16. The method of claim 10 , wherein there is a separate transaction domain between the source system and the replica system.
A system and method for managing data replication between a source system and a replica system involves a separate transaction domain to ensure data consistency and integrity during replication. The transaction domain acts as an intermediary layer that isolates the source and replica systems, allowing transactions to be processed independently before being committed to the replica. This separation prevents direct conflicts between the source and replica systems, reducing the risk of data corruption or inconsistencies. The transaction domain may include mechanisms for validating, sequencing, or transforming transactions before they are applied to the replica system, ensuring that the replica remains an accurate and up-to-date reflection of the source system. The method may also include monitoring the transaction domain to detect and resolve conflicts or errors, maintaining high availability and reliability of the replication process. This approach is particularly useful in distributed systems, cloud computing environments, or any scenario where maintaining data consistency across multiple systems is critical. The transaction domain may be implemented as a software module, a dedicated server, or a network-based service, depending on the specific requirements of the source and replica systems.
17. The method of claim 10 , wherein there is a separate metadata domain between the source system and the replica system.
A system and method for data replication involves a separate metadata domain positioned between a source system and a replica system to facilitate efficient and reliable data synchronization. The source system generates data that needs to be replicated to the replica system, which may be located remotely or in a different environment. The metadata domain acts as an intermediary layer that manages metadata associated with the data being replicated, ensuring consistency, tracking changes, and optimizing the replication process. This metadata domain may include mechanisms for conflict resolution, version control, and performance monitoring to enhance the reliability and efficiency of data replication. The system may also incorporate a replication engine that processes the data and metadata, applying rules and policies to ensure accurate and timely replication. The metadata domain may further support features such as data validation, error handling, and recovery procedures to maintain data integrity during the replication process. This approach improves the scalability and robustness of data replication across distributed systems.
18. The method of claim 10 , wherein there are different software binary versions between the source system and the replica system.
A system and method for managing software updates in distributed computing environments addresses the challenge of maintaining consistency between a source system and a replica system when different software binary versions exist. The method involves detecting discrepancies in software versions between the source and replica systems, identifying the specific differences in binary versions, and applying selective updates to the replica system to align it with the source system while minimizing downtime and ensuring operational continuity. The process includes analyzing the binary versions to determine compatibility and compatibility requirements, generating a compatibility report, and applying updates in a phased manner to avoid disruptions. The system may also include mechanisms for rollback in case of update failures, ensuring system stability. The method is particularly useful in environments where continuous availability is critical, such as cloud computing, distributed databases, or high-availability systems. By dynamically adjusting updates based on version differences, the system ensures that the replica system remains synchronized with the source system without requiring a full system reboot or complete reinstallation of software. This approach reduces the risk of errors and improves efficiency in maintaining system consistency across distributed environments.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
November 18, 2019
March 1, 2022
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.