Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A method for a key-value (KV) store database management system (DBMS) to provide a KV store to an application, comprising: providing a plurality of low-level stores and a top-level store, wherein: each of the plurality of low-level stores includes key-value pairs and has a respective low-level running transaction accumulating a plurality of changes to the key-value pairs of the low-level store; and the top-level store includes for each of the plurality of low-level stores information leading to the respective low-level store and information indicating which respective low-level running transaction was last committed in the low-level store, and has a top-level running transaction, distinct of a plurality of low-level running transactions of the plurality of low-level stores, accumulating a plurality of changes to the information indicating which respective low-level running transaction was last committed to a low-level store of the plurality of low-level stores; and in response to a plurality of changes made to a plurality of key-value pairs in at least two of the plurality of low-level stores, such that each of the respective at least two low-level transactions of the at least two low-level stores accumulate some of the plurality of changes: committing the respective low-level running transaction of one of the at least two low-level stores, without committing another respective low-level transaction of another of the at least two low-level stores to reduce an amount of writes to secondary memory, comprising: persisting changes to the one low-level store accumulated by the respective low-level running transaction to secondary memory; and updating the top-level store to indicate the respective low-level running transaction was last committed in the one low-level store; and after committing the respective low-level running transaction of the one low-level store, opening a new respective low-level running transaction for the one low-level store.
A key-value (KV) store database management system (DBMS) manages data across multiple low-level stores and a top-level store to optimize write operations to secondary memory. The system addresses inefficiencies in traditional KV stores where frequent commits across multiple stores lead to excessive write operations, degrading performance. Each low-level store contains key-value pairs and maintains a running transaction that accumulates changes to those pairs. The top-level store tracks references to each low-level store and records the last committed transaction for each. The top-level store also has its own running transaction that updates the commit status of low-level stores. When changes are made to key-value pairs across multiple low-level stores, the system selectively commits only one low-level transaction at a time, rather than committing all pending transactions simultaneously. This reduces the number of writes to secondary memory. After committing a low-level transaction, the system persists the accumulated changes to secondary memory and updates the top-level store to reflect the commit. A new transaction is then opened for the committed low-level store. This approach minimizes write overhead while ensuring data consistency across the distributed stores. The system efficiently manages concurrent modifications by decoupling low-level commits from the top-level store, allowing incremental persistence of changes.
2. The method of claim 1 , further comprising determining the one low-level store has accumulated a first number of changes, wherein said committing the respective low-level running transaction for the one low-level store occurs in response to said determining the one low-level store has accumulated the first number of changes.
This invention relates to transaction management in distributed database systems, specifically addressing the challenge of efficiently committing low-level transactions to maintain data consistency while minimizing performance overhead. The system involves multiple low-level stores, each handling independent transactions, and a higher-level transaction manager coordinating these transactions. The method includes monitoring changes in each low-level store and triggering a commit of a running transaction for a particular store when that store accumulates a predefined number of changes. This approach ensures that transactions are committed in batches rather than individually, reducing the frequency of commit operations and improving system performance. The invention also includes mechanisms to handle dependencies between transactions across different stores, ensuring atomicity and consistency even when multiple stores are involved in a single high-level transaction. By dynamically adjusting the commit threshold based on system load or other factors, the system optimizes performance while maintaining data integrity. This method is particularly useful in distributed databases where minimizing commit latency and ensuring consistency are critical.
3. The method of claim 2 , further comprising: determining the top-level store has accumulated a second number of changes to the information indicating which respective low-level running transaction was last committed to the low-level store; and in response to said determining the top-level store has accumulated the second number of changes: committing the top-level running transaction, and opening a new top-level running transaction for the top-level store.
This invention relates to transaction management in distributed database systems, specifically addressing the challenge of efficiently handling transaction commits and rollbacks in multi-level storage architectures. The system involves a top-level store and one or more low-level stores, where transactions are managed hierarchically. The method ensures data consistency by tracking changes in the top-level store and triggering commits based on predefined thresholds. The process begins by monitoring the top-level store for changes, where each change corresponds to a committed transaction from a low-level store. When the top-level store accumulates a specified number of changes, it indicates that multiple low-level transactions have been committed. In response, the system commits the current top-level running transaction and initiates a new top-level transaction. This mechanism prevents excessive accumulation of uncommitted changes, reducing the risk of data inconsistency and improving system performance. The method also includes determining whether the top-level store has reached a second threshold of changes, which may differ from the initial threshold. Upon reaching this second threshold, the system commits the top-level transaction and starts a new one, ensuring periodic synchronization between the top-level and low-level stores. This approach optimizes transaction processing by balancing commit frequency with system efficiency, particularly in environments with high transaction volumes.
4. The method of claim 3 , wherein said committing the top-level running transaction comprises: persisting changes to the top-level store accumulated by the top-level running transaction to secondary memory; and updating the top-level store to indicate the top-level running transaction was last committed in the top-level store.
A method for managing transactional data in a computing system involves handling top-level transactions that may include nested transactions. The system maintains a top-level store that tracks transactional changes and their commit status. When committing a top-level transaction, the method persists all accumulated changes from that transaction to secondary memory, such as a disk or other non-volatile storage. Additionally, the method updates the top-level store to reflect that the transaction was successfully committed, ensuring the system's state remains consistent. This approach allows for reliable recovery and rollback operations by maintaining a clear record of committed transactions. The method is particularly useful in database systems or distributed computing environments where transactional integrity is critical. By separating the persistence and state update steps, the system ensures that committed transactions are durable and that the top-level store accurately reflects the current state of the system. This method improves data consistency and reliability in systems that handle complex, nested transactions.
5. The method of claim 1 , wherein the top-level store includes leaf nodes each pointing to one of the plurality of low-level stores.
A system and method for organizing and accessing data in a hierarchical storage structure. The invention addresses the challenge of efficiently managing large datasets by providing a multi-tiered storage architecture that improves retrieval performance and scalability. The system includes a top-level store that serves as an index or directory, containing leaf nodes that each point to one of multiple low-level stores. Each low-level store contains the actual data entries, allowing the top-level store to act as a navigational layer that directs queries to the appropriate low-level store. This hierarchical structure enables faster data access by reducing the search space at each level and distributing the data across multiple storage units. The invention is particularly useful in applications requiring high-speed data retrieval, such as databases, file systems, or distributed storage networks. The top-level store may use various indexing techniques, such as hash tables, trees, or other data structures, to optimize the mapping between leaf nodes and low-level stores. The low-level stores can be implemented as separate files, databases, or memory segments, depending on the specific use case. This approach enhances performance by minimizing the overhead of searching through large datasets and allows for parallel access to different low-level stores. The system can be adapted for different storage media, including solid-state drives, hard disk drives, or cloud storage, making it versatile for various computing environments.
6. The method of claim 5 , wherein the top-level store further includes a root node pointing to one or more index nodes, each index node pointing to one or more of: one of the leaf nodes and another index node.
A hierarchical data storage system organizes data in a tree structure to improve access efficiency. The system includes a top-level store containing a root node that links to one or more index nodes. Each index node can further connect to either a leaf node, which stores actual data, or another index node, enabling multi-level indexing. This nested structure allows for scalable and efficient data retrieval by reducing the number of direct comparisons needed to locate specific data entries. The hierarchical arrangement minimizes search time by leveraging intermediate index nodes to narrow down the search path before reaching the leaf nodes. This approach is particularly useful in large datasets where direct linear searches would be inefficient. The system ensures that data can be accessed quickly by traversing the tree from the root through the index nodes to the relevant leaf nodes, optimizing performance for read and write operations. The flexible design allows for dynamic expansion by adding new index or leaf nodes as the dataset grows, maintaining efficiency without requiring a complete restructuring of the storage system.
7. A database system, comprising: processor, main memory, and secondary memory implementing: a plurality of low-level stores, each including key-value (KV) pairs and having a respective low-level running transaction accumulating a plurality of changes to the key-value pairs of the low-level store; and a top-level store including for each of the plurality of low-level stores information leading to the respective low-level store and information indicating which respective low-level running transaction was last committed in the low-level store, and has a top-level running transaction, distinct of a plurality of low-level running transactions of the plurality of low-level stores, accumulating a plurality of changes to the information indicating which respective low-level running transaction was last committed to a low-level store of the plurality of low-level stores; and a KV store database management system (DBMS) configured to: in response to a plurality of changes made to a plurality of key-value pairs in at least two of the plurality of low-level stores, such that each of the respective at least two low-level transactions of the at least two low-level stores accumulate some of the plurality of changes: commit the respective low-level running transaction of one of the at least two low-level stores, without committing another respective low-level transaction of another of the at least two low-level stores to reduce an amount of writes to secondary memory, by: persisting changes to the one low-level store accumulated by the respective low-level running transaction to secondary memory; and updating the top-level store to indicate the respective low-level running transaction was last committed in the one low-level store; and after committing the respective low-level running transaction of the one low-level store, open a new respective low-level transaction for the one low-level store.
The database system is designed to optimize transaction handling in a distributed key-value (KV) store by reducing write operations to secondary memory. The system includes multiple low-level stores, each containing KV pairs and a running transaction that accumulates changes to those pairs. A top-level store maintains references to each low-level store and tracks the last committed transaction for each. The top-level store also has its own running transaction that updates the commit status of low-level transactions. When changes are made to KV pairs across multiple low-level stores, the system commits transactions selectively. Instead of committing all pending transactions, it commits only one low-level transaction at a time, persisting its changes to secondary memory and updating the top-level store to reflect the commit. This approach minimizes write operations by avoiding simultaneous commits across multiple stores. After committing a transaction in one store, a new transaction is opened for that store to continue accumulating changes. The system ensures consistency by tracking committed transactions in the top-level store while reducing the overhead of frequent writes to secondary memory. This design is particularly useful in distributed databases where minimizing I/O operations is critical for performance.
8. The system of claim 7 , wherein the KV store DBMS is further configured to determine the one low-level store has accumulated a first number of changes, wherein said commit the respective low-level running transaction of the one low-level store occurs in response to said determine the one low-level store has accumulated at least the first number of changes.
A distributed database management system (DBMS) manages data across multiple low-level storage systems, each handling independent transactions. The system includes a key-value (KV) store DBMS that coordinates these low-level stores, ensuring data consistency and availability. The KV store DBMS monitors changes in each low-level store and triggers a commit of the running transaction for a particular low-level store when it detects that the store has accumulated a predefined number of changes. This mechanism optimizes performance by reducing the frequency of commits, minimizing overhead while maintaining data integrity. The system ensures that transactions are only committed when a sufficient number of changes have been accumulated, balancing efficiency and consistency. The low-level stores operate independently but are synchronized by the KV store DBMS, which enforces commit conditions based on change thresholds. This approach is particularly useful in distributed environments where frequent commits could degrade performance, allowing the system to batch changes before finalizing transactions. The solution addresses the challenge of maintaining efficient and consistent data management in distributed storage systems.
9. The system of claim 8 , wherein the KV store DBMS is further configured to: determine the top-level store has accumulated a second number of changes to the information indicating which respective low-level running transaction was last committed to the low-level store; and in response to said determine the top-level store has accumulated at least the second number of changes: commit the top-level running transaction, and open a new top-level transaction for the top-level store.
This invention relates to database management systems (DBMS) for key-value (KV) stores, specifically addressing performance and consistency challenges in distributed transaction processing. The system manages a hierarchical database structure where a top-level store interacts with one or more low-level stores, each handling independent transactions. The core problem is ensuring efficient and consistent synchronization between these stores, particularly when transactions are committed across different levels. The system monitors changes in the top-level store to track which low-level transactions have been committed. When the top-level store accumulates a predefined number of changes (a second number), it triggers a commit of the current top-level transaction and initiates a new top-level transaction. This mechanism prevents excessive delays in transaction processing while maintaining data consistency. The low-level stores independently manage their own transactions, but the top-level store ensures that committed changes from low-level stores are properly reflected in the higher-level data structure. This approach optimizes performance by reducing the frequency of top-level commits, which can be resource-intensive, while still ensuring that the system remains responsive to changes in the underlying low-level stores. The invention is particularly useful in distributed database environments where transactional integrity and efficiency are critical.
10. The system of claim 9 , wherein said commit the top-level running transaction comprises: persisting changes to the top-level store accumulated by the top-level running transaction to secondary memory; and updating the top-level store to indicate the top-level running transaction was last committed in the top-level store.
A system for managing transactional data in a hierarchical storage architecture addresses the challenge of efficiently committing top-level transactions while maintaining data consistency. The system operates within a multi-tiered storage environment where a top-level store manages high-level transactions, and lower-level stores handle nested transactions. When committing a top-level running transaction, the system first persists all accumulated changes from the transaction to secondary memory, ensuring durability. It then updates the top-level store to reflect that the transaction was successfully committed, marking the transaction's completion. This process ensures that the top-level store accurately reflects the latest committed state, while lower-level transactions remain isolated until their respective commits. The system optimizes performance by minimizing redundant operations and maintaining consistency across storage tiers, particularly in distributed or nested transaction scenarios. The hierarchical approach allows for efficient rollback and recovery mechanisms, as changes are only finalized upon explicit commit, reducing the risk of partial or inconsistent updates. This method is particularly useful in systems requiring high availability and data integrity, such as databases or distributed ledgers.
11. The system of claim 7 , wherein the top-level store includes leaf nodes each pointing to one of the plurality of low-level stores.
A data storage system is designed to improve access efficiency and scalability by organizing data across multiple hierarchical storage layers. The system addresses the challenge of managing large datasets with varying access patterns, where some data is frequently accessed while other data is rarely used. Traditional storage systems often struggle with balancing performance and cost, leading to either high latency for infrequently accessed data or excessive resource consumption for frequently accessed data. The system includes a top-level store that acts as an index or directory, containing leaf nodes that each point to one or more low-level stores. The low-level stores are responsible for storing the actual data, while the top-level store efficiently directs access requests to the correct low-level store. This hierarchical structure allows the system to optimize storage and retrieval operations by separating frequently accessed data from less frequently accessed data, reducing latency and improving overall performance. The top-level store may use metadata or other indexing mechanisms to quickly locate the relevant low-level store for a given data access request. The low-level stores can be implemented using different storage technologies, such as solid-state drives for high-speed access and magnetic disks for cost-effective, high-capacity storage. This modular design enables the system to scale efficiently by adding or removing low-level stores as needed, while the top-level store remains a centralized reference point for data access.
12. The system of claim 11 , wherein the top-level store further includes a root node pointing to index nodes, each index node pointing to one or more of: one of the leaf nodes and another index node.
A hierarchical data storage system organizes data in a tree-like structure to improve search efficiency and reduce access latency. The system includes a top-level store containing a root node that serves as the entry point for data retrieval. The root node connects to multiple index nodes, which act as intermediate navigational points. Each index node can point to either a leaf node, which contains the actual data, or another index node, enabling multi-level indexing. This nested structure allows for scalable and efficient data organization, particularly in large datasets where direct access to leaf nodes would be impractical. The system optimizes search operations by reducing the number of traversal steps required to locate data, leveraging the hierarchical relationships between nodes. This approach is beneficial in applications requiring fast data retrieval, such as databases, file systems, or distributed storage architectures. The design ensures that data can be accessed through a logical path, minimizing the overhead associated with flat or unstructured storage methods. The hierarchical indexing also supports dynamic updates, allowing nodes to be added, modified, or removed without disrupting the overall structure. This system is particularly useful in environments where data volume and complexity are high, and performance is critical.
13. A non-transitory, computer-readable storage medium encoded with instructions executable by a processor to implement a key-value (KV) store database management system (DBMS) to provide a KV store to an application, the instructions comprising: provide a plurality of low-level stores and a top-level store, wherein: each of the plurality of low-level stores includes key-value pairs and has a respective low-level running transaction accumulating a plurality of changes to the key-value pairs of the low-level store; and the top-level store includes for each of the plurality of low-level stores information leading to the respective low-level store and information indicating which respective low-level running transaction was last committed in the low-level store, and has a top-level running transaction, distinct of a plurality of low-level running transactions of the plurality of low-level stores, accumulating a plurality of changes to the information indicating which respective low-level running transaction was last committed to a low-level store of the plurality of low-level stores; and in response to a plurality of changes made to a plurality of key-value pairs in at least two of the plurality of low-level stores, such that each of the respective at least two low-level transactions of the at least two low-level stores accumulate some of the plurality of changes: commit the respective low-level running transaction of one of the at least two low-level stores, without committing another respective low-level transaction of another of the at least two low-level stores to reduce an amount of writes to secondary memory, comprising: persisting changes to the one low-level store accumulated by the respective low-level running transaction to secondary memory; and updating the top-level store to indicate the respective low-level running transaction was last committed in the one low-level store; and after committing the respective low-level running transaction of the one low-level store, opening a new respective low-level running transaction for the one low-level store.
A key-value (KV) store database management system (DBMS) is designed to efficiently manage distributed transactions across multiple low-level stores while minimizing writes to secondary memory. The system includes a plurality of low-level stores, each containing key-value pairs and maintaining a running transaction that accumulates changes to those pairs. A top-level store tracks each low-level store, storing references to the low-level stores and indicating the last committed transaction for each. The top-level store also maintains its own running transaction, distinct from the low-level transactions, to update the commit status of the low-level stores. When changes are made to key-value pairs across at least two low-level stores, the system commits the transaction of one low-level store while deferring the commit of the other. This selective commit reduces the number of writes to secondary memory by persisting only the changes from the committed low-level store and updating the top-level store to reflect the commit. After committing a low-level transaction, a new transaction is opened for that store to continue accumulating changes. This approach optimizes performance by minimizing disk I/O operations while ensuring consistency across distributed stores. The system is particularly useful in distributed database environments where transactional integrity and write efficiency are critical.
14. The storage medium of claim 13 , wherein the instructions further comprise determine the one low-level store has accumulated a first number of changes, wherein said commit the respective low-level running transaction of the one low-level store occurs in response to said determine the one low-level store has accumulated at least the first number of changes.
This invention relates to data storage systems, specifically optimizing transaction commits in distributed or multi-level storage architectures. The problem addressed is inefficient resource usage and performance bottlenecks caused by frequent or unnecessary transaction commits in storage systems, particularly when handling large volumes of data changes. The system involves a storage medium containing instructions for managing transactions across multiple low-level stores, each handling a subset of data. The instructions include determining when a low-level store has accumulated a threshold number of changes (the first number of changes) and automatically committing the running transaction for that store in response. This ensures transactions are only committed when a significant amount of data has been modified, reducing the overhead of frequent commits while maintaining data consistency. The approach improves system performance by minimizing the computational and I/O overhead associated with transaction commits, particularly in environments where small, incremental changes are common. By batching changes before committing, the system reduces the frequency of disk writes and other resource-intensive operations, leading to more efficient storage management. The threshold-based commit mechanism can be dynamically adjusted based on system conditions or workload characteristics to further optimize performance. This method is particularly useful in distributed databases, file systems, or other storage architectures where transaction management is critical.
15. The storage medium of claim 14 , wherein the instructions further comprise: determine the top-level store has accumulated a second number of changes to the information indicating which respective low-level running transaction was last committed to the low-level store; and in response to said determine the top-level store has accumulated at least the second number of changes: commit the top-level running transaction, and open a new top-level running transaction for the top-level store.
This invention relates to transaction management in database systems, specifically addressing the challenge of efficiently handling top-level and low-level transactions to ensure data consistency and performance. The system involves a storage medium containing instructions for managing transactions in a hierarchical database structure, where a top-level store interacts with one or more low-level stores. The instructions monitor changes in the top-level store to track which low-level transactions have been committed to the low-level store. When the top-level store accumulates a predefined number of changes, the system commits the current top-level transaction and initiates a new top-level transaction. This mechanism ensures that the top-level transaction remains synchronized with the underlying low-level transactions, preventing data inconsistencies and improving system performance by reducing the frequency of transaction commits. The approach optimizes resource usage by balancing the need for data consistency with the overhead of frequent transaction commits, particularly in systems where low-level transactions are processed independently before being reflected in the top-level store. The solution is applicable in distributed databases, multi-tiered storage systems, and other environments where hierarchical transaction management is required.
16. The storage medium of claim 15 , wherein said commit the top-level running transaction comprises: persisting changes to the top-level store accumulated by the top-level running transaction to secondary memory; and updating the top-level store to indicate the last top-level running transaction was last committed transaction in the top-level store.
This invention relates to transactional data management systems, specifically optimizing the persistence and commit process of top-level transactions in a hierarchical transactional storage system. The problem addressed is ensuring efficient and reliable commit operations for top-level transactions while maintaining data consistency across multiple transaction levels. The system involves a hierarchical transactional storage architecture where transactions are organized in a nested structure, with top-level transactions managing lower-level transactions. When a top-level transaction is committed, the system persists all changes made by that transaction to secondary memory, such as a disk or other non-volatile storage. This ensures that the data modifications are durable and survive system failures. Additionally, the system updates the top-level store to reflect that the last committed top-level transaction is now the most recent one, maintaining an accurate record of the transactional state. The commit process involves two key steps: first, writing all accumulated changes from the top-level transaction to secondary memory, and second, updating the top-level store metadata to mark the transaction as committed. This approach ensures that the system can recover to a consistent state after a failure by referencing the last committed transaction. The invention improves transactional reliability and performance by minimizing the overhead of commit operations while maintaining data integrity.
17. The storage medium of claim 13 , wherein the top-level store includes leaf nodes each pointing to one of the plurality of low-level stores.
A data storage system organizes data across multiple storage levels to improve efficiency and performance. The system includes a top-level store and multiple low-level stores. The top-level store contains leaf nodes, each of which directly references one of the low-level stores. This hierarchical structure allows for efficient data retrieval and management by distributing data across different storage tiers while maintaining a clear mapping between the top-level store and the low-level stores. The system may also include a data structure that tracks the locations of data within the low-level stores, enabling quick access and updates. The top-level store may be implemented as a tree or another hierarchical structure, where each leaf node serves as a direct pointer to a specific low-level store. This design helps optimize storage operations by reducing the overhead of searching through multiple layers of data. The system may further include mechanisms for dynamically adjusting the distribution of data between the top-level and low-level stores based on access patterns or other performance metrics. The overall goal is to enhance storage efficiency, reduce latency, and improve scalability in large-scale data storage environments.
18. The storage medium of claim 17 , wherein the top-level store further includes a root node pointing to index nodes, each index node pointing to one or more of: one of the leaf nodes and another index node.
A hierarchical data storage system organizes data in a tree structure to improve search efficiency and reduce access latency. The system includes a top-level store containing a root node that links to multiple index nodes, which in turn point to either leaf nodes or additional index nodes. Leaf nodes store actual data entries, while index nodes act as intermediaries, directing searches through the hierarchy. This multi-level indexing structure allows for faster data retrieval by narrowing down search paths, reducing the number of nodes that need to be traversed. The system is particularly useful in large-scale databases or file systems where quick access to specific data is critical. By separating indexing from data storage, the system optimizes memory usage and speeds up query performance. The hierarchical design also supports scalable data management, accommodating growth without significant structural changes. This approach is beneficial for applications requiring efficient data organization and retrieval, such as database management systems, file storage solutions, or distributed computing environments.
Unknown
January 5, 2021
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.