System and Methods for Managing Time-Based Events to Support Rewinding of a Database

This is the first application filed for the present invention.

The present disclosure pertains to the field of database management, and in particular to systems and methods for organizing or managing time-based events to support (e.g., fast iterative) rewinding of a database.

Backtracking in a distributed database system refers to the process of reverting a database to a previous state, often essential when errors, erroneous or problematic user actions, or data corruption occur. Backtracking may also be useful when data at a past time is required for special analysis or usage (e.g. testing or training data). However, several challenges can arise due to backtracking. Firstly, dealing with irrelevant data can be complex, as not all historical data may be relevant to the analysis or recovery effort, requiring careful filtering mechanisms. Conflicting data due to time paradoxes can also occur, where multiple versions of data conflict temporally, leading to ambiguity in determining the correct state.

Moreover, supporting backtracking within a defined window requires efficient management of data, ensuring that relevant historical information is retained while minimizing storage overhead. In some cases, data may need to be held for extended periods to support backtracking, which can significantly impact storage size and resource utilization. Balancing the need for comprehensive historical data with storage efficiency and performance becomes a key consideration in designing and operating distributed database systems with robust backtracking capabilities.

Therefore, there is a need systems and methods for organizing or managing time-based events to support (e.g., fast iterative) rewinding of a database that obviates or mitigates one or more limitations of the prior art.

This background information is provided to reveal information believed by the applicant to be of possible relevance to the present invention. No admission is necessarily intended, nor should be construed, that any of the preceding information constitutes prior art against the present invention.

Systems and methods may be provided for organizing or managing time-based events to support (e.g., fast iterative) rewinding of a database. Systems and methods described herein may provide for identifying and maintaining time-based events or events (e.g., one or more of: log records, anchor snapshots and recovery checkpoints) necessary for multiple reconstructions of a database state as of different points in the past.

According to an aspect of the present disclosure, there is provided a method performed by a distributed database system. The method includes performing at least one backtracking of a database, where each backtracking reverts the database to a respective target timepoint. The method includes determining a desired state of the database to reconstruct after the at least one backtracking. The method includes determining, based on the at least one backtracking, one or more of a set of relevant log sequence number (LSN) ranges and a set of relevant storage offset ranges to reconstruct the database to the desired state. The method includes reconstructing the database to the desired sate based on the one or more of the set of relevant LSN ranges and the set of relevant storage offset ranges.

In some embodiments, the method further includes periodically generating clock sampling records. Each clock sampling record indicates a time point to which the database can be reconstructed. Each clock sampling record includes an LSN corresponding to the time point, and a storage offset of one or more events. Each event can be: a log record, an anchor snapshot, or a recovery checkpoint. The method further includes generating backtracking information records based on the at least one backtracking.

In some embodiments, generating backtracking information records includes generating a set of backtracking information records based on a time order. Each backtracking information record of the set of backtracking information records indicates a respective LSN range and further indicates one or more of: a start LSN of the respective LSN range, an end LSN of the respective LSN range, a start storage offset of an event corresponding to the start LSN of the respective LSN range, and an end storage offset of the event corresponding to the end LSN of the respective LSN range. The event may be a log record, an anchor snapshot, or a recovery checkpoint. The set of backtracking information records includes an initial backtracking information record. The respective start LSN is based on when a backtracking feature of the distributed database system is activated. The respective end LSN is based on when a first backtracking of the at least one backtracking is performed. The set of backtracking information records includes at least one backtracking information record corresponding to the at least one backtracking. Each backtracking information record further indicates a target LSN of the corresponding target timepoint to which the database backtracked. For each backtracking information record, the respective start LSN is based on the end LSN of a previous backtracking information record as determined based on the time order. The respective end LSN of each backtracking information record is based on when a next backtracking is performed.

In some embodiments, the above-mentioned determining, based on the at least one backtracking, the one or more of the set of relevant LSN ranges and the set of relevant storage offset ranges to reconstruct the database to the desired state includes the following: determining, from the set of backtracking information records, a first base backtracking information record of a set of base backtracking information records based on the respective LSN range that includes a desired LSN. The desired LSN corresponds to the desired state. The backtracking information records includes the set of base backtracking information records. The above-mentioned determining may further include determining a relevant LSN range of the set of relevant LSN ranges based on a start LSN of the first base backtracking information record and the desired LSN. The above-mentioned determining may further include, for each event of one or more events associated with the first base backtracking information record, determining a relevant storage offset range of the set of relevant storage offset ranges based on a start storage offset of that event of the first base backtracking information record and a storage offset value of that event corresponding to the desired LSN. When the desired LSN is a target for backtracking, one of the clock sampling records corresponds to the desired LSN, and the storage offset value of the event corresponding to the desired LSN is a storage offset value of the event in the clock sampling record corresponding to the desired LSN. Otherwise, to facilitate recovery the storage offset value of the event corresponding to the desired LSN is an indefinite storage offset value. Each event of the set of events is a log record, an anchor snapshot, or a recovery checkpoint.

In some embodiments, the method further includes, beginning from the first base backtracking information record, for each base backtracking information record of the set of base backtracking information records that indicates a target LSN, performing the following action: Determining, from the backtracking information records, a subsequent base backtracking information record of the set of base backtracking information records based on a corresponding LSN range that includes the target LSN of the base backtracking information record. The method may further include determining another relevant LSN range of the set of relevant LSN ranges based on a start LSN of the subsequent base backtracking information record and the target LSN of the base backtracking information record. The method may further include, for each event of one or more events associated with the subsequent base backtracking information record, determining another storage offset range of the set of relevant storage offset ranges based on a start storage offset of said event of the subsequent base backtracking information record and a storage offset value of said event in a clock sampling record corresponding to the target LSN of the base backtracking information record.

In some embodiments, reconstructing the database to the desired state based on the one or more of the set of relevant LSN ranges and the set of relevant storage offset ranges includes determining one or more events for reconstructing the database based on the one or more of the set of relevant LSN ranges and the set of relevant storage offset ranges. The database is then reconstructed based on the one or more events determined for reconstructing the database.

In some embodiments, the method further includes generating a set of timelines based on a time order. Each timeline indicates a set of LSN ranges and a set of storage offset ranges for an event based on the set of the LSN ranges. The event may be a log record, an anchor snapshot, or a recovery checkpoint. The set of timelines includes an initial timeline indicating a respective last LSN range having a start LSN based on when a backtracking feature of the distributed database system is activated. The respective last LSN range of the initial timeline has an end LSN of when a first backtracking of the at least one backtracking is performed. The set of timelines includes at least one timeline corresponding to the at least one backtracking. Each timeline of this at least one timeline corresponds to a respective backtracking of the at least one backtracking and is associated with a respective target LSN corresponding to the target timepoint of the respective backtracking. A latest timeline of the at least one timeline corresponds to a current timeline associated with a latest backtracking of the at least one backtracking. The current timeline indicates a respective last LSN range with an end LSN being based on when a next backtracking is performed after the at least one backtracking. Each timeline of the at least one timeline indicates a respective set of LSN ranges. The set of LSN ranges includes a respective last LSN range with a start LSN based on an end LSN of a previous timeline. The previous timeline is determined according to the time order. The last LSN range ends based on an LSN of when a next backtracking is performed. The set of LSN ranges includes one or more pertinent LSN ranges determined from a set of LSN ranges of a corresponding base timeline. The corresponding base timeline is determined from the set of timelines and includes in its last LSN range the respective target LSN of said each timeline of the at least one timeline. The one or more pertinent LSN ranges includes all LSN ranges up to a last pertinent LSN range. The last pertinent LSN range ends at the respective target LSN of each timeline of the at least one timeline.

In some embodiments, the above-mentioned determining, based on the at least one backtracking, of the one or more of the set of relevant LSN ranges and the set of relevant storage offset ranges to reconstruct the database to the desired state, includes the following: A target timeline from the set of timelines is determined based on a respective last LSN range that includes a desired LSN. The desired LSN corresponds to the desired state. The set of relevant LSN ranges are determined based on all LSN ranges of the target timeline up to a last relevant LSN range. The last relevant LSN range ends at the desired LSN. The set of relevant storage offset ranges is determined based on a set of storage offset ranges corresponding to the determined set of relevant LSN ranges.

According to an aspect of the present disclosure, there is provided another method performed by a distributed database system. The method includes generating clock sampling records indicating a backtracking window via one or more time points to which a database can be backtracked. The method includes performing at least one backtracking of the database. Each backtracking backtracks the database to a target timepoint. The method includes, for each backtracking of the at least one backtracking: pausing the generation of the clock sampling records; backtracking the database to the corresponding target timepoint; and resuming the generation of the clock sampling records following the backtracking.

In some embodiments, the method further includes generating a set of backtracking information records based on a time order. Each backtracking information record of the set of backtracking information records indicates a respective LSN range and further indicates one or more of: a start LSN of the respective LSN range, an end LSN of the respective LSN range, a start storage offset of an event corresponding to the start LSN of the respective LSN range, and an end storage offset of the event corresponding to the end LSN of the respective LSN range. The event may be a log record, an anchor snapshot, or a recovery checkpoint. The set of backtracking information records includes an initial backtracking information record, where the respective start LSN is based on when a backtracking feature of the distributed database system is activated. The respective end LSN is based on when a first backtracking of the at least one backtracking is performed. The set of backtracking information records includes at least one backtracking information record corresponding to the at least one backtracking. Each backtracking information record further indicates a target LSN of the corresponding target timepoint to which the database is backtracked. For each such backtracking information record, the respective start LSN is based on the end LSN of a previous backtracking information record as determined based on the time order. The respective end LSN of each backtracking information record is based on when a next backtracking is performed.

In some embodiments, for each backtracking of the at least one backtracking, after pausing and before resuming the generation of the clock sampling records, the method further includes requesting each slice of the database to generate at least one anchor snapshot; and receiving an indication from said each slice of the database that the anchor snapshot has been generated.

In some embodiments, the method further includes generating a set of timelines based on a time order. Each timeline indicates a set of LSN ranges and a set of storage offset ranges for an event based on the set of the LSN ranges. The event may be a log record, an anchor snapshot, or a recovery checkpoint. The set of timelines includes an initial timeline indicating a respective last LSN range having a start LSN based on when a backtracking feature of the distributed database system is activated. This last LSN range of the initial timeline has an end LSN of when a first backtracking of the at least one backtracking is performed. The set of timelines includes at least one timeline corresponding to the at least one backtracking. Each timeline of the at least one timeline corresponds to a respective backtracking and is associated with a respective target LSN corresponding to the target timepoint of the respective backtracking. A latest timeline of the at least one timeline corresponds to a current timeline associated with a latest backtracking of the at least one backtracking. The current timeline indicates a respective last LSN range with an end LSN that is based on when a next backtracking is performed after the at least one backtracking. Each timeline of the at least one timeline indicates a respective set of LSN ranges including: a respective last LSN range with a start LSN based on an end LSN of a previous timeline. The previous timeline is determined according to the time order, and the last LSN range ends based on an LSN of when a next backtracking is performed. The respective set of LSN ranges further includes one or more pertinent LSN ranges determined from a set of LSN ranges of a corresponding base timeline. The base timeline is determined from the set of timelines and includes in its last LSN range the respective target LSN of the timeline associated with the set of LSN ranges (e.g., said each timeline of the at least one timeline). The one or more pertinent LSN ranges includes all LSN ranges up to a last pertinent LSN range. The last pertinent LSN range ends at the respective target LSN of the timeline associated with the set of LSN ranges.

In some embodiments, for each backtracking of the at least one backtracking, after pausing and before resuming the generation of the clock sampling records, the method further comprises: requesting each slice of the database to generate an anchor snapshot; and receiving an indication from said each slice of the database that the anchor snapshot has been generated.

Aspects and embodiments further provide for an apparatus, system, or both, which includes database and/or computing hardware and which is configured to perform one or more of the methods as described above and elsewhere herein.

According to another aspect, an apparatus may be provided. The apparatus includes modules or electronics configured to perform one or more of the methods and systems described herein.

According to one aspect, an apparatus may be provided, where the apparatus includes: a memory, configured to store a program; a processor, configured to execute the program stored in the memory, and when the program stored in the memory is executed, the processor is configured to perform one or more of the methods and systems described herein.

According to another aspect, a computer readable medium may be provided, where the computer readable medium stores program code executed by a device and the program code is used to perform one or more of the methods and systems described herein.

According to one aspect, a chip may be provided, where the chip includes a processor and a data interface, and the processor reads, by using the data interface, an instruction stored in a memory, to perform one or more of the methods and systems described herein. Aspects may further include the memory.

Other aspects of the disclosure provide for apparatus, and systems configured to implement the methods according to the first aspect disclosed herein. For example, wireless stations and access points can be configured with machine readable memory containing instructions, which when executed by the processors of these devices, configures the device to perform one or more of the methods and systems described herein.

Embodiments have been described above in conjunctions with aspects of the present invention upon which they can be implemented. Those skilled in the art will appreciate that embodiments may be implemented in conjunction with the aspect with which they are described, but may also be implemented with other embodiments of that aspect. When embodiments are mutually exclusive, or are otherwise incompatible with each other, it will be apparent to those skilled in the art. Some embodiments may be described in relation to one aspect, but may also be applicable to other aspects, as will be apparent to those of skill in the art.

It will be noted that throughout the appended drawings, like features are identified by like reference numerals.

Systems and methods may be provided for organizing or managing time-based events to support (e.g., fast iterative) rewinding of a database. Systems and methods described herein may provide for identifying and maintaining time-based events or events (e.g., one or more of: log records, anchor snapshots and recovery checkpoints) necessary for multiple reconstructions of a database state as of different points in the past. Database rewinding is also described in International Patent Application No. PCT/CN2023/139312, the contents of which are incorporated herein by reference.

According to an aspect, a method is provided for identifying relevant events for backtracking a database. The method includes performing at least one backtracking of a database, where each backtracking reverts the database to a respective target timepoint. The method further includes determining a desired state of the database to reconstruct after the at least one backtracking. The method further includes determining, based on the at least one backtracking, one or more of a set of relevant LSN ranges and a set of relevant storage offset ranges to reconstruct the database to the desired state. The method further includes reconstructing the database to the desired sate based on the one or more of the set of relevant LSN ranges and the set of relevant storage offset ranges.

According to another aspect, a method is provided for maintaining a minimum set of time-based events for supporting backtracking. The method includes generating clock sampling records indicating a backtracking window via one or more time points to which a database can be backtracked. The method further includes performing at least one backtracking of the database. Each backtracking backtracks the database to a target timepoint. The method further includes, for each backtracking of the at least one backtracking, pausing the generation of the clock sampling records. The method further includes, for each backtracking of the at least one backtracking, backtracking the database to the corresponding target timepoint. The method further includes, for each backtracking of the at least one backtracking, resuming the generation of the clock sampling records following the backtracking.

A data page or a page may refer to a unit of data storage within a database management system. Data of a database may be organized based on smaller-sized items, e.g., pages. A page may have different sizes, such as 4K, 8K, and 16K.

A log record or a change log record may refer to a log entry that documents a change made to data (or a data page) within the database. All changes to a database are done by generating change log records for each individual item (page). An example of a change log record may be a redo log for My Structured Query Language (MySQL), which is a relational database management system.

An LSN may refer to a unique and incremental identifier assigned to each log record. log records may be ordered using LSNs. LSN order may correspond to the order of log record creation. When applying one or more log records to a page to derive a later version of that page, LSN order must be strictly obeyed, otherwise, data may be incorrect.

Log apply may refer to the process of applying changes recorded in a change log record. log records can be applied on one or more pages. For example, a change log record can indicate to write a 10 bytes content on page #20 at storage offset (or offset). After a log record is applied on a page, a new version of the page is generated. Each page may have a corresponding page LSN, which is updated according to the LSN of the last change log record applied.

Database state may refer to status of a database at a specific point in time. Database state may represent how the data is organized and what values are stored in the database tables and other structures. The database state can change as data is inserted, updated, or deleted, and it can be influenced by various database operations and transactions. Database state, at a particular time, may be identified by an LSN, which includes all data page changes up to this LSN. This LSN corresponds to the LSN of the last change log record (corresponding to the last database change) at the particular time.

illustrates a cloud-native distributed database architecture, according to an embodiment. A database having the architecturemay be referred to as a cloud-native distributed database. A cloud-native distributed database may be designed for cloud infrastructure with separate compute layerand storage layer. An example of such a database is Huawei Cloud GaussDB (for My Structured Query Language (MySQL)).

In an embodiment, the compute layercomprises one or more of: a Structured Query Language (SQL) master and one or more SQL replicas responsible for accepting and processing user queries. In an embodiment, one or more transactions including inserting, updating and deleting data pages, of user queries, are handled by the SQL master. In some embodiments, the SQL master generates log records and ships log records to the storage layerover a network (e.g., a low-latency remote direct memory access (RDMA) storage network).

In some embodiments, the storage layercomprises one or more of: a log storeand a page storeas illustrated. In some embodiments, the log storeis used to persist log records generated by the SQL master, with LSN order, which can be read out and re-distributed to the one or more page storesat any time. In some embodiments, the one or more page stores store the pages, and pages are partitioned and organized into one or more data slices (called slices) in the one or more page stores. In some embodiments, the one or more page stores serve page read requests coming from the SQL master or replicas, with applying log records generated and distributed by the SQL master. In some embodiments, a database includes tens to thousands of slices located in several to tens of page stores.

Backtracking in databases involves rewinding the database to any point in the past, navigating back and forth iteratively. In some embodiments, this process reconstructs the desired database state within the database itself, without the need for backup files or restoration. As a result, backtracking may be performed in a relatively short time, often taking only minutes. Moreover, this backtracking can be performed iteratively, which may allow for precise time-point adjustments down to the second level.

In some instances of a database crash, a database instance may not restart. The only way to bring the system back may be to restore from backup images, but since the instance may have a very large dataset, restoration may take hours to complete. As part of restoration, a new instance with changed virtual internet protocol (IP) address is created, which further requires the configuration of the original client application to be modified or adjusted to ensure that the client application functions correctly. As a result, the client business services may be interrupted for quite a long time.

Similarly, for any database product, various mistakes or problems can cause failure. For example, data can be lost or corrupted due to any software or hardware error or failure. Further, erroneous user actions can occur occasionally. For instance, a user may mistakenly perform a destructive action, such as a DELETE without proper WHERE clause. In addition, malicious actions may be taken to break into, alter or damage the database on purpose.

Therefore, customers may require a functionality that allows them to rewind their database cluster back and forth to a specific point in time. This capability helps them identify when a specific data change occurred, rectify significant errors, or restore the database to a state prior to corruption. Backtracking may offer a fast and unique method to retrieve data from any past time, particularly useful for test environments, data history tracking, and historical data analysis purposes.

illustrates a backtracking architecture for a distributed database, according to an aspect. In an embodiment, each slice of the one or more page storesregularly (e.g., every 20 minutes) generates an anchor snapshot (or snapshot) of all the pages in the slice, at certain timepoint (as denoted by an LSN value). In an embodiment, with loading an anchor snapshot, the data of the pages of the slice can be reconstructed to the state as represented by the LSN.

In an embodiment, backtracking a distributed database involves rewinding the distributed database to a timepoint in the past represented by a certain LSN value, which may be called the target LSN. This target LSN corresponds to the timepoint in the past to which the database is rewound. For example, a distributed database may be backtracked to a target LSN.

In an embodiment, backtracking includes, for each sliceof the one or more page stores, from the end of the anchor snapshot list, finding or determiningan anchor snapshot with LSN that is less than and closest to (or equal to) the target LSN (e.g., the anchor snapshot of LSNin).

In an embodiment, backtracking further includes, for said each sliceof the one or more page stores, the one or more page stores creatinga new slice, loading the anchor snapshot found (e.g., the anchor snapshot of LSN), and replacing the original slice. In the example, after loading the anchor snapshot, the new sliceis at a state indicated by LSN.

As may be appreciated, the new slicestill needs the log records based on an LSN range indicated from the LSN of the anchor snapshot (LSN) to the target LSN, to update the pages to the target LSN. Accordingly, the new slicestill need log records from LSNto LSN.

Thus, in some embodiments, backtracking further includes, for said each new slice, obtainingthe relevant log records (referring to one or more log records needed to update the new slice from the LSN of the anchor snapshot to the target LSN). In some embodiments, after creating a corresponding new slice for said each slice of the one or more page stores, and loading a corresponding anchor snapshot for said each new slice, the compute layer SQL masterreads the log records that the slices needed, from the log store, and then re-distributesthem to all the slices.

In some embodiments, backtracking further includes applying the obtained relevant log records to said each new slice. In some embodiments, each new slice applies the log records received, to the data pages, to reconstruct the database state corresponding to the target LSN.