Systems and methods are disclosed herein for performing a failover operation between a first distributed database cluster and a second distributed database cluster. In one aspect, the method includes blocking a plurality of software processes from making any new access requests to the first distributed database cluster and completing a current access request to the first distributed database cluster that was initiated by one of the plurality of software processes before the failover operation was initiated, waiting for data to replicate from the first distributed database cluster to the second distributed database cluster, wherein replicating includes any changes to the data from the current access request, and allowing the plurality of software processes to make the new access requests to the second distributed database cluster once the replicating has completed.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A method of performing a failover operation between a first distributed database cluster and a second distributed database cluster, the method comprising: blocking a plurality of software processes from making any new access requests to the first distributed database cluster; completing a current access request to the first distributed database cluster that was initiated by one of the plurality of software processes before the failover operation was initiated; waiting for data to replicate from the first distributed database cluster to the second distributed database cluster, wherein replicating includes any changes to the data from the current access request; and allowing the plurality of software processes to make the new access requests to the second distributed database cluster once the replicating has completed; wherein the software processes each have a database abstraction layer that performs the blocking of the plurality of software processes from making any new access requests to the first distributed database cluster, the completing of a current access request to the first distributed database cluster, and the allowing of the plurality of software processes to make the new access requests to the second distributed database cluster.
2. The method of claim 1 , wherein the new access requests and the current access request comprise requests to write data to the first distributed database cluster.
3. The method of claim 1 , wherein blocking the plurality of software processes from making new access requests to the first distributed database cluster occurs concurrently with the completing the current access request to the first distributed database cluster.
4. The method of claim 1 , wherein any software processes that do not use their database abstraction layer are aborted when the failover operation is initiated.
5. The method of claim 1 , wherein the completing is subject to a first time period and the method further comprises aborting the current access request if it does not complete within the first time period.
6. The method of claim 1 , wherein the waiting is subject to a third time period and if the replicating does not complete within the third time period, an additional action is taken.
7. The method of claim 6 , wherein the additional action comprises notifying a system administrator that the failover operation has not completed.
8. A method of performing a failover operation between a first distributed database cluster and a second distributed database cluster, the method comprising: blocking a plurality of software processes from making any new access requests to the first distributed database cluster; completing a current access request to the first distributed database cluster that was initiated by one of the plurality of software processes before the failover operation was initiated, the completing subject to a first time period; aborting the current access request if it does not complete within the first time period; waiting for data to replicate from the first distributed database cluster to the second distributed database cluster, wherein replicating includes any changes to the data from the current access request; and allowing the plurality of software processes to make the new access requests to the second distributed database cluster once the replicating has completed; wherein the aborting is subject to a second time period and the method proceeds to the waiting after the second time period elapses.
9. A method of performing a failover operation between a first distributed database cluster and a second distributed database cluster, the method comprising: blocking a plurality of software processes from making any new access requests to the first distributed database cluster; completing a current access request to the first distributed database cluster that was initiated by one of the plurality of software processes before the failover operation was initiated; waiting for data to replicate from the first distributed database cluster to the second distributed database cluster, wherein replicating includes any changes to the data from the current access request; allowing the plurality of software processes to make the new access requests to the second distributed database cluster once the replicating has completed; and selecting a leader software process from the plurality of software processes, wherein the leader software process edits a failover state file that coordinates the failover operation among the plurality of software processes.
10. The method of claim 9 , wherein the failover state file comprises the following variables: a failover sequence number that uniquely identifies the failover operation; a current state of the failover operation; a time elapsed since the beginning of the current state of the failover operation; an address of the first distributed database cluster; and an address of the second distributed database cluster.
11. The method of claim 9 , wherein the failover state file is stored on a lock server.
12. The method of claim 9 , wherein the plurality of software processes reads the failover state file during the failover operation to ensure that each software process is in the same portion of the failover operation as the other software processes.
13. The method of claim 12 , wherein if a software process is not participating in the same portion of the failover operation as the other software processes, the software process will abort.
14. The method of claim 9 , wherein the leader software process is chosen from the plurality of software processes through an election scheme.
15. The method of claim 14 , wherein if the leader software process experiences a failure during the failover operation, the plurality software processes selects a replacement leader software process from among the plurality of software processes through an election scheme.
16. A non-transitory computer readable medium storing computer executable instructions, which, when executed by a processor, cause the processor to carry out a method for controlling a plurality of software processes through a failover operation between a first distributed database cluster and a second distributed database cluster, wherein the instructions comprise: instructions to block a plurality of software processes from making any new access requests to the first distributed database cluster; instructions to complete a current access request to the first distributed database cluster that was initiated by one of the plurality of software processes before the failover operation was initiated; instructions to wait for data to replicate from the first distributed database cluster to the second distributed database cluster, wherein replicating includes any changes to the data from the current access request; and instructions to allow the plurality of software processes to make the new access requests to the second distributed database cluster once the replicating has completed; wherein the plurality of software processes each have a database abstraction layer, and wherein the database abstraction layer comprises the computer executable instructions.
17. The non-transitory computer readable medium of claim 16 , wherein the instructions to complete is subject to a first time period and the instructions to complete further comprises instructions to abort the current access request if it does not complete within the first time period.
18. The non-transitory computer readable medium of claim 16 , wherein the instructions to wait are subject to a third time period and if the replicating does not complete within the third time period, a system administrator is notified that the failover operation has not completed.
19. A non-transitory computer readable medium storing computer executable instructions, which, when executed by a processor, cause the processor to carry out a method for controlling a plurality of software processes through a failover operation between a first distributed database cluster and a second distributed database cluster, wherein the instructions comprise: instructions to block a plurality of software processes from making any new access requests to the first distributed database cluster; instructions to complete a current access request to the first distributed database cluster that was initiated by one of the plurality of software processes before the failover operation was initiated, the instructions to complete subject to a first time period; instructions to abort the current access request if it does not complete within the first time period; instructions to wait for data to replicate from the first distributed database cluster to the second distributed database cluster, wherein replicating includes any changes to the data from the current access request; and instructions to allow the plurality of software processes to make the new access requests to the second distributed database cluster once the replicating has completed; wherein the instructions to abort are subject to a second time period, and at the expiration of the second time period the instructions proceed to the instructions to wait.
20. A replicated distributed database system, the system comprising: a plurality of distributed database clusters; a plurality of computing devices running a plurality of software processes, wherein each software process in the plurality of software processes has a database abstraction layer that is configured to instruct the software process to: block any new access requests to a first distributed database cluster within the plurality of distributed database clusters when a failover operation occurs; complete a current access request to the first distributed database cluster that was initiated by the software process before the failover operation occurs; and allow the new access requests to be directed to a second distributed database cluster within the plurality of distributed database clusters after the failover operation is complete; and a plurality of lock servers configured to: store a failover state file; and execute an election scheme to select a leader software process from the plurality of software processes; and monitor the replication of data from the first distributed database cluster to the second distributed database cluster.
21. The system of claim 20 , wherein the leader software processes edit the failover state file.
22. The system of claim 20 , wherein the plurality of distributed databases is arranged in a multi-master scheme.
23. The system of claim 20 , wherein the failover state file comprises the following variables: a failover sequence number that uniquely identifies the failover operation; a current state of the failover operation; a time elapsed since the beginning of the current state of the failover operation; an address of the first distributed database cluster; and an address of the second distributed database cluster.
24. The system of claim 20 , wherein the plurality of software processes reads the failover state file during the failover operation to ensure that each software process is in the same portion of the failover operation as the other software processes.
25. The system of claim 24 , wherein if a software process is not participating in the same portion of the failover operation as the other software processes, the software process will abort.
26. The system of claim 20 , wherein the new access requests and the current access request comprises requests to write data to the first distributed database cluster.
27. The system of claim 20 , wherein each software process in the plurality of software processes generates a plurality of failover variables.
28. The system of claim 20 , wherein if the leader software process experiences a failure during the failover operation, the plurality of software processes selects a replacement leader software process from among the plurality of software processes through the election scheme.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
January 20, 2012
October 7, 2014
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.