Fault Tolerant Systems and Methods Using Shared Memory Configurations

1. A fault tolerant computer system comprising: one or more shared memory complexes, each memory complex comprising a group of M computer-readable memory storage devices; one or more cache coherent switches comprising two or more host ports and one or more downstream device ports, the cache coherent switch in electrical communication with the one or more shared memory complexes; a first management processor in electrical communication with the cache coherent switch, the management processor comprising firmware configured to coordinate and assist with one or more failover functions; an interconnect comprising one or more front-end interconnects and one or more back-end interconnects, wherein the one or more cache coherent switches are in electrical communication with the one or more back-end interconnects, a first compute node comprising a first processor and a first cache, the first compute node in electrical communication with the one or more cache coherent switches and the one or more shared memory complexes, the first compute node configured to run an operating system and a customer application; and a second compute node comprising a second processor and a second cache, the second compute node in electrical communication with the one or more cache coherent switches and the one or more shared memory complexes, wherein the first compute node and the second compute node are in electrical communication with the one or more front-end interconnects, wherein data stored in the one or more shared memory complexes by the first compute node and modified thereby because of operations executing on the first compute node is available for the second compute node to use and modify on a substantially real time basis in the event the second compute node takes over for the first node upon the first compute node undergoing a performance degradation event, wherein the management processor is configured to signal that the second compute node is able to serve as the standby node and take over for the first compute node using the firmware, wherein the second compute node runs the operating system and the customer application after a failover time.

2. The system of claim 1 wherein the one or more cache coherent switches is one or more CXL switches.

3. The system of claim 1 wherein the one or more cache coherent switches is one or more Cache Coherent Interconnect for Accelerator (CCIX) switches.

4. The system of claim 1 wherein each of the M computer-readable memory storage devices is a DDR5 RAM module.

5. The system of claim 1 wherein the second compute node takes over for the active node upon the first node undergoing a performance degradation event during a fail over time that ranges from about 1 millisecond to about 800 milliseconds.

6. The system of claim 2 wherein the one or more CXL switches are in electrical communication with one or more secondary devices selected from the group consisting of a storage device, an I/O device, and an accelerator.

7. The system of claim 2 wherein the first cache comprises state information, wherein the first cache is configured to flush the state information to one or more of the M computer-readable memory storage devices in response to detection of a performance degradation event on the first compute node.

8. The system of claim 2 wherein the one or more shared memory complexes is protected by one or more RAS features such as hardware, software or firmware systems implemented in the one or more cache coherent switches for memory recovery or error correction.

9. The system of claim 6 wherein state information is accessible by the second compute node.

10. The system of claim 2 wherein the first compute node is executing an operating system and one or more customer applications.

11. The system of claim 10 wherein the data associated with the one or more customer applications is accessible from the one or more shared memory complexes by the first compute node and the second compute node.

12. The system of claim 8 wherein the second compute node takes over for the first compute node and continues to execute the one or more customer applications and modify the data associated with the one or more customer applications.

13. The system of claim 2 wherein the first compute node is configured to generate a non-transparent bridging (NTB) window between a local memory of the first compute node and a local memory of the second compute node.

14. The system of claim 1, wherein the management processor is a PCIe fabric switch processing agent that is configured to process PCIe transactions within a domain of one or more cache coherent switches.

15. A method of reducing recovery time in a fault tolerant system, the method comprising: providing an integrated fault tolerant system comprising a shared memory complex, a cache coherent switch in electrical communication with the shared memory complex, a first CPU node, and a second CPU node and a primary management processor; upon occurrence of a performance degradation event, requesting permission for the first CPU node to failover to the second CPU node; signaling by the primary management processor that the second CPU node is able to serve as the standby node and take over for the first CPU node; messaging between first CPU node and second CPU node to transfer or support transfer of state from the failing first CPU node to the standby second CPU node; and flushing one or more caches of the first CPU node to the shared memory.

16. The method of claim 15 further comprising pausing direct memory access traffic from IO devices to the first CPU node.

17. The method of claim 16 wherein the IO devices are selected from the group consisting of I/O, storage, and accelerators.

18. The method of claim 15 further comprising avoiding copying local memory from the failing first CPU node to second CPU node, wherein the second CPU node is transitioning to become an active node.

19. The method of claim 15 wherein the permission is requested of the primary management processor.

20. The method of claim 15 further comprising the second CPU node taking over for the first CPU node upon the first CPU node undergoing a performance degradation event during a failover time that ranges from about 1 millisecond to about 800 milliseconds.

21. The method of claim 15, wherein the primary management processor is a PCIe fabric switch processing agent that is configured to process PCIe transactions within a domain of one or more cache coherent switches.