Unified System Services Layer for a Distributed Processing System

1. A distributed processing system having a plurality of computing nodes coupled by a network, wherein each of the computing nodes comprises: one or more programmable processors for executing instructions; one or more software processes executing on the one or more programmable processors within an execution environment provided by an operating system kernel for a general-purpose operating system; and a plug-in software module for the general-purpose operating system executing on the one or more programmable processors and within the kernel space of the general-purpose operating system, wherein the plug-in software module of the general-purpose operating system extends the general-purpose operating system to provide a distributed memory operating system having a single process execution space that spans the computing nodes of the distributed processing system, wherein the plug-in software module comprises a process virtualization module for the operating system kernel, wherein the process virtualization module defines a global process identifier (PID) space across the computing nodes, assigns a first unique range of process identifiers within the global PID space to a first computing node, and assigns a second unique range of process identifiers within the global PID surface to a second computing node, wherein the first computing node assigns a PID from the first unique range of process identifiers to a first software process, and the second computing node assigns a PID from the second unique range of process identifiers to a second software process such that any of the one or more software processes can determine which computing node launched the first software process and the second software process, wherein the first computing node launches the first software process on a third computing node, and wherein the second computing node launches the second software process on a fourth computing node.

2. The distributed processing system of claim 1 , wherein the plug-in software module of the general-purpose operating system provides an inter-process communication mechanism between each computing node.

3. The distributed processing system of claim 1 , wherein the general-purpose operating system is a lightweight operating system that is optimized based on the functionality of the respective computing node.

4. The distributed processing system of claim 1 , wherein the general-purpose operating system is the Linux operating system.

5. The distributed processing system of claim 1 , wherein the plug-in software module communicates with a software hook installed within the operating system and is invoked by the operating system when the operating system receives requests from the software processes.

6. The distributed processing system of claim 1 , wherein the first computing node automatically assigns a process identifier selected from the first unique range of process identifiers to the first software process.

7. The distributed processing system of claim 1 , wherein each of the nodes comprises a software module for communicating between software processes executing on the nodes, wherein the software module receives a messaging request directed to a target one of the software processes, and identifies one of the nodes that launched the target software process based on a process identifier specified within the request, and wherein the software module issues the messaging request to the identified node for forwarding to the software module.

8. The distributed processing system of claim 1 , wherein the process virtualization module communicates the range of the process identifiers to the operating system kernel for use when the operating system kernel launches one or more of the software processes on any of the computing nodes.

9. The distributed processing system of claim 1 , further comprising: at least one network interconnect software module; and a transport application programming interface (API) that provides a mechanism for sending and receiving communications between the process virtualization module and other computing nodes in the distributed processing system.

10. The distributed processing system of claim 1 , further comprising a two-way application programming interface (API) for communications to flow between the operating system kernel and the process virtualization module.

11. The distributed processing system of claim 1 , wherein the plug-in software module further comprises a distributed I/O virtualization module that provides a distributed file system for the distributed processing system.

12. A method comprising: installing plug-in software modules on nodes of a distributed processing system, wherein each of the plug-in software modules is installed within a general-purpose operating system that provides an execution environment for software processes executing on the respective node; executing the plug-in software modules within kernel spaces of the nodes to extend the general-purpose operating system on each of the nodes to provide a distributed memory operating system having a single process execution space that spans the nodes of the distributed processing system, wherein executing the plug-in software modules within kernel spaces of the nodes comprises: defining a global process identifier (PID) space across the nodes; assigning, with one or more process virtualization modules, a unique range of process identifiers within the global PID space to each of the nodes in the distributed processing system, wherein assigning, with the one or more process virtualization modules, the unique range of process identifiers comprises assigning a first unique range of process identifiers within the global PID space to a first node, and assigning a second unique range of process identifiers within the global PID surface to a second node; and assigning to each of one or more software processes a process identifier selected from the range assigned to a node that launched the respective software process such that any of the one or more software processes can determine which node launched the respective software process, wherein assigning to each of the one or more software processes the process identifier selected from the range assigned to the node that launched the respective software process comprises: assigning, with the first node, a PID from the first unique range of process identifiers to a first software process, launching, with the first node, the first software process on a third node, assigning, with the second node, a PID from the second unique range of process identifiers to a second software process, and launching, with the second node, the second software process on a fourth node.

13. The method of claim 12 , further comprising providing an inter-process communication mechanism between each node with the plug-in software modules.

14. The method of claim 13 , further comprising receiving with the plug-in software module communications from a software hook installed within the operating system; and invoking the inter-process communication mechanism in response to the communication to forward the communications between the nodes.

15. The method of claim 12 , wherein the general-purpose operating system for each node is a lightweight operating system that is optimized based on the functionality of the respective node.

16. The method of claim 12 , wherein the general-purpose operating system is the Linux operating system.

17. The method of claim 12 , further comprising: launching one of the software processes with a first one of the nodes on a second one of the nodes; selecting a process identifier from the unique range of process identifiers assigned to the first one of the nodes; and assigning the processor identifier to the launched process with the second one of the nodes.

18. The method of claim 12 , further comprising installing the plug-in software modules to extend the operating system to provide a distributed file system for the distributed processing system.

19. The distributed processing system of claim 1 , wherein the third computing node and the fourth computing node are the same computing node.

20. The method of claim 1 , wherein the third node and the fourth node are the same node.