Database Management System

1. A database management system that enables users to interact with a database comprised of data and metadata, said system comprising A) a plurality of nodes, each node comprising: i) an interface between high level-input and output commands at a user level and input and output commands at the system level that control a sequence of operations for interacting with the data base, wherein, in response to certain system level commands, atom objects generate atoms, each atom containing a specified fragment of data or metadata whereby a set of all instances of atoms collectively define all the metadata and data in the database, ii) a communications control for establishing a communications path with each other node in the system, iii) a method responsive to a system command from the interface for requesting from a selected node copy of an atom that is relevant to the query but is not present in that node, iv) a method responsive to a request for an atom from another node for replicating the requested atom for transfer to the requesting node whereby only atoms required to complete a query need be located in any transaction node at any given time, and v) a method responsive to a change in an atom at that node for replicating that atom for transfer to every other node in the system in which that atom is resident, B) means connected to the plurality of nodes for providing communications paths between all the nodes, and C) means connected to said communications means for providing persistent storage for information in atom whereby the collection of atoms in persistent storage collectively contain all the data and metadata in the database.

2. A database management system as recited in claim 1 wherein certain of the atom objects produce catalog atoms that keep track of all atoms that are resident at that node and the location of each node that has a copy of each of the resident atoms.

3. A database management system as recited in claim 2 wherein one of the catalog atoms is a master catalog atom that is replicated in each node connected to the system, wherein another of the atoms is a single transaction manager atom for the database for tracking the progress of transactions initiated by the interface and wherein the master catalog identifies the location of the transaction manager atom.

4. A database management system as recited in claim 3 wherein the database at the user level comprises named tables with records and fields and operates in accordance with at least one named schema and others of the atom objects produce: i) a table atom for each table that contains a list of fields, ii) a schema atom establishes a correspondence between a schema and the named tables relevant to that schema, and iii) a database atom that establishes a correspondence between the database and the named schema.

5. A database management system as recited in claim 4 wherein the database tables at the user level comprises identified records containing fields and data, wherein other atom objects produce data atoms that contain a predetermined number of data records and a record states atom that manages record versions and wherein said record states atom includes an array that identifies the managed data atoms assigned to that record states atom, an identification of each record version and, for each record version, information about that version and the location of the data atom containing that version.

6. A database management system as recited in claim 5 wherein each data atom additionally includes the address and length of each record within the data atom and the data records.

7. A database management system as recited in claim 4 wherein the database tables at the user level comprises identified blobs, wherein other atom objects produce blob atoms that contain a blob and a blob record states atom that includes a list of all associated blob atoms and the location of the blob in the blob atom.

8. A database management system as recited in claim 7 wherein each blob atom includes the address and length of each blob and the blob.

9. A database management system as recited in claim 1 wherein each node additionally comprises: i) messaging means for generating and receiving predefined messages in response to system level commands, and ii) means for selecting one of the communications paths for transmitting the message.

10. A database management system as recited in claim 9 wherein each node comprises means for asynchronously testing the efficiency of each communications path.

11. A database management system as recited in claim 10 wherein said testing means includes means for generating a ping message and receiving a ping acknowledgement message for determining the ping time for each communications path.

12. A database management system as recited in claim 9 wherein said messaging means includes means for serializing an atom into a message and de-serializing a message into an atom and wherein each message is sent over the selected communications path asynchronously.

13. A database management system as recited in claim 12 wherein messages are sent under a protocol with controls that maintain message sequences.

14. A database management system as recited in claim 1 wherein individual nodes may be active and connected to the system and may be disconnected from the system and one of the atoms is a master catalog atom that identifies each active node in the system and that is replicated in each node connected to the system, the system comprising means for enabling a node to join the system including: i) means for establishing a connection between the joining node and a selected one of the active nodes, ii) means in the joining node for asking the selected node for connection, iii) means in the selected node for updating its master catalog atom to indicate the joining node as an active node, iv) means in the selected node for transferring a message to the joining node including the selected node's identification and for broadcasting a message indicating the availability of the joining node to all other nodes in the system and persistent storage, v) means in said joining node for responding to the message by requesting a copy of the master catalog atom from the selected node, and vi) means in the selected node for transferring its updated master catalog atom to the joining node.

15. A database management system as recited in claim 14 wherein said joining node includes means responsive to the receipt of the updated master catalog atom for requesting copies of additional atoms in response to requests from the interface.

16. A database management system as recited in claim 14 wherein the selected node broadcasts the availability of the updated master catalog to all other nodes and to persistent storage and each other node and persistent storage includes means for acknowledging its receipt of the updated master catalog to the selected node.

17. A database management system as recited in claim 1 wherein an interface in one of the nodes makes a request that requires the production of a new atom wherein certain of the atom objects produce catalog atoms that keep track of atoms that are resident at that node, said one of said nodes additionally comprising: i) means in said catalog atom that will track the new atom responsive to the interface request for causing the catalog atom to create that new atom with no content and to designate the new atom as a chairman for that atom, ii) means in the catalog atom that assigns an object identification to the new atom, iii) means in the catalog atom for establishing a status for the new atom that indicates there is no redundant copy of the new atom and that the new atom is not available to other nodes, iv) means in the catalog atom for broadcasting to all other corresponding catalog atoms in the system the existence of the new atom, v) means for populating the new atom and for making the new atom available on the system.

18. A database management system as recited in claim 17 wherein persistent storage includes means responsive to the storage of the new atom for sending a message to node with the new atom updating its status to indicate the existence of a redundant copy.

19. A database management system as recited in claim 1 wherein an interface in one of the nodes makes a request for an atom that only exists in other nodes and wherein certain of the atom objects produce catalog atoms that keep track of atoms that are resident at that node and other nodes, said system including: i) means in the requesting catalog atom responsive to the interface request that will track the requested atom responsive to the interface request for creating that empty atom, ii) means in the requesting catalog atom for establishing selecting the most responsive node and requesting a copy of the atom from the most responsive node, iii) means in the selected node for sending a copy of the requested atom in an object message to the requesting node and for broadcasting the availability of the atom to other nodes in which the atom, iv) means for assessing the validity of the received object message, for populating the empty atom and for forwarding an object acknowledgment message to the selected node.

20. A database management system as recited in claim 19 wherein said selected node generates a relay list of all nodes containing the requested atom to which the selected node will forward messages from other related to that atom to the requesting node.

21. A database management system as recited in claim 20 wherein each of the other nodes that receive the broadcast object available message includes means for sending an object acknowledgment message to the selected node, said selected node including means for updating the relay list in response to the receipt of each object acknowledgment message from another node to terminate the relay of any subsequent messages to the requesting node.

22. A database management system as recited in claim 1 adapted for transactional processing for assuring consistency of the database wherein the system includes a first node involving an atom to be committed, at least one other node including a copy of that atom and a copy of that atom included in persistent storage, said system including: i) means in the first node for generating a precommit message with a transaction identification for transfer to persistent storage, ii) means associated with persistent storage responsive to the precommit message for committing the transaction and thereafter broadcasting a commit message with the transaction identification to all the nodes, iii) means in the first and at least one other node responsive to the receipt of the commit message for updating the identified transaction.

23. A database management system that enables users to interact with a database comprised of data and metadata, said system comprising A) at least one transactional node that provides users with access to the database, and at least one archival node that maintains an archive of the entire database, each transactional node including a database request engine that provides an interface between high level-input and output query commands at a user level and input and output commands at the system level that control a sequence of operations for interacting with the database, wherein, in response to certain system level commands, atom objects generate atoms, each atom containing a specified fragment of data or metadata whereby a set of all instances of atoms collectively define all the metadata and data in the database, B) a database system network interconnecting all the nodes, C) a communications control in each of the nodes for establishing a communications path with each other node in the system, D) a method in each transactional node responsive to a system command from the database request engine for requesting a copy of an atom that is relevant to the query command but is not present in that node, E) a method in each node responsive to a request for an atom from another node for replicating the requested atom for transfer to the requesting node whereby only atoms required to complete a query command need be located in any transactional node at any given time, and F) a method in each transactional node responsive to a change in an atom at that node for replicating that change to every other node in the system that contains a copy of that atom.

24. A database management system as recited in claim 23 wherein certain of the atom objects produce catalog atoms that keep track of all atoms that are resident at that node and the location of each node that has a copy of each of the resident atoms.

25. A database management system as recited in claim 24 wherein one of the catalog atoms is a master catalog atom that is replicated in each node connected to the system, wherein another the atoms is a single transaction manager atom for the database for tracking the progress of transactions initiated by the interface and wherein the master catalog identifies the location of the transaction manager atom.

26. A database management system as recited in claim 25 wherein the database at the user level comprises named tables with records and fields and operates in accordance with at least one named schema and others of the atom objects produce: i) a table atom for each table that contains a list of fields, ii) a schema atom establishes a correspondence between a schema and the named tables relevant to that schema, and iii) a database atom that establishes a correspondence between the database and the named schema.

27. A database management system as recited in claim 26 wherein the database tables at the user level comprises identified records containing fields and data, wherein other atom objects produce data atoms that contain a predetermined number of data records and a record states atom that manages record versions and wherein said record states atom includes an array that identifies the managed data atoms assigned to that record states atom, an identification of each record version and, for each record version, information about that version and the location of the data atom containing that version.

28. A database management system as recited in claim 27 wherein each data atom additionally includes the address and length of each record within the data atom and the data records.

29. A database management system as recited in claim 26 wherein the database tables at the user level comprises identified blobs, wherein other atom objects produce blob atoms that contain a blob and a blob record states atom that includes a list of all associated blob atoms and the location of the blob in the blob atom.

30. A database management system as recited in claim 29 wherein each blob atom includes the address and length of each blob and the blob.

31. A database management system as recited in claim 23 wherein each node additionally comprises: i) messaging means for generating and receiving predefined messages in response to system level commands, and ii) means for selecting one of the communications paths for transmitting the message.

32. A database management system as recited in claim 31 wherein each node comprises means for asynchronously testing the efficiency each communications path.

33. A database management system as recited in claim 32 wherein said testing means includes means for generating a ping message and receiving a ping acknowledgement message for determining the ping time for each communications path.

34. A database management system as recited in claim 31 wherein said messaging means includes means for serializing an atom into a message and de-serializing the message into an atom and wherein each message is sent over the selected communications path asynchronously.

35. A database management system as recited in claim 34 wherein messages are sent under a protocol with controls that maintain message sequences.

36. A database management system as recited in claim 23 wherein individual nodes may be active and connected to the system and may be disconnected from the system and one of the atoms is a master catalog atom that identifies each active node in the system and that is replicated in each node connected to the system, the system comprising means for enabling a node to join the system including: i) means for establishing a connection between the joining transactional node and a selected one of the active nodes, ii) means in the joining transactional node for asking the selected node for connection to the system, iii) means in the selected node for updating its master catalog atom to indicate the joining transactional node as an active node, iv) means in the selected node for transferring a message to the joining transactional node including the selected node's identification and for broadcasting a message indicating the availability of the joining transactional node to all other nodes in the system, v) means in said joining transactional node for responding to the message by requesting a copy of the master catalog atom from the selected node, and vi) means in the selected node for transferring its updated master catalog atom to the joining transactional node.

37. A database management system as recited in claim 36 wherein said joining transactional node includes means responsive to the receipt of the updated master catalog atom for requesting copies of additional atoms in response to system commands from its database request engine.

38. A database management system as recited in claim 36 wherein the selected node broadcasts the availability of the updated master catalog to all other nodes and each other node includes means for acknowledging its receipt of the updated master catalog to the selected node.

39. A database management system as recited in claim 23 wherein the database request engine in one of the transactional nodes makes a request that requires the production of a new atom and wherein certain of the atom objects produce catalog atoms that keep track of atoms that are resident at that node, said transactional node additionally comprising: i) means in said catalog atom for tracking the new atom responsive to the database request engine request, for causing the catalog atom to create that new atom with no content and for designating the new atom as a chairman for that atom, ii) means in the catalog atom for assigning an object identification to the new atom, iii) means in the catalog atom for establishing a status for the new atom that indicates there is no redundant copy of the new atom and that the new atom is not available to other nodes, iv) means in the catalog atom for broadcasting to all other corresponding catalog atoms in the system the existence of the new atom, v) means for populating the new atom and for making the new atom available to other nodes in the system.

40. A database management system as recited in claim 39 additionally including means in each archival node responsive to the storage of the new atom for sending a message to the new atom updating its status to indicate the existence of a redundant copy.

41. A database management system as recited in claim 23 wherein a database request engine in one transactional nodes requests for an atom that only exists in other nodes and wherein certain of the atom objects produce catalog atoms that keep track of atoms that are resident at that node and other nodes, said system including: i) means responsive to the request in the requesting catalog atom that will track the requested atom for creating that empty atom, ii) means in the requesting catalog atom for establishing selecting the most responsive node and requesting a copy of the atom from the most responsive node, iii) means in the selected node for sending a copy of the requested atom in an object message to the requesting node and for broadcasting the availability of the atom to other nodes in which the atom resides, iv) means for assessing the validity of the received object message, for populating the empty atom and for forwarding an object acknowledgment message to the selected node.

42. A database management system as recited in claim 41 wherein said selected node generates a relay list of all nodes containing the requested atom to which the selected node will forward messages from other related to that atom to the requesting node.

43. A database management system as recited in claim 42 wherein each of the other nodes that receive the broadcast object available message includes means for sending an object acknowledgment message to the selected node, said selected node including means for updating the relay list in response to the receipt of each object acknowledgment message from another node to terminate the relay of any subsequent messages to the requesting node.

44. A database management system as recited in claim 23 adapted for assuring consistency of the database during transactional processing wherein the system includes a first node involving a transaction to be committed and at least one other node including an archival node includes a copy of that atom including, said system including: i) means in the first node for generating a precommit message with a transaction identification for transfer to an archival node, ii) means associated with archival node responsive to the precommit message for committing the transaction and thereafter broadcasting a commit message with the transaction identification to all the nodes, and iii) means in the first and at least one other node responsive to the receipt of the commit message for updating the identified transaction.