Data Replication Based on Capacity Optimization

1. A method for replicating data based on capacity optimization, said method comprising: a local node receiving the data in an extent configured for a system, the extent being associated with a key for indexing, wherein the system comprises at least one domain coupled to a communication network, wherein each domain of said at least one domain comprises at least one node and a respective distributed hash table (DHT) for said each domain, wherein each node of said at least one node comprises a respective storage array storing respective primary data and respective replicas of selected data stored in the respective primary data of said at least one node; determining, by a processor of a computer in which the local node operates, that the received extent does not exist in primary data of the local node; querying said at least one domain in the system for a duplicate of the extent by use of the respective DHT for said each domain such that only a respective owner node is queried for the duplicate of the extent within said each domain, wherein the respective owner node of said each domain owns a key range in which the key associated with the extent falls in; determining a replication strategy based on factors and a result from said querying, wherein said replication strategy is selected from the group consisting of establishing replica relationship among existing duplicates, creating a new remote duplicate, and creating a new local duplicate; and recording the extent into said primary data of the local node by updating a local hash table and extent metadata of the local node.

2. The method of claim 1 , wherein said at least one node further comprises a respective failure responder module that is configured to respond to a partial node failure in steps comprising: determining that the partial node failure has occurred in the local node; and responsive to said determining, recreating an extent corrupted by the partial node failure in the local node from at least one existing duplicate of the extent, wherein said at least one existing duplicate is stored in any node in said at least one domain.

3. The method of claim 2 , wherein said respective failure responder module further comprising: determining that the partial node failure has occurred in a remote node; responsive to said determining, identifying at least one affected extent in the local node; and initiating a new extent processing cycle for said at least one affected extent, wherein said new extent processing cycle comprises said querying, said determining the replication strategy, and said recording the extent.

4. The method of claim 1 , said querying said at least one domain comprising: querying a local owner node of the key range for the duplicate of the extent, wherein the local owner node is an owner node in a local domain of said at least one domain, and wherein the local node is a member to the local domain; responsive to said querying the local owner node, receiving from the local owner node a first negative result indicating that the duplicate of the extent does not exist in the local domain; querying each remote owner node of the key range for the duplicate of the extent, wherein said each remote owner node is a respective owner node in a respective remote domain of said at least one domain; responsive to said querying said each remote owner node, receiving from said each remote owner node a respective negative result indicating that the duplicate of the extent does not exist in the respective remote domain; and responsive to said receiving from the local owner node the first negative result and said receiving from said each remote owner node the respective negative result, marking the extent as unique in said at least one domain.

5. The method of claim 1 , said querying said at least one domain comprising: querying a local owner node of the key range for the duplicate of the extent, and subsequently querying each remote owner node of the key range for the duplicate of the extent, wherein the local owner node is an owner node in a local domain of said at least one domain, wherein the local node is a member to the local domain, and wherein said each remote owner node is a respective owner node in a respective remote domain of said at least one domain; responsive to said querying the local owner node and subsequently querying said each remote owner node, receiving from queried owner node at least one positive result indicating that at least one duplicate of the extent exists in a respective domain corresponding to the queried owner node; and recording respective location information of said at least one duplicate of the extent in the respective domain.

6. A computer program product comprising: a computer readable storage medium having a computer readable program code embodied therein, said computer readable program code containing instructions that perform replicating data based on capacity optimization, said replicating comprising: a local node receiving the data in an extent configured for a system, the extent being associated with a key for indexing, wherein the system comprises at least one domain coupled to a communication network, wherein each domain of said at least one domain comprises at least one node and a respective distributed hash table (DHT) for said each domain, wherein each node of said at least one node comprises a respective storage array storing respective primary data and respective replicas of selected data stored in the respective primary data of said at least one node; determining that the received extent does not exist in primary data of the local node; querying said at least one domain in the system for a duplicate of the extent by use of the respective DHT for said each domain such that only a respective owner node is queried for the duplicate of the extent within said each domain, wherein the respective owner node of said each domain owns a key range in which the key associated with the extent falls in; determining a replication strategy based on factors and a result from said querying, wherein said replication strategy is selected from the group consisting of establishing replica relationship among existing duplicates, creating a new remote duplicate, and creating a new local duplicate; and recording the extent into said primary data of the local node by updating a local hash table and extent metadata of the local node.

7. The computer program product of claim 6 , wherein said at least one node further comprises a respective failure responder module that is configured to respond to a partial node failure in steps comprising: determining that the partial node failure has occurred in the local node; and responsive to said determining, recreating an extent corrupted by the partial node failure in the local node from at least one existing duplicate of the extent, wherein said at least one existing duplicate is stored in any node in said at least one domain.

8. The computer program product of claim 7 , wherein said respective failure responder module further comprising: determining that the partial node failure has occurred in a remote node; responsive to said determining, identifying at least one affected extent in the local node; and initiating a new extent processing cycle for said at least one affected extent, wherein said new extent processing cycle comprises said querying, said determining the replication strategy, and said recording the extent.

9. The computer program product of claim 6 , said querying said at least one domain comprising: querying a local owner node of the key range for the duplicate of the extent, wherein the local owner node is an owner node in a local domain of said at least one domain, and wherein the local node is a member to the local domain; responsive to said querying the local owner node, receiving from the local owner node a first negative result indicating that the duplicate of the extent does not exist in the local domain; querying each remote owner node of the key range for the duplicate of the extent, wherein said each remote owner node is a respective owner node in a respective remote domain of said at least one domain; responsive to said querying said each remote owner node, receiving from said each remote owner node a respective negative result indicating that the duplicate of the extent does not exist in the respective remote domain; and responsive to said receiving from the local owner node the first negative result and said receiving from said each remote owner node the respective negative result, marking the extent as unique in said at least one domain.

10. The computer program product of claim 6 , said querying said at least one domain comprising: querying a local owner node of the key range for the duplicate of the extent, and subsequently querying each remote owner node of the key range for the duplicate of the extent, wherein the local owner node is an owner node in a local domain of said at least one domain, wherein the local node is a member to the local domain, and wherein said each remote owner node is a respective owner node in a respective remote domain of said at least one domain; responsive to said querying the local owner node and subsequently querying said each remote owner node, receiving from queried owner node at least one positive result indicating that at least one duplicate of the extent exists in a respective domain corresponding to the queried owner node; and recording respective location information of said at least one duplicate of the extent in the respective domain.

11. A computer system comprising a processor and a computer readable memory unit coupled to the processor, said computer readable memory unit containing instructions that when run by the processor implement a method for replicating data based on capacity optimization, said method comprising: a local node receiving the data in an extent configured for a system, the extent being associated with a key for indexing, wherein the system comprises at least one domain coupled to a communication network, wherein each domain of said at least one domain comprises at least one node and a respective distributed hash table (DHT) for said each domain, wherein each node of said at least one node comprises a respective storage array storing respective primary data and respective replicas of selected data stored in the respective primary data of said at least one node; determining that the received extent does not exist in primary data of the local node; querying said at least one domain in the system for a duplicate of the extent by use of the respective DHT for said each domain such that only a respective owner node is queried for the duplicate of the extent within said each domain, wherein the respective owner node of said each domain owns a key range in which the key associated with the extent falls in; determining a replication strategy based on factors and a result from said querying, wherein said replication strategy is selected from the group consisting of establishing replica relationship among existing duplicates, creating a new remote duplicate, and creating a new local duplicate; and recording the extent into said primary data of the local node by updating a local hash table and extent metadata of the local node.

12. The computer system of claim 11 , wherein said at least one node further comprises a respective failure responder module that is configured to respond to a partial node failure in steps comprising: determining that the partial node failure has occurred in the local node; and responsive to said determining, recreating an extent corrupted by the partial node failure in the local node from at least one existing duplicate of the extent, wherein said at least one existing duplicate is stored in any node in said at least one domain.

13. The computer system of claim 12 , wherein said respective failure responder module further comprising: determining that the partial node failure has occurred in a remote node; responsive to said determining, identifying at least one affected extent in the local node; and initiating a new extent processing cycle for said at least one affected extent, wherein said new extent processing cycle comprises said querying, said determining the replication strategy, and said recording the extent.

14. The computer system of claim 11 , said querying said at least one domain comprising: querying a local owner node of the key range for the duplicate of the extent, wherein the local owner node is an owner node in a local domain of said at least one domain, and wherein the local node is a member to the local domain; responsive to said querying the local owner node, receiving from the local owner node a first negative result indicating that the duplicate of the extent does not exist in the local domain; querying each remote owner node of the key range for the duplicate of the extent, wherein said each remote owner node is a respective owner node in a respective remote domain of said at least one domain; responsive to said querying said each remote owner node, receiving from said each remote owner node a respective negative result indicating that the duplicate of the extent does not exist in the respective remote domain; and responsive to said receiving from the local owner node the first negative result and said receiving from said each remote owner node the respective negative result, marking the extent as unique in said at least one domain.

15. The computer system of claim 11 , said querying said at least one domain comprising: querying a local owner node of the key range for the duplicate of the extent, and subsequently querying each remote owner node of the key range for the duplicate of the extent, wherein the local owner node is an owner node in a local domain of said at least one domain, wherein the local node is a member to the local domain, and wherein said each remote owner node is a respective owner node in a respective remote domain of said at least one domain; responsive to said querying the local owner node and subsequently querying said each remote owner node, receiving from queried owner node at least one positive result indicating that at least one duplicate of the extent exists in a respective domain corresponding to the queried owner node; and recording respective location information of said at least one duplicate of the extent in the respective domain.

16. A machine-executable process for supporting computer infrastructure, said process embodied in a computer-readable storage medium comprising: providing at least one support service for at least one of creating, integrating, hosting, maintaining, and deploying computer-readable code in a computing system, wherein the code in combination with the computing system is capable of performing a method for replicating data based on capacity optimization, said method comprising: a local node receiving the data in an extent configured for a system, the extent being associated with a key for indexing, wherein the system comprises at least one domain coupled to a communication network, wherein each domain of said at least one domain comprises at least one node and a respective distributed hash table (DHT) for said each domain, wherein each node of said at least one node comprises a respective storage array storing respective primary data and respective replicas of selected data stored in the respective primary data of said at least one node; determining that the received extent does not exist in primary data of the local node; querying said at least one domain in the system for a duplicate of the extent by use of the respective DHT for said each domain such that only a respective owner node is queried for the duplicate of the extent within said each domain, wherein the respective owner node of said each domain owns a key range in which the key associated with the extent falls in; determining a replication strategy based on factors and a result from said querying, wherein said replication strategy is selected from the group consisting of establishing replica relationship among existing duplicates, creating a new remote duplicate, and creating a new local duplicate; and recording the extent into said primary data of the local node by updating a local hash table and extent metadata of the local node.

17. The process of claim 16 , wherein said at least one node further comprises a respective failure responder module that is configured to respond to a partial node failure in steps comprising: determining that the partial node failure has occurred in the local node; and responsive to said determining, recreating an extent corrupted by the partial node failure in the local node from at least one existing duplicate of the extent, wherein said at least one existing duplicate is stored in any node in said at least one domain.

18. The process of claim 17 , wherein said respective failure responder module further comprising: determining that the partial node failure has occurred in a remote node; responsive to said determining, identifying at least one affected extent in the local node; and initiating a new extent processing cycle for said at least one affected extent, wherein said new extent processing cycle comprises said querying, said determining the replication strategy, and said recording the extent.

19. The process of claim 16 , said querying said at least one domain comprising: querying a local owner node of the key range for the duplicate of the extent, wherein the local owner node is an owner node in a local domain of said at least one domain, and wherein the local node is a member to the local domain; responsive to said querying the local owner node, receiving from the local owner node a first negative result indicating that the duplicate of the extent does not exist in the local domain; querying each remote owner node of the key range for the duplicate of the extent, wherein said each remote owner node is a respective owner node in a respective remote domain of said at least one domain; responsive to said querying said each remote owner node, receiving from said each remote owner node a respective negative result indicating that the duplicate of the extent does not exist in the respective remote domain; and responsive to said receiving from the local owner node the first negative result and said receiving from said each remote owner node the respective negative result, marking the extent as unique in said at least one domain.

20. The process of claim 16 , said querying said at least one domain comprising: querying a local owner node of the key range for the duplicate of the extent, and subsequently querying each remote owner node of the key range for the duplicate of the extent, wherein the local owner node is an owner node in a local domain of said at least one domain, wherein the local node is a member to the local domain, and wherein said each remote owner node is a respective owner node in a respective remote domain of said at least one domain; responsive to said querying the local owner node and subsequently querying said each remote owner node, receiving from queried owner node at least one positive result indicating that at least one duplicate of the extent exists in a respective domain corresponding to the queried owner node; and recording respective location information of said at least one duplicate of the extent in the respective domain.