Distributed Storage of Recoverable Data

1. A method for replacing anode storing distributed data, the method comprising: receiving, by a first storage node, a first set of shares generated from a data file wherein each share in the first set includes a linear combination of pieces of the data file along with a set of coefficients used to generate the linear combination; receiving, by the first storage node, an indication of a new storage node replacing a failed node, the failed node including a second set of shares generated from the data file; generating, by the first storage node, a first replacement share responsive to the indication, wherein the first replacement share is generated by: multiplying each share in the first set and the set of coefficients by a random scaling value; and combining the multiplied first set of shares and the multiplied set of coefficients; and transmitting, by the first storage node, the generated first replacement share to the new storage node; wherein the first replacement share and at least one other replacement share form the second set of shares at the new storage node, the other replacement share generated by a second storage node.

2. The method of claim 1 , wherein an overall number of replacement shares generated to form the second set of shares at the new storage node equals a reliability parameter, wherein the reliability parameter is based on a total number of storage nodes required for the data file to remain recoverable.

3. The method of claim 1 , further comprising: verifying the integrity of the generated first replacement share.

4. The method of claim 1 , wherein the shares of the first set are generated using erasure coding based on pieces of the data file.

5. The method of claim 1 , wherein the first replacement share and at least one other replacement share are used to reconstruct the data file.

6. The method of claim 1 , wherein the first replacement share and at least one other replacement share are used to generate subsequent replacement shares for a subsequent new node.

7. The method of claim 1 , wherein the indication is received from a tracker that monitors the status of storage nodes and tracks locations of shares.

8. The method of claim 1 , further comprising: sending hash check information to the new storage node for verifying the first replacement share.

9. The method of claim 1 , wherein the integrity of the first replacement node is verified by the new storage node using hash check information provided by an additional storage node.

10. A computer system for replacing a failed node storing distributed data, the system comprising: a computer processor; and a non-transitory computer-readable storage medium storing executable computer program instructions executed by the computer processor, the computer program instructions configured to: receive a first set of shares generated from a data file wherein each share in the first set includes a linear combination of pieces of the data file along with a set of coefficients used to generate the linear combination; receive an indication of a new storage node replacing a failed node, the failed node including a second set of shares generated from the data file; generate a first replacement share responsive to the indication, wherein the first replacement share is generated by: multiplying each share in the first set and the set of coefficients by a random scaling value; and combining the multiplied first set of shares and the multiplied set of coefficients; and transmit the generated first replacement share to the new storage node; wherein the first replacement share and at least one other replacement share form the second set of shares at the new storage node, the other replacement share generated by a second storage node.

11. The system of claim 10 , wherein an overall number of replacement shares generated to form the second set of shares at the new storage node equals a reliability parameter, wherein the reliability parameter is based on a total number of storage nodes required for the data file to remain recoverable.

12. The system of claim 10 , further comprising: sending hash check information to the new storage node for verifying the first replacement share.

13. The system of claim 10 , wherein the first set of shares are generated using erasure coding based on pieces of the data file.

14. The system of claim 10 , wherein the first replacement share and at least one other replacement share are used to reconstruct the data file.

15. The system of claim 10 , wherein the first replacement share and at least one other replacement share are used to generate subsequent replacement shares for a subsequent new node.

16. The system of claim 10 , wherein the indication is received from a tracker that monitors the status of storage nodes and tracks locations of shares.

17. A computer program product having a non-transitory computer-readable storage medium having executable computer program instructions recorded thereon for replacing a failed node storing distributed data, the computer program instructions further configured for: receiving, by a first storage node, a first set of shares generated from a data file wherein each share in the first set includes a linear combination of pieces of the data file along with a set of coefficients used to generate the linear combination; receiving, by the first storage node, an indication of a new storage node replacing a failed node, the failed node including a second set of shares generated from the data file; generating, by the first storage node, a first replacement share responsive to the indication, wherein the first replacement share is generated by: multiplying each share in the first set and the set of coefficients by a random scaling value; and combining the multiplied first set of shares and the multiplied set of coefficients; and transmitting, by the first storage node, the first replacement share to the new storage node; wherein the first replacement share and at least one other replacement share form the second set of shares at the new storage node, the other replacement share generated by a second storage node.

18. The computer program product of claim 17 , wherein the computer program instructions are further configured for: sending hash check information to the new storage node for verifying the first replacement share.

19. The computer program product of claim 17 , wherein the first replacement share and at least one other replacement share are used to reconstruct the data file.

20. The computer program product of claim 17 , wherein an overall number of replacement shares generated to form the second set of shares at the new storage node equals a reliability parameter, wherein the reliability parameter is based on a total number of storage nodes required for the data file to remain recoverable.

21. The computer program product of claim 17 , wherein the shares of the first set are generated using erasure coding based on pieces of the data file.

22. The computer program product of claim 17 , wherein the first replacement share and at least one other replacement share are used to generate subsequent replacement shares for a subsequent new node.

23. The computer program product of claim 17 , wherein the indication is received from a tracker that monitors the status of storage nodes and tracks locations of shares.