A method performed by two or more devices of a group of devices in a distributed data replication system may include receiving a group of chunks having a same unique temporary identifier, the group of chunks comprising an object to be uploaded; creating an entry for the object in a replicated index, the entry being keyed by the unique temporary identifier, and the replicated index being replicated at each of the two or more devices; and determining, by an initiating device of the two or more devices, that a union of the group of chunks contains all data of the object. The method may also include calculating a content-based identifier to the object; creating another entry for the object in the replicated index, the other entry being keyed by the content-based identifier; and updating the replicated index to point from the unique temporary identifier to the content-based identifier.
Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A method comprising: receiving, at a device, a plurality of chunks of an object to be uploaded, each chunk, of the plurality of chunks, including a first identifier associated with the object; creating, by the device, a first entry, for the object, in a memory associated with the device, the first entry being associated with the first identifier; reconstructing, at the device, the object from the plurality of chunks; preventing, by the device, content of the object from being modified based on reconstructing the object; determining, by the device, a second identifier associated with the object based on the content; and creating, by the device and in the memory, a second entry for the object, the second entry being associated with the second identifier.
A system uploads an object by splitting it into chunks. Each chunk includes a temporary identifier. The system creates an initial index entry using this temporary identifier. The object is reconstructed from the chunks. After reconstruction, the object's content cannot be changed. A content-based identifier (e.g., a hash) is then calculated based on the object’s content. A second index entry is created using this content-based identifier. Thus, the object can be located using both identifiers.
2. The method of claim 1 , where the device is included in a data replication system, the method further comprising: replicating a portion of the memory, storing the first entry, to one or more other devices included in the data replication system.
A data replication system uploads an object by splitting it into chunks. Each chunk includes a temporary identifier. The system creates an initial index entry using this temporary identifier. This index entry is replicated across multiple devices in the system. The object is reconstructed from the chunks. After reconstruction, the object's content cannot be changed. A content-based identifier (e.g., a hash) is then calculated based on the object’s content. A second index entry is created using this content-based identifier. The temporary identifier and the content-based identifier allow finding the object.
3. The method of claim 1 , further comprising: storing the plurality of chunks in a first portion of the memory that is different than a second portion of the memory; and storing the first identifier and the second identifier in the second portion of the memory.
During object upload, the system stores the chunks of data in one area of memory, and the associated identifiers (both the temporary and content-based identifiers) in a separate area of memory. The system uploads an object by splitting it into chunks. Each chunk includes a temporary identifier. The system creates an initial index entry using this temporary identifier. The object is reconstructed from the chunks. After reconstruction, the object's content cannot be changed. A content-based identifier (e.g., a hash) is then calculated based on the object’s content. A second index entry is created using this content-based identifier.
4. The method of claim 1 , further comprising: receiving a request from a client, the plurality of chunks being uploaded to the device via the client; and determining that the plurality of chunks have been uploaded based on receiving the request, the object being reconstructed based on determining that the plurality of chunks have been uploaded.
A client uploads chunks of an object to the system. The system knows all chunks have been uploaded when the client sends a finalization request. The system uploads an object by splitting it into chunks. Each chunk includes a temporary identifier. The system creates an initial index entry using this temporary identifier. The object is reconstructed from the chunks. After reconstruction, the object's content cannot be changed. A content-based identifier (e.g., a hash) is then calculated based on the object’s content. A second index entry is created using this content-based identifier.
5. The method of claim 1 , where reconstructing the object includes: receiving, via a data replication system, information identifying a location in the memory that each chunk, of the plurality of chunks, is stored; determining, based on the received information identifying the location in the memory that each chunk, of the plurality of chunks, is stored, that each chunk, of the plurality of chunks, has been uploaded; and reconstructing the object based on determining that each chunk, of the plurality of chunks, has been uploaded.
To reconstruct an uploaded object, the data replication system keeps track of the memory locations where each chunk is stored. The system uploads an object by splitting it into chunks. Each chunk includes a temporary identifier. The system creates an initial index entry using this temporary identifier. The object is reconstructed from the chunks. After reconstruction, the object's content cannot be changed. A content-based identifier (e.g., a hash) is then calculated based on the object’s content. A second index entry is created using this content-based identifier.
6. The method of claim 1 , where determining the second identifier includes: computing a hash based on the content; and determining the second identifier based on the computed hash.
The system calculates the content-based identifier of an object by computing a hash of the object's data. The system uploads an object by splitting it into chunks. Each chunk includes a temporary identifier. The system creates an initial index entry using this temporary identifier. The object is reconstructed from the chunks. After reconstruction, the object's content cannot be changed. A content-based identifier (e.g., a hash) is then calculated based on the object’s content. A second index entry is created using this content-based identifier.
7. The method of claim 1 , where creating the second entry includes: updating an index to allow the object to be accessed by one or more devices of a data replication system based on the first identifier and the second identifier.
The system creates a second index entry (keyed by the content-based identifier) and updates the index to allow the object to be accessed using either the temporary identifier or the content-based identifier across the data replication system. The system uploads an object by splitting it into chunks. Each chunk includes a temporary identifier. The system creates an initial index entry using this temporary identifier. The object is reconstructed from the chunks. After reconstruction, the object's content cannot be changed. A content-based identifier (e.g., a hash) is then calculated based on the object’s content. A second index entry is created using this content-based identifier.
8. A non-transitory computer-readable medium storing instructions, the instructions comprising: one or more instructions which, when executed by at least one processor, cause the at least one processor to: receive a plurality of chunks of an object to be uploaded, each chunk, of the plurality of chunks, including a first identifier, create, in a memory, a first entry for the object, the first entry being associated with the first identifier, reconstruct the object from the plurality of chunks based on the first identifier, prevent content of the object from being modified based on reconstructing the object, determine a second, different identifier for the object based on the content, and create, in the memory, a second entry for the object, the second entry being associated with the second identifier.
A computer-readable medium stores instructions for uploading an object. The process involves receiving chunks of the object, where each chunk has a temporary identifier. An initial entry is created in memory, associated with this temporary identifier. The object is reconstructed using the chunks. Once reconstructed, the object's content cannot be modified. A content-based identifier is determined based on the object's content. Finally, a second entry is created in memory, associated with the content-based identifier.
9. The non-transitory computer-readable medium of claim 8 , where the instructions further comprise: one or more instructions to replicate the first entry to one or more devices included in a data replication system.
A computer-readable medium contains instructions for uploading an object that includes replicating the initial index entry (associated with the temporary identifier) across multiple devices in the data replication system. The process involves receiving chunks of the object, where each chunk has a temporary identifier. An initial entry is created in memory, associated with this temporary identifier. The object is reconstructed using the chunks. Once reconstructed, the object's content cannot be modified. A content-based identifier is determined based on the object's content. Finally, a second entry is created in memory, associated with the content-based identifier.
10. The non-transitory computer-readable medium of claim 8 , where the instructions further comprise: one or more instructions to store the plurality of chunks in a first portion of the memory that is different than a second portion of the memory, and one or more instructions to store the first identifier and the second identifier in the second portion of the memory.
A computer-readable medium stores instructions that manage memory allocation during object uploading by storing the data chunks in one memory region and the temporary and content-based identifiers in a separate memory region. The process involves receiving chunks of the object, where each chunk has a temporary identifier. An initial entry is created in memory, associated with this temporary identifier. The object is reconstructed using the chunks. Once reconstructed, the object's content cannot be modified. A content-based identifier is determined based on the object's content. Finally, a second entry is created in memory, associated with the content-based identifier.
11. The non-transitory computer-readable medium of claim 8 , where the instructions further comprise: one or more instructions to receive a request from a client, the plurality of chunks being uploaded to a device associated with the at least one processor via the client, and one or more instructions to determine that the plurality of chunks have been uploaded based on receiving the request, the object being reconstructed based on determining that the plurality of chunks have been uploaded.
A computer-readable medium stores instructions for managing the upload process by receiving a signal (e.g., a request) from the client when all chunks have been uploaded. The system then reconstructs the object. The process involves receiving chunks of the object, where each chunk has a temporary identifier. An initial entry is created in memory, associated with this temporary identifier. The object is reconstructed using the chunks. Once reconstructed, the object's content cannot be modified. A content-based identifier is determined based on the object's content. Finally, a second entry is created in memory, associated with the content-based identifier.
12. The non-transitory computer-readable medium of claim 8 , where the one or more instructions to reconstruct the object include: one or more instructions to receive, via a data replication system, information identifying one or more locations in the memory storing the plurality of chunks, one or more instructions to determine, based on the received information, that the plurality of chunks have been uploaded, and one or more instructions to reconstruct the object based on the plurality of chunks having been uploaded.
A computer-readable medium contains instructions to reconstruct the object. The instructions include receiving information via the data replication system about the memory location of each chunk, determining that all chunks are uploaded based on the location information, and reconstructing the object. The process involves receiving chunks of the object, where each chunk has a temporary identifier. An initial entry is created in memory, associated with this temporary identifier. The object is reconstructed using the chunks. Once reconstructed, the object's content cannot be modified. A content-based identifier is determined based on the object's content. Finally, a second entry is created in memory, associated with the content-based identifier.
13. The non-transitory computer-readable medium of claim 8 , where the one or more instructions to determine the second identifier include: one or more instructions to determine the second identifier based on a hash of data included in the content.
A computer-readable medium storing instructions calculates the content-based identifier by hashing the object's data. The process involves receiving chunks of the object, where each chunk has a temporary identifier. An initial entry is created in memory, associated with this temporary identifier. The object is reconstructed using the chunks. Once reconstructed, the object's content cannot be modified. A content-based identifier is determined based on the object's content. Finally, a second entry is created in memory, associated with the content-based identifier.
14. The non-transitory computer-readable medium of claim 8 , where the one or more instructions to create the second entry include: one or more instructions to modify an index, where modifying the index allows the object to be accessed based on the first identifier and the second identifier.
A computer-readable medium storing instructions modifies the index to allow access to the object using either the temporary identifier or the content-based identifier. The process involves receiving chunks of the object, where each chunk has a temporary identifier. An initial entry is created in memory, associated with this temporary identifier. The object is reconstructed using the chunks. Once reconstructed, the object's content cannot be modified. A content-based identifier is determined based on the object's content. Finally, a second entry is created in memory, associated with the content-based identifier.
15. A system comprising: a device to: receive a plurality of chunks of an object to be uploaded, each chunk, of the plurality of chunks, including a first identifier, create, in a memory, a first entry for the object, the first entry being associated with the first identifier, reconstruct the object from the plurality of chunks based on the first identifier, determine a second, different identifier for the object based on content of the object, and create, in the memory, a second entry for the object, the second entry being associated with the second identifier.
A system for uploading an object splits it into chunks, with each chunk including a temporary identifier. The system creates an initial index entry using this temporary identifier. The object is reconstructed from the chunks. After reconstruction, the object's content cannot be changed. A content-based identifier is calculated based on the object's content. A second index entry is created using this content-based identifier.
16. The system of claim 15 , where the device is further to: replicate the first entry to one or more other devices included in a data replication system.
A data replication system uploads an object by splitting it into chunks, each with a temporary identifier. The system creates an initial index entry with the temporary identifier. This initial entry is replicated across other devices in the replication system. The object is reconstructed from the chunks. After reconstruction, the object's content cannot be changed. A content-based identifier is calculated. A second index entry uses this identifier.
17. The system of claim 15 , where the device is further to: determine that the plurality of chunks have been uploaded based on receiving a request to finalize the object, the object being reconstructed based on determining that the plurality of chunks have been uploaded.
In the object upload system, all data chunks are uploaded, as indicated by a finalization request. Reconstruction only occurs after receiving this finalization request. A system for uploading an object splits it into chunks, with each chunk including a temporary identifier. The system creates an initial index entry using this temporary identifier. The object is reconstructed from the chunks. After reconstruction, the object's content cannot be changed. A content-based identifier is calculated based on the object's content. A second index entry is created using this content-based identifier.
18. The system of claim 15 , where, when reconstructing the object, the device is further to: receive, via an underlying replication layer of a data replication system, information identifying one or more locations in the memory storing the plurality of chunks; and reconstruct the object based on the information identifying the one or more locations in the memory storing the plurality of chunks.
During the reconstruction phase, the underlying data replication layer provides information about the memory locations of the chunks. The reconstruction process utilizes these locations. A system for uploading an object splits it into chunks, with each chunk including a temporary identifier. The system creates an initial index entry using this temporary identifier. The object is reconstructed from the chunks. After reconstruction, the object's content cannot be changed. A content-based identifier is calculated based on the object's content. A second index entry is created using this content-based identifier.
19. The system of claim 15 , where, when determining the second identifier, the device is further to: compute a hash based on data included in the content, and determine the second identifier based on the computed hash.
The system calculates the content-based identifier by computing a hash of the object's data, then using that hash as the content-based identifier. A system for uploading an object splits it into chunks, with each chunk including a temporary identifier. The system creates an initial index entry using this temporary identifier. The object is reconstructed from the chunks. After reconstruction, the object's content cannot be changed. A content-based identifier is calculated based on the object's content. A second index entry is created using this content-based identifier.
20. The system of claim 15 , where, when creating the second entry, the device is further to: modify an index to allow the object to be accessed by a second device, based on the first identifier and the second identifier.
The system modifies the index to allow other devices in the system to access the object using either its temporary or content-based identifier. A system for uploading an object splits it into chunks, with each chunk including a temporary identifier. The system creates an initial index entry using this temporary identifier. The object is reconstructed from the chunks. After reconstruction, the object's content cannot be changed. A content-based identifier is calculated based on the object's content. A second index entry is created using this content-based identifier.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
April 11, 2012
June 18, 2013
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.