Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A method, comprising: generating a write signature command, the write signature command having parameters used in execution comprising an offset, a number of blocks of data and a signature, the write signature command configured to: write the signature to the offset in a storage array without the data, the signature being a hash value generated using a hash function on the data; and enable the storage array to write the data with the same signature to a volume if the data is available at the storage array; executing the write signature command to synchronize data between a second site and a first site, executing the write signature command comprising: reading a first hash value having a first offset at the first site associated with a block of data at the first site; sending the first hash value to the second site; using the write signature command with the first hash value received from the first site to determine if there is a block of data stored at the second site associated with the first hash value, wherein the write signature command uses the first hash value to copy a block of data, from a location at the second site indicated in a hash table associated with the first hash value, to a second offset at the second site if the first hash value is found in the hash table; determining if the write signature command was successful by determining there is the block of data stored at the second site associated with the first hash value; and if the write command is not successful: reading the block of data from the first site associated with the first hash value; and writing the block of data from the first site associated with the first hash value to the second offset at the second site, the second offset being equal to the first offset.
A method for efficiently synchronizing data between two storage sites involves generating a "write signature command." This command includes parameters like the data's offset, block count, and a signature (hash value). The command first writes the signature to the specified offset at the destination site without transferring the actual data. The destination site then checks if it already has the data associated with that signature in its storage array. If a match is found (based on a hash table lookup), the destination site copies the existing data to the specified offset, avoiding data transfer. If no match is found, the data is read from the source site and written to the destination offset.
2. The method of claim 1 , further comprising synchronizing data between a first site and a second site without sending the data from the first site to the second site if the second site already contains data from first site located in any location in the storage array.
This method builds upon the data synchronization process and avoids transferring data between two storage sites if the destination site already holds a copy of that data anywhere in its storage array. The destination site utilizes the signature to check if a matching block exists, copying it to the correct location instead of requesting the data from the source.
3. The method of claim 1 , further comprising: receiving a write at the storage array; reading a signature for the write; invalidating a previous hash table entry for the signature; determining a hash value for the write; and updating the hash table.
When a write operation occurs at the storage array, the method reads the existing signature associated with that write location. It then invalidates any previous hash table entry related to that signature. A new hash value is calculated for the new data being written, and the hash table is updated to reflect this new data and its location.
4. The method of claim 3 , further comprising: receiving a write signature command containing a hash value, searching for hash value in the hash table; copying the data from the location indicated by the hash table to a location indicated by the write signature command if the hash value is found in the hash table; and returning a mismatch message if the data is not found in the hash table.
When the storage array receives a "write signature command" containing a specific hash value, the method searches its hash table for that value. If the hash value is found, the data pointed to by the hash table entry is copied to the location specified in the "write signature command". If the hash value isn't found in the hash table, a "mismatch" message is returned, indicating that the data needs to be transferred from the source.
5. The method of claim 1 , further comprising: intercepting a write command; marking a write in a bitmap as dirty; reading the data from offset indicated as dirty in the bit map; calculating a signature for the data; and adding the signature, for a hash table indicating the signature points to a write offset.
The method intercepts a standard write command. It then marks the corresponding data block in a bitmap as "dirty," indicating it needs to be processed. The actual data from the offset indicated as "dirty" is then read. A signature (hash value) is calculated for this data, and the signature is added to a hash table, linking the signature to the write offset where the data is stored.
6. The method of claim 5 , further comprising: intercepting a write signature command; searching for the hash value in the hash table; copying the data from the location indicated by the hash table to a location indicated by the write signature command if the hash value is found and the hash value of the data indicated by the hash table is identical to the hash value; and returning a mismatch message if the data is not found or is not identical.
Upon intercepting a "write signature command", the method searches for the provided hash value in the hash table. If the hash value is found, and the hash value of the data indicated by the hash table is identical to the received hash value, the data pointed to by the hash table entry is copied to the location specified in the "write signature command". If the hash value is not found in the hash table, or if the hash values don't match, a "mismatch" message is returned.
7. The method of claim 1 , further comprising using the write signature command to save bandwidth when transferring deduplicated data over a network.
The "write signature command" is used to reduce network bandwidth usage when transferring deduplicated data across a network. By sending only signatures and leveraging existing data at the destination, the amount of data transferred is minimized.
8. The method of claim 7 wherein using the write signature command to save bandwidth when transferring deduplicated data over a network comprises: generating, at the first site, the first signature associated with data to be sent to the second site; sending the first signature to the second site; attempting to write the first signature to a location at a storage LUN at the second site; reading the data from the location at the storage LUN at the second site if the attempt to write the first signature was successful; sending a status message indicating success if the attempting to write the first signature was successful; sending a status message indicating failure if the attempting to write the first signature was unsuccessful; sending the data from the first site to the second site and writing the data to an offset at the second site if the attempting to write the first signature was unsuccessful.
This invention relates to optimizing bandwidth usage during deduplicated data transfers between storage sites. The problem addressed is the inefficiency of transferring large volumes of data over networks, particularly when redundant data already exists at the destination. The solution involves a write signature command that checks for existing data at the receiving site before transmitting new data, reducing unnecessary transfers. The method operates between a first site and a second site, each with storage logical unit numbers (LUNs). At the first site, a signature is generated for data intended for the second site. This signature is sent to the second site, where an attempt is made to write it to a specific location in a storage LUN. If the write is successful, it indicates the data already exists at the second site, so the first site reads the data from the LUN location and sends a success status message. If the write fails, the data is not present, so the first site transmits the data to the second site, which then writes it to a designated offset. This approach minimizes bandwidth by avoiding redundant data transfers when duplicates are detected. The method ensures efficient storage and network resource utilization in distributed storage systems.
9. The method of claim 1 , further comprising saving bandwidth in a snapshot shipping replication environment.
The method utilizes the "write signature command" to save bandwidth within a snapshot shipping replication environment.
10. The method of claim 9 wherein saving bandwidth in a snapshot shipping replication environment comprises: generating a first snapshot at a first site; tracking changes occurring at the first site; performing an initial sweep from the first snapshot to the second site; erasing the first snapshot; generating a second snapshot at the first site; transferring the changes from the first site to the second site; and renaming the second snapshot to be the first snapshot, where transferring the changes comprises: reading signatures of the changed locations; writing signatures to a volume at the replica site using the signatures read at the first site; and on failure reading the data from the location at the first site and writing the data to the changed locations at the replica site.
Saving bandwidth in snapshot replication involves creating a first snapshot at the source site and tracking subsequent changes. An initial data sweep populates the replica. Later, only changes are transferred. A second snapshot is created, and the changes since the first snapshot are transferred by reading signatures of changed locations, writing those signatures to the replica site's volume, and, upon failure (signature not found), transferring the actual data from the source to the replica for the changed locations. Finally the second snapshot becomes the first snapshot.
11. An article comprising: a non-transitory machine-readable medium that stores executable instructions, the instructions causing a machine to: generate a write signature command, the write signature command having parameters used in execution comprising an offset, a number of blocks of data and a signature, the write signature command configured to: write the signature to the offset in a storage array without the data, the signature being a hash value generated using a hash function on the data; and enable the storage array to write the data with the same signature to a volume if the data is available at the storage array; execute the write signature command to synchronize data between a second site and a first site, the instructions causing a machine to execute the write signature command comprising instructions causing a machine to: read a first hash value having a first offset at the first site associated with a block of data at the first site; send the first hash value to the second site; use the write signature command with the first hash value received from the first site to determine if there is a block of data stored at the second site associated with the first hash value, wherein the write signature command uses the first hash value to copy a block of data, from a location at the second site indicated in a hash table associated with the first hash value, to a second offset at the second site if the first hash value is found in the hash table; determine if the write signature command was successful by determining there is the block of data stored at the second site associated with the first hash value; and if the write command is not successful: read the block of data from the first site associated with the first hash value; and write the block of data from the first site associated with the first hash value to the second offset at the second site, the second offset being equal to the first offset.
A non-transitory machine-readable medium stores instructions to efficiently synchronize data between two storage sites by generating a "write signature command." This command includes parameters like the data's offset, block count, and a signature (hash value). The command first writes the signature to the specified offset at the destination site without transferring the actual data. The destination site then checks if it already has the data associated with that signature in its storage array. If a match is found (based on a hash table lookup), the destination site copies the existing data to the specified offset, avoiding data transfer. If no match is found, the data is read from the source site and written to the destination offset.
12. The article of claim 11 , further comprising instructions causing the machine to synchronize data between a first site and a second site without sending the data from the first site to the second site if the second site already contains data from first site located in any location in the storage array.
The non-transitory machine-readable medium from the previous data synchronization description contains further instructions to avoid transferring data between two storage sites if the destination site already holds a copy of that data anywhere in its storage array. The destination site utilizes the signature to check if a matching block exists, copying it to the correct location instead of requesting the data from the source.
13. The article of claim 11 , further comprising instructions causing the machine to: receive a write at the storage array; read a signature for the write; invalidate a previous hash table entry for the signature; determine a hash value for the write; and update the hash table.
The non-transitory machine-readable medium from the data synchronization description contains further instructions that when a write operation occurs at the storage array, the method reads the existing signature associated with that write location. It then invalidates any previous hash table entry related to that signature. A new hash value is calculated for the new data being written, and the hash table is updated to reflect this new data and its location.
14. An apparatus, comprising: hardware circuitry to generate a write signature command, the write signature command configured to: generate a write signature command, the write signature command having parameters used in execution comprising an offset, a number of blocks of data and a signature, the write signature command configured to: write the signature to the offset in a storage array without the data, the signature being a hash value generated using a hash function on the data; and enable the storage array to write the data with the same signature to a volume if the data is available at the storage array; execute the write signature command to synchronize data between a second site and a first site, the circuitry configured to execute the write signature command comprising circuitry configured to: read a first hash value having a first offset at the first site associated with a block of data at the first site; send the first hash value to the second site; use the write signature command with the first hash value received from the first site to determine if there is a block of data stored at the second site associated with the first hash value, wherein the write signature command uses the first hash value to copy a block of data, from a location at the second site indicated in a hash table associated with the first hash value, to a second offset at the second site if the first hash value is found in the hash table; determine if the write signature command was successful by determining there is the block of data stored at the second site associated with the first hash value; and if the write command is not successful: read the block of data from the first site associated with the first hash value; and write the block of data from the first site associated with the first hash value to the second offset at the second site, the second offset being equal to the first offset.
An apparatus includes hardware circuitry designed to efficiently synchronize data between two storage sites by generating a "write signature command." This command includes parameters like the data's offset, block count, and a signature (hash value). The command first writes the signature to the specified offset at the destination site without transferring the actual data. The destination site then checks if it already has the data associated with that signature in its storage array. If a match is found (based on a hash table lookup), the destination site copies the existing data to the specified offset, avoiding data transfer. If no match is found, the data is read from the source site and written to the destination offset.
15. The apparatus of claim 14 wherein the circuitry comprises at least one of a processor, a memory, programmable logic and logic gates.
The apparatus from the previous data synchronization description uses circuitry that comprises at least one of a processor, a memory, programmable logic, and logic gates.
16. The apparatus of claim 14 , further comprising circuitry to synchronize data between a first site and a second site without sending the data from the first site to the second site if the second site already contains data from first site located in any location in the storage array.
The apparatus from the previous data synchronization description includes further circuitry to avoid transferring data between two storage sites if the destination site already holds a copy of that data anywhere in its storage array. The destination site utilizes the signature to check if a matching block exists, copying it to the correct location instead of requesting the data from the source.
17. The apparatus of claim 14 , further comprising circuitry to: receive a write at the storage array; read a signature for the write; invalidate a previous hash table entry for the signature; determine a hash value for the write; and update the hash table.
The apparatus from the previous data synchronization description includes further circuitry that when a write operation occurs at the storage array, the method reads the existing signature associated with that write location. It then invalidates any previous hash table entry related to that signature. A new hash value is calculated for the new data being written, and the hash table is updated to reflect this new data and its location.
Unknown
November 25, 2014
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.