Folded-Hashtable Synchronization Mechanism

1. A system comprising: one or more storage devices having stored thereon a plurality of records; one or more computing devices; and one or more memories having stored thereon computer readable instructions that, upon execution by the one or more computing devices, cause the system at least to: select an output range of a hash function, the output range selected to be greater than or equal to a number of records of a plurality of records stored in a repository, the output range further selected to be larger than a number of storage locations in a plurality of storage locations associated with a hashtable; select a first storage location of the plurality of storage locations, the first storage location selected for storing information indicative of a change to a first record, the first storage location selected based at least in part on a first hash value formed by applying the hash function to a first identifier of the first record; select the first storage location for storing, in addition to the information indicative of the change to the first record, information indicative of changes to one or more additional records, the first storage location selected based at least in part on application of the hash function to identifiers of the one or more additional records and a mapping of output of the hash function onto the first storage location; store, in the first storage location, a first sum including at least the first identifier and the identifiers of the one or more additional records; store, in the first storage location, a first count of identifiers included in the first sum; transmit data indicative of the first sum and the first count in the first storage location to a remote computing device; determine to unfold the first storage location into a plurality of storage locations, based at least in part on receiving an indication that the first sum and the first count were not sufficient to determine the first identifier, wherein unfolding the first storage location comprises determining a subset of information indicative of the first identifier that was not represented by the first sum and the first count; and transmit at least the subset of information to the remote computing device.

2. The system of claim 1 , further comprising one or more memories having stored thereon computer readable instructions that, upon execution by the one or more computing devices, cause the system at least to: select a second storage location of the plurality of storage locations for combining with the first storage location, wherein the second storage location comprises information indicative of a third record; and include an identifier of the third record in the first sum stored in the first storage location and incrementing the first count.

3. The system of claim 1 , further comprising one or more memories having stored thereon computer readable instructions that, upon execution by the one or more computing devices, cause the system at least to: store information indicative of a change to an additional record in an additional storage location of the plurality of storage locations; and transmit data indicative of a subset of the plurality of storage locations to the remote computing device, wherein the subset of the plurality includes the additional storage location and contains fewer storage locations than the plurality of storage locations.

4. The system of claim 1 , further comprising one or more memories having stored thereon computer readable instructions that, upon execution by the one or more computing devices, cause the system at least to: receive, from the remote computing device, data corresponding to the first storage location, the data comprising a second sum including at least the first identifier, the identifiers of the one or more additional records, and a third identifier; and determine that a third record corresponding to the third identifier has been modified, based at least in part on the second sum.

5. A computer-implemented method comprising: selecting a first storage location of a plurality of storage locations associated with a hashtable, the first storage location selected for storing information indicative of a first record of a plurality of records, the first storage location selected based at least in part on a first hash value formed by applying a hash function to a first identifier of the first record, the hash function having an output range greater than a number of storage locations in the plurality of storage locations and having an output range greater than or equal to a number of records in the plurality of records; selecting the first storage location for storing information indicative of a second record, the first storage location selected for storing the second record based at least in part on a second hash value formed by applying a second identifier of the second record to the hash function; storing the information indicative of the first record and the information indicative of the second record by at least storing, in the first storage location, a first sum indicative of the first identifier and the second identifier and a first count indicative of a number of identifiers included in the first sum; transmitting the first sum and the first count from the first storage location to a remote computing device; and transmitting, in response to an indication that the first sum and the first count were not sufficient to determine the first identifier, additional information indicative of the first identifier, the information determined based at least in part on determining a subset of information indicative of the first identifier that was not represented by the first sum and the first count.

6. The computer-implemented method of claim 5 , further comprising: selecting, in response to determining to form a compressed version of the plurality of storage locations, a second storage location of the plurality of storage locations for combining with the first storage location, wherein the second storage location comprises information indicative of a third record; and including an identifier of the third record in the first sum stored in the first storage location and incrementing the first count stored in the first storage location.

7. The computer-implemented method of claim 5 , further comprising: storing in an additional storage location information indicative of a change to an additional record, the information comprising a second sum including an identifier of the additional record and a second count of identifiers included in the sum; and transmitting a subset of the plurality of storage locations to the remote computing device, wherein the subset of the plurality includes the additional storage location and contains fewer storage locations than the plurality of storage locations.

8. The computer-implemented method of claim 5 , further comprising: storing, in the first storage location, a value formed based at least in part on applying a checksum function to the first identifier and applying the checksum function to the second identifier.

9. The computer-implemented method of claim 5 , wherein applying the hash function to the first identifier of the first record produces at least the first hash value and one or more additional hash values, the method further comprising: selecting a second storage location, in addition to the first storage location, for storing the information indicative of the first record, the selecting based at least in part on one of the one or more additional hash values.

10. The computer-implemented method of claim 5 , further comprising: receiving, from the remote computing device, information corresponding to the first storage location, the information comprising a second sum including an additional identifier of a modified record, and a second count indicative of a number of identifiers included in the second sum; and determining the identifier of the modified record based at least in part on the second sum and the second count indicative of the number of identifiers.

11. The computer-implemented method of claim 10 , further comprising: send to the remote computing device a request to transmit the modified record.

12. The computer-implemented method of claim 5 , further comprising: storing, in the first storage location, a checksum value computed based at least in part on a sum of cryptographic hash values of document identifiers, wherein the cryptographic hash values are computed with a cryptographic hash function that is different than the hash function; and determining the subset of information indicative of the first identifier that was not represented by the first value and the second value, based at least in part on the checksum value.

13. A non-transitory computer-readable storage medium having stored thereon instructions that, upon execution by one or more computing devices, cause the one or more computing devices at least to: configure an output range of a hash function, the output range selected to be greater than or equal to a number of records of a plurality of records stored in a repository, the output range further selected to be larger than a number of storage locations in a plurality of storage locations associated with a hashtable; select a first storage location of the plurality of storage locations, the first storage location selected for storing information indicative of a change to a first record, the first storage location selected based at least in part on a first hash value formed by applying the hash function to a first identifier of the first record; select the first storage location for storing information indicative of a change to a second record, the first storage location selected based at least in part on a second hash value formed by applying the hash function to a second identifier of the second record, the second hash value corresponding to at least one of a virtual storage location or one or more of the plurality of storage locations; store a first value in the first storage location, the first value based at least in part on a plurality of identifiers including the first identifier and the second identifier; store, in the first storage location, a second value based at least in part on a number of identifiers in the plurality of identifiers; and transmit, in response to an indication that the first value and the second value were not sufficient to determine the first identifier, additional information indicative of unfolding the first storage location into a plurality of storage locations, wherein the additional information is determined based at least in part on determining a subset of information indicative of the first identifier that was not represented by the first value and the second value.

14. The non-transitory computer-readable storage medium of claim 13 , comprising further instructions that, upon execution by the one or more computing devices, cause the one or more computing devices to at least: update the first value to include a number of additional document identifiers, wherein a count of the additional document identifiers is determined based at least in part on a degree of compression.

15. The non-transitory computer-readable storage medium of claim 13 , comprising further instructions that, upon execution by the one or more computing devices, cause the one or more computing devices to at least: store in an additional storage location information indicative of a change to an additional record; and transmit a subset of the plurality of storage locations to a remote computing device, wherein the subset of the plurality includes the additional storage location and contains fewer storage locations than the plurality of storage locations.

16. The non-transitory computer-readable storage medium of claim 13 , comprising further instructions that, upon execution by the one or more computing devices, cause the one or more computing devices to at least: form a checksum value for storage in the first storage location, the checksum value based at least in part on applying a checksum function to the first identifier and applying the checksum function to the second identifier.

17. The non-transitory computer-readable storage medium of claim 13 , comprising further instructions that, upon execution by the one or more computing devices, cause the one or more computing devices to at least: receive information corresponding to the first storage location, the information comprising a third value determined based at least in part on the first identifier, the second identifier, and a third identifier; and identify a third record corresponding to the third identifier based at least in part on the third value.

18. The non-transitory computer-readable storage medium of claim 17 , comprising further instructions that, upon execution by the one or more computing devices, cause the one or more computing devices to at least: send to a remote computing device a request to send transmit contents of the third record.

19. The non-transitory computer-readable storage medium of claim 13 , comprising further instructions that, upon execution by the one or more computing devices, cause the one or more computing devices to at least: store, in the first storage location, a checksum value computed based at least in part on a sum of cryptographic hash values of document identifiers, wherein the cryptographic hash values are computed with a cryptographic hash function that is different than the hash function; and determine the subset of information indicative of the first identifier that was not represented by the first value and the second value, based at least in part on the checksum value.

20. The non-transitory computer-readable storage medium of claim 13 , wherein an output range of the hash function is equal to or greater than a square of the number of records in the plurality of records.