Data Differencing Across Peers in an Overlay Network

1. Apparatus operative in a receiving peer entity of an overlay network, comprising: a processor; a dictionary comprising, for a piece of content associated with a host/path, one or more versions of the piece of content and, for each version, a fingerprint; an HTTP proxy; computer memory holding program code executed by the processor to perform data differencing between the receiving peer entity and a sending peer entity of the overlay network, the program code comprising: program code operative to receive from the HTTP proxy a request for a new version of the piece of content that is not available at the HTTP proxy, to associate with the request one or more fingerprints corresponding to the one or more versions of the piece of content in the dictionary, and to return to the HTTP proxy, for delivery forward towards an origin server, a modified request that includes the one or more fingerprints; and program code operative to receive from the HTTP proxy a data stream, the data stream having been received by the HTTP proxy in response to the modified request, the data stream including data representing a difference between the new version of the piece of content and that version of the piece of content represented by a given one of the one or more fingerprints that was included in the modified request, the data stream also including an identification of the given one of the one or more fingerprints, to generate the new version of the piece of content from the difference and the version of the piece of content represented by the given one of the fingerprints, and to return to the HTTP proxy, for delivery back to a requesting client, the new version of the piece of content so generated.

2. The apparatus as described in claim 1 wherein the data differencing is performed when the piece of content has a given content encoding.

3. The apparatus as described in claim 2 wherein the given content encoding is an identity content encoding.

4. The apparatus as described in claim 1 wherein the program code is further operative to generate and store in the dictionary a new fingerprint corresponding to the new version of the piece of content so generated.

5. The apparatus as described in claim 1 wherein the data differencing is enabled for any piece of content that is other than in a compressed format.

6. The apparatus as described in claim 1 wherein the one or more fingerprints corresponding to the one or more versions of the piece of content in the dictionary are associated with the request in an HTTP header.

7. The apparatus as described in claim 1 wherein the HTTP proxy is configurable and communicates with the program code over an interface.

8. The apparatus as described in claim 1 wherein a fingerprint is generated by hashing a version of the piece of content.

9. Apparatus operative in a sending peer entity of an overlay network, comprising: a processor; a dictionary comprising, for a piece of content associated with a host/path, one or more versions of the piece of content and, for each version, a fingerprint; an HTTP proxy; computer memory holding program code executed by the processor to perform data differencing between the sending peer entity and a receiving peer entity, the program code comprising: program code operative to receive from the HTTP proxy a request for a new version of the piece of content, the request having been modified to include one or more fingerprints corresponding to the one or more versions of the piece of content that are then available at the receiving peer entity, and to select a given one of the one or more fingerprints, to instruct the HTTP proxy to obtain from an origin server the new version of the piece of content; and program code operative to receive from the HTTP proxy the new version of the piece of content provided by the origin server, to generate data representing a difference between the new version of the piece of content and that version of the piece of content represented by the given one of the one or more fingerprints, and to return to the HTTP proxy, for delivery to the receiving peer entity, a data stream including the differenced data and an identification of the given one of the one or more fingerprints.

10. The apparatus as described in claim 9 wherein the data differencing is performed when the piece of content has a given content encoding.

11. The apparatus as described in claim 10 wherein the given content encoding is an identity content encoding.

12. The apparatus as described in claim 9 wherein the program code is further operative to generate and store in the dictionary a new fingerprint corresponding to the new version of the piece of content provided by the origin server.

13. The apparatus as described in claim 9 wherein the data differencing is enabled for any piece of content that is other than in a compressed format.

14. The apparatus as described in claim 9 wherein the one or more fingerprints corresponding to the one or more versions of the piece of content are associated with the request in an HTTP header.

15. The apparatus as described in claim 9 wherein the HTTP proxy is configurable and communicates with the program code over an interface.

16. The apparatus as described in claim 9 wherein a fingerprint is generated by hashing a version of the piece of content.

17. A data differencing system, comprising: a receiving peer entity comprising a first HTTP proxy, comprising: a first dictionary comprising, for a piece of content associated with a host/path, one or more versions of the piece of content and, for each version, a fingerprint; processor-executed program code operative to receive from the first HTTP proxy a request for a new version of the piece of content that is not available at the first HTTP proxy, to associate with the request one or more fingerprints corresponding to the one or more versions of the piece of content in the first dictionary, and to return to the first HTTP proxy, for delivery forward towards an origin server, a modified request that includes the one or more fingerprints; a sending peer entity including a second HTTP proxy, comprising: a second dictionary comprising, for a piece of content associated with a host/path, one or more versions of the piece of content and, for each version, a fingerprint; processor-executed program code operative to receive from the second HTTP proxy the modified request, to select a given one of the one or more fingerprints, and to instruct the second HTTP proxy to obtain from an origin server the new version of the piece of content; and processor-executed program code operative to receive from the second HTTP proxy the new version of the piece of content provided by the origin server, to generate data representing a difference between the new version of the piece of content and that version of the piece of content represented by the given one of the one or more fingerprints, and to return to the second HTTP proxy, for delivery to the receiving peer entity, a data stream including the data and an identification of the given one of the one or more fingerprints; the receiving peer entity further including processor-executed program code operative to receive from the first HTTP proxy the data stream, to generate the new version of the piece of content from the data representing the difference and the version of the piece of content represented by the given one of the fingerprints, and to return to the first HTTP proxy, for delivery back to a requesting client, the new version of the piece of content so generated.