Distributed Caching System

1. A system for distributed data caching, the system comprising: an inter-process communication interface for receiving a plurality of requests via a computer network, wherein the requests include a first request for a first data item from a first client device and a second request for a second data item from a second client device and wherein the first client device differs from the second client device; computer memory for storing a local cache; and one or more processors configured to: determine whether the first data item is stored in the local cache; in response to determining that the first data item is not stored in the local cache, determine a first external cache from a plurality of external caches, the first external cache designated to store the first data item; that the second request for the second data item is designated for the first external cache; multiplex the first request for the first data item from the first client device and the second request for the second data item from the second client device to the first external cache over a single negotiated streaming protocol connection socket; receive the first data item from the first external cache; and provide the first data item to the first client device.

2. The system of claim 1 , wherein the one or more processors are configured to detect inactive connections and close the inactive connections.

3. The system of claim 1 , wherein the single negotiated streaming protocol connection socket is a transmission control protocol (TCP) socket or an inter-process communication (IPC) socket.

4. The system of claim 1 , wherein the one or more processors are configured to: associate multiple threads that handle requests from client devices to a plurality of network connections to the client devices; and dynamically assign a request received on one of the plurality of network connections to one of the associated multiple threads.

5. The system of claim 1 further comprising a distributed caching client configured to: obtain the first data item from the external cache; and transmit the first data item directly on a network connection to the first client device without waking a thread assigned to the first client device.

6. The system of claim 1 , wherein the one or more processors are configured to determine a first external cache from a plurality of external caches by obtaining an identifier based at least partly on the first request and applying a deterministic algorithm to the identifier such that the deterministic algorithm identifies the first external cache.

7. The system of claim 1 , wherein the first data item is received prior to the second data item.

8. The system of claim 1 , wherein the first data item is received after the second data item.

9. A method of caching data comprising: receiving, on a first caching system, a first request for a first data item from a first computing device and a second request for a second data item from a second computing device, wherein the first computing device differs from the second computing device and wherein the first and second requests are received via a computer network; in response to determining that the first data item is not stored in a local cache, identifying a first external cache designated to store the first data item from a plurality of external caches; determining that the second request for the second data item is designated for the first external cache; multiplexing the first request for the first data item from the first computing device and the second request for the second data item from the second computing over a single negotiated streaming protocol connection socket to the first external cache; receiving the first data item from the first external cache; and providing the first data item to the first computing device.

10. The method of claim 9 , wherein the single protocol connection comprises a single negotiated streaming protocol connection.

11. The method of claim 10 , wherein the negotiated streaming protocol connection is a Transmission Control Protocol (TCP) connection.

12. The method of claim 10 , wherein the negotiated streaming protocol connection uses a UNIX domain socket.

13. The method of claim 9 , further comprising detecting inactive connections to one or more external caches and attempting to connect to one or more external caches using a background thread of execution.

14. The method of claim 9 , further comprising: associating multiple threads that handle requests from computing devices to a plurality of network connections to the computing devices; and dynamically assigning a request received on one of the plurality of network connections to one of the associated multiple threads.

15. The method of claim 9 , wherein identifying the first external cache from a plurality of external caches comprises: obtaining an identifier based at least partly on the first request; and applying a deterministic function to the identifier, wherein the deterministic function identifies the first external cache.

16. The method of claim 9 , wherein the first computing device comprises a service module configured to handle requests received from a plurality of client devices.

17. The method of claim 9 , wherein the first computing is a first client device.

18. The method of claim 9 , wherein the first computing device executes a thread associated with the first request.

19. The method of claim 9 , wherein the local cache stores data using fixed size storage blocks.

20. The method of claim 9 , wherein the local cache stores data using a slab allocation technique or a best fit allocation technique.

21. The method of claim 9 , further comprising storing the first data item in the local cache.

22. Non-transitory computer storage having stored thereon instructions that, when executed by a computer system, cause the computer system to perform operations comprising: receiving a first request for a first data item from a first computing device and a second request for a second data item from a second request for a second data item from a second computing device, wherein the first computing device differs from the second computing device and wherein the first and second requests are received via a computer network; identifying a first external cache designated to store the first data item from a plurality of external caches; determining the first external cache is designated to store the second data item; multiplexing the first request for the first data item from the first computing device and the second request for the second data item from the second computing device over a single connection socket to the first external cache; and receiving the first data item or the second data item from the first external cache.

23. The non-transitory computer storage of claim 22 , wherein the first request and the second request are multiplexed over a single transmission control protocol (TCP) socket or an inter-process communication (IPC) socket.

24. The non-transitory computer storage of claim 22 , wherein the first request and the second request are multiplexed over a first connection protocol for local communications in the computer system and a second connection protocol for network communications.

25. The non-transitory computer storage of claim 22 , wherein the first request is received from a first client device from a plurality of client devices.

26. The non-transitory computer storage of claim 25 , wherein the instructions further cause the computer system to perform operations comprising: detecting inactive connections to one or more external caches; and attempting to connect to the one or more external caches using a background thread of execution.

27. The non-transitory computer storage of claim 25 , wherein the instructions further cause the computer system to perform operations comprising: associating multiple threads that handle requests from the client devices to a plurality of network connections to the client devices; and assigning a request received on one of the plurality of network connections to one of the associated multiple threads.

28. The non-transitory computer storage of claim 22 , wherein identifying the first external cache comprises: obtaining an identifier based at least partly on the first request; and applying a deterministic function to the identifier, wherein the deterministic function identifies the first external cache.

29. The non-transitory computer storage of claim 22 , wherein either the first data item or the second data item is received by the computing system first.