In-Memory Distributed Cache

1. A computer-implemented method for implementing a distributed cache, the method comprising executing instructions in a computer system to perform the operations of: receiving a request from a cache client for a requested value associated with a cache key; in response to receiving the request, searching for the requested value in the distributed cache; in response to determining that the requested value is not stored in the cache: determining a quantity of cache clients that are currently fetching the requested value, determining that the quantity of cache clients that are currently fetching the requested value meets or exceeds a predefined threshold number, determining, based at least in part on the quantity of cache clients that are currently fetching the requested value meeting or exceeding the predefined threshold number, that the cache client is permitted to wait for the requested value, and providing, to the cache client, an indication in response to the request that the cache client is permitted to wait for the requested value to become available in the distributed cache.

2. The computer-implemented method of claim 1 , further comprising returning a null value as the indication in response to determining that the quantity of cache clients that are currently fetching the requested value does not meet or exceed the predefined threshold number.

3. The computer-implemented method of claim 1 , wherein the indication is a first indication, the method further comprising: receiving a second indication from the cache client that the cache client is waiting for the requested value to become available in the distributed cache; and in response to receiving the second indication, adding the cache client to a data structure containing data that identifies cache clients waiting for the requested value.

4. The computer-implemented method of claim 1 , wherein the request is a first request, the cache client is a first cache client, the cache key is a first cache key, and the value is a first value, the method further comprising: receiving a second request from a second cache client at a cache server to add a second cache key and a second value to the distributed cache; in response to receiving the second request, adding the second cache key and the second value to the distributed cache, and determining based upon contents of a data structure that a third cache client is waiting on the second value; and in response to determining that the third cache client is waiting on the second value, providing the second value to the third cache client.

5. The computer-implemented method of claim 4 , further comprising removing the third cache client from the data structure following the provision of the second value to the third cache client.

6. The computer-implemented method of claim 1 , wherein the cache client is a first cache client, the cache key is a first cache key, and the value is a first value, the method further comprising: receiving an indication from a second cache client indicating that the second cache client is fetching a value for a second cache key; and in response to receiving the indication that the second cache client is fetching a value for the second cache key, incrementing a variable associated with the second cache key that is utilized to keep track of a number of cache clients that are fetching the value for the second cache key.

7. The computer-implemented method of claim 1 , wherein the predefined threshold number is user-configurable.

8. A computer-implemented method comprising executing instructions in a computer system to perform the operations of: exposing, by a service, an application programming interface (API) configured to receive a request for the service to cause one or more updates to a value stored in a cache of a requesting entity, the request comprising the value and a cache key associated with the value as stored in the cache of the requesting entity; determining that the value has changed to a new value; and transmitting, by the service and based on the value having changed, a cache update directly to the cache of the requesting entity, wherein the cache update causes a stored value in the cache of the requesting entity to be changed to the new value, wherein the service and the requesting entity are separate entities.

9. The computer-implemented method of claim 8 , wherein the request further comprises identifying, by a node identifier (ID), a cache node in the cache of the requesting entity that is storing the stored value.

10. The computer-implemented method of claim 9 , wherein transmitting the cache update comprises utilizing, by the service, the node ID provided with the request to cause the stored value to be updated by the cache node in the cache of the requesting entity that is storing the stored value.

11. The computer-implemented method of claim 8 , wherein the request further comprises a time to live (TTL) value defining a TTL for the request.

12. The computer-implemented method of claim 11 , wherein the operations for determining that the value has changed and transmitting the cache update are performed by the service only until the TTL specified with the request has expired.

13. The computer-implemented method of claim 8 , wherein the cache of the requesting entity comprises an in-memory distributed cache comprised of a plurality of cache nodes.

14. A computer-implemented method for storing data in a cache, the method comprising executing instructions in a computer system to perform the operations of: storing a first set of cache keys in the cache, individual keys of the first set of cache keys being associated with a corresponding value stored in the cache; and storing a second set of cache keys in the cache, individual keys of the second set of cache keys corresponding to a received request for two or more values, individual keys of the second set of cache keys comprising two or more corresponding cache keys of the first set of cache keys stored in the cache.

15. The computer-implemented method of claim 14 , wherein the two or more corresponding cache keys are associated with cache keys for two or more corresponding values that a corresponding request is dependent upon.

16. The computer-implemented method of claim 14 , wherein individual cache keys of the first set of cache keys has an assigned time to live (TTL) value stored in the cache.

17. The computer-implemented method of claim 16 , further comprising: invalidating one or more cache keys of the first set of cache keys based upon a corresponding TTL value; and in response to invalidating the one or more cache keys, utilizing a cache key of the second set of cache keys to invalidate one or more other cache keys that are dependent upon the invalidated one or more cache keys.

18. The computer-implemented method of claim 17 , wherein the cache comprises an in-memory distributed cache.

19. The computer-implemented method of claim 17 , wherein the cache comprises a front-end cache utilized by a service to cache request to the service.

20. The computer-implemented method of claim 19 , wherein the first set of cache keys correspond to values obtained by the service from one or more other services for use in satisfying a request.