System and Method for Adaptive Server Shielding

1. A method comprising: one or more computers implementing an upstream server, the upstream server: receiving a request from a client for service, wherein the service is dependent on a downstream service provided by a downstream server separate from the upstream server; determining that the downstream server should be shielded from a request for the downstream service, wherein said determining is dependent on ascertaining that the downstream server is in a hotspot situation; in response to said determining, shielding the downstream server from the request for the downstream service, said shielding comprising: servicing the client request without making the request for the downstream service to the downstream server; and providing the downstream server with an indication that the upstream server shielded the downstream server from the request for the downstream service.

2. The method of claim 1 , wherein said determining that the downstream server should be shielded from the request is further dependent on a shielding policy of the upstream server.

3. The method of claim 2 , wherein the shielding policy indicates whether the downstream server should be shielded dependent on an identity of the client.

4. The method of claim 2 , wherein the shielding policy indicates whether the downstream server should be shielded dependent on one or more parameters of the request for the downstream service, the one or more parameters having been indicated to the upstream server by the downstream server in response to an earlier request for service sent by the upstream server to the downstream server.

5. The method of claim 1 , wherein said servicing the client request without making the request to the downstream server comprises making the request for the downstream service to a different downstream server.

6. The method of claim 1 , wherein the service requested by the client is dependent on service data provided authoritatively by the downstream service and wherein said servicing the client request without making the request to the downstream server comprises using a cached version of the service data.

7. The method of claim 6 , further comprising, determining that the cached version of the service data will become stale within a given interval and in response: requesting a fresh version of the service data from the downstream server; in response to said requesting, receiving and caching the fresh version of the service data; and shielding the downstream server from a subsequent request, said shielding comprising servicing the subsequent request using the cached fresh version of the service data without requesting the service data from the downstream server in response to receiving the subsequent client request.

8. The method of claim 7 , further comprising: subsequent to determining that the cached version of the service data will become stale within the given interval and previous to said receiving the fresh version of the service data, servicing one or more client requests using the cached version of the service data.

9. The method of claim 1 , wherein the indication that the upstream server shielded the downstream server from the request is part of a shielding report sent by the upstream server to the downstream server, wherein the shielding report further includes indications that the upstream system shielded the downstream server from a plurality of other requests.

10. The method of claim 9 , wherein the shielding report indicates an aggregate workload metric, the aggregate workload metric being usable to determine a rate at which the plurality of client requests were made.

11. A non-transitory computer readable storage medium storing program instructions executable to implement an upstream server configured to: receive a request from a client for service, wherein the service is dependent on a downstream service provided by a downstream server separate from the upstream server; determine that the downstream server should be shielded from a request for the downstream service, wherein said determining is dependent on ascertaining that the downstream server is in a hotspot situation; in response to said determining, to shield the downstream server from the request for the downstream service, said shielding comprising: servicing the client request without making the request for the downstream service to the downstream server; and providing the downstream server with an indication that the upstream server shielded the downstream server from the request for the downstream service.

12. The non-transitory computer-readable storage medium of claim 11 , wherein said determining that the downstream server should be shielded from the request is further dependent on a shielding policy of the upstream server.

13. The non-transitory computer-readable storage medium of claim 12 , wherein the shielding policy indicates whether the downstream server should be shielded dependent on one or more parameters of the request for the downstream service, the one or more parameters having been indicated to the upstream server by the downstream server in response to an earlier request for service sent by the upstream server to the downstream server.

14. The non-transitory computer-readable storage medium of claim 11 , wherein the service requested by the client is dependent on service data provided authoritatively by the downstream service and wherein said servicing the client request without making the request to the downstream server comprises using a cached version of the service data.

15. The non-transitory computer-readable storage medium of claim 14 , wherein the upstream server is further configured to determine that the cached version of the service data will become stale within a given interval and in response to: request a fresh version of the service data from the downstream server; in response to said requesting, receive and caching the fresh version of the service data; and shield the downstream server from a subsequent request, said shielding comprising servicing the subsequent request using the cached fresh version of the service data without requesting the service data from the downstream server in response to receiving the subsequent client request.

16. The non-transitory computer-readable storage medium of claim 11 , wherein the indication that the upstream server shielded the downstream server from the request is part of a shielding report sent by the upstream server to the downstream server, wherein the shielding report further includes indications that the upstream system shielded the downstream server from a plurality of other requests, wherein the shielding report indicates an aggregate workload metric, the aggregate workload metric being usable to determine a rate at which the plurality of client requests were made.

17. A computer system comprising: a processor; memory coupled to the processor, the memory storing program instructions executable by the processor to implement an upstream server configured to: receive a request from a client for service, wherein the service is dependent on a downstream service provided by a downstream server separate from the upstream server; determine that the downstream server should be shielded from a request for the downstream service, wherein said determining is dependent on ascertaining that the downstream server is in a hotspot situation; in response to said determining, to shield the downstream server from the request for the downstream service, said shielding comprising: servicing the client request without making the request for the downstream service to the downstream server; and providing the downstream server with an indication that the upstream server shielded the downstream server from the request for the downstream service.

18. The system of claim 17 , wherein said determining that the downstream server should be shielded from the request is further dependent on a shielding policy of the upstream server.

19. The system of claim 18 , wherein the shielding policy indicates whether the downstream server should be shielded dependent on one or more parameters of the request for the downstream service, the one or more parameters having been indicated to the upstream server by the downstream server in response to an earlier request for service sent by the upstream server to the downstream server.

20. The system of claim 17 , wherein the service requested by the client is dependent on service data provided authoritatively by the downstream service and wherein said servicing the client request without making the request to the downstream server comprises using a cached version of the service data.

21. The system of claim 20 , wherein the upstream server is further configured to determine that the cached version of the service data will become stale within a given interval and in response to: request a fresh version of the service data from the downstream server; in response to said requesting, receive and caching the fresh version of the service data; and shield the downstream server from a subsequent request, said shielding comprising servicing the subsequent request using the cached fresh version of the service data without requesting the service data from the downstream server in response to receiving the subsequent client request.

22. The system of claim 17 , wherein the indication that the upstream server shielded the downstream server from the request is part of a shielding report sent by the upstream server to the downstream server, wherein the shielding report further includes indications that the upstream system shielded the downstream server from a plurality of other requests, wherein the shielding report indicates an aggregate workload metric, the aggregate workload metric being usable to determine a rate at which the plurality of client requests were made.

23. A method comprising: a downstream server implemented by one or more computer systems performing: receiving a request from a separate upstream server for service; determining that the downstream server is in a hotspot situation; in response to said receiving and said determining, sending the upstream server a notification that the downstream server is in a hotspot situation, wherein the notification includes one or more hints usable by the upstream server to determine a benefit to the downstream server of the upstream server shielding the downstream server from a future request for service.

24. The method of claim 23 , wherein the one or more hints indicate at least one of: a form of the future request for service, one or more parameters of the future request for service, or a relative measure of the benefit to the downstream server.

25. The method of claim 23 , further comprising: receiving a shielding report from the upstream server, the shielding report indicating that the upstream server shielded the downstream server from a plurality of service requests; calculating an aggregate workload estimate for the downstream server based at least in part on one or more shielding reports received by the downstream server, the one or more shielding reports including the shielding report; determining whether the downstream server is in a hotspot situation, said determining being dependent at least in part on the aggregate workload estimate; and providing an indication to the upstream server of whether the downstream server is in a hotspot situation.

26. The method of claim 25 , wherein said calculating the workload estimate comprises calculating a decaying moving average of a rate of requests shielded by a group of one or more upstream servers including the upstream server.

27. A non-transitory computer-readable storage medium comprising program instructions executable by a processor to implement a downstream server configured to: receive a request from a separate upstream server for service; determine that the downstream server is in a hotspot situation; and in response to said receiving and said determining, to send the upstream server a notification that the downstream server is in a hotspot situation, wherein the notification includes one or more hints usable by the upstream server to determine a benefit to the downstream server of the upstream server shielding the downstream server from a future request for service.

28. The non-transitory computer-readable storage medium of claim 27 , wherein the one or more hints indicate at least one of: a form of the future request for service, one or more parameters of the future request for service, or a relative measure of the benefit to the downstream server.

29. The non-transitory computer-readable storage medium of claim 27 , wherein the downstream server is further configured to: receive a shielding report from the upstream server, the shielding report indicating that the upstream server shielded the downstream server from a plurality of service requests; calculate an aggregate workload estimate for the downstream server based at least in part on one or more shielding reports received by the downstream server, the one or more shielding reports including the shielding report; determine whether the downstream server is in a hotspot situation, said determining being dependent at least in part on the aggregate workload estimate; and provide an indication to the upstream server of whether the downstream server is in a hotspot situation.

30. The non-transitory computer-readable storage medium of claim 29 , wherein said calculating the workload estimate comprises calculating a decaying moving average of a rate of requests shielded by a group of one or more upstream servers including the upstream server.

31. A system comprising: a processor; a memory coupled to the processor and storing program instructions executable by the processor to implement a downstream server configured to: receive a request from a separate upstream server for service; determine that the downstream server is in a hotspot situation; and in response to said receiving and said determining, to send the upstream server a notification that the downstream server is in a hotspot situation, wherein the notification includes one or more hints usable by the upstream server to determine a benefit to the downstream server of the upstream server shielding the downstream server from a future request for service.

32. The system of claim 31 , wherein the downstream server is further configured to: receive a shielding report from the upstream server, the shielding report indicating that the upstream server shielded the downstream server from a plurality of service requests; calculate an aggregate workload estimate for the downstream server based at least in part on one or more shielding reports received by the downstream server, the one or more shielding reports including the shielding report; determine whether the downstream server is in a hotspot situation, said determining being dependent at least in part on the aggregate workload estimate; and provide an indication to the upstream server of whether the downstream server is in a hotspot situation.

33. A system comprising one or more computers implementing a downstream server and one or more separate computer systems implementing one or more upstream servers; wherein the downstream server is configured to: determine that the downstream server is overloaded and in response, to indicate to one or more of the upstream servers that the downstream server is in a hotspot situation; wherein the one or more upstream servers are configured to respond to receiving the indication of the hotspot situation by shielding the downstream server from subsequent requests, said shielding comprises: serving one or more client requests without requesting service from the downstream server; and reporting one or more measures of the shielded requests to the downstream server; wherein the downstream server is further configured to determine whether the hotspot situation still exists, said determining being dependent on one or more of the one or more reported measures.

34. The system of claim 33 wherein the one or more upstream servers correspond to web servers in a web tier of a distributed application.

35. The system of claim 33 , wherein the downstream server and the one or more upstream servers correspond to components of a distributed storage service.

36. The system of claim 33 , wherein serving the one or more client requests without requesting service from the downstream server comprises serving the one or more client requests using a locally cached version of service data required by the one or more client requests.