Load Controller Framework

1. A computer-implemented method executed by one or more processors, the method performed at a load controller of a cloud-based network system, the method comprising: receiving a first request for a network service from a client, wherein the load controller is separate from the network service; evaluating a load condition associated with the network service, the load condition indicating an availability of the network service to receive requests, based at least in part on a determination of whether a flow control indication is received by the load controller from the network service indicating that the network service is currently unavailable to receive the requests due to load; returning a unique token associated with the first request to the client in response to the load condition indicating that the network service is not available to receive the requests, wherein the unique token is associated with a unique identifier and wherein an entry to a token database at the load controller associated with the unique token is entered, the entry at the token database including a set of statistics associated with the token including a number of times the unique token has been submitted to the load controller; after returning the unique token to the client, receiving a second request for the network service from the client, the second request including at least a portion of the first request and the unique token; re-evaluating the load condition associated with the network service in response to receiving the second request; prioritizing the second request based on the unique token in response to the load condition indicating that the network service is available to receive the requests, wherein prioritizing the second request includes placing the second request into a queue with a plurality of other requests; and in response to receiving a flow control indication after prioritizing the second request and prior to fulfilling the second request, removing at least the second request from the queue and returning the unique token to the client.

2. The method of claim 1 , where prioritizing the second request based on the unique token includes sending the second request to the network service before a third request that was received prior to the second request.

3. The method of claim 1 , wherein evaluating the load condition associated with the network service further comprises: analyzing current statistics associated with the availability of the network service; and evaluating the statistics based at least in part on one or more rules, the one or more rules including thresholds associated with the availability of the network service.

4. The method of claim 3 , wherein the one or more rules each include one or more actions to be taken when the statistics associated with the availability of the network service indicate that the network service is not available.

5. The method of claim 4 , wherein the one or more actions include at least one of: reducing a rate that requests are sent to the network service for a period of time or blocking access to the network service for a period of time.

6. The method of claim 3 , wherein the thresholds each include a time value indicating a period of time that the threshold is to be used in evaluating the statistics.

7. The method of claim 3 , wherein the current statistics associated with the availability of the network service include at least one of a number of requests sent to the network service in a recent time period, a total amount of data sent to the network service in the recent time period, and a total number of outstanding requests that the network service is currently processing.

8. The method of claim 1 , wherein evaluating the load condition associated with the network service further comprises receiving an indication from the network service indicating the availability of the network service to receive requests.

9. The method of claim 1 , wherein the network service is a network service in the cloud-based network system.

10. The method of claim 1 , wherein the network service is a customer-premise network service separate from the cloud-based network system.

11. A computer program product encoded on a tangible, non-transitory storage medium, the product comprising computer readable instructions for causing one or more processors to perform operations comprising: receiving, at a load controller of a cloud-based network system, a first request for a network service from a client, wherein the load controller is separate from the network service; evaluating a load condition associated with the network service, the load condition indicating an availability of the network service to receive requests, based at least in part on a determination of whether a flow control indication is received by the load controller from the network service indicating that the network service is currently unavailable to receive the requests due to load; returning a unique token associated with the first request to the client in response to the load condition indicating that the network service is currently unavailable to receive the requests, wherein the unique token is associated with a unique identifier and wherein an entry to a token database at the load controller associated with the unique token is entered, the entry at the token database including a set of statistics associated with the token including a number of times the unique token has been submitted to the load controller; after returning the unique token to the client, receiving a second request for the network service from the client, the second request including at least a portion of the first request and the unique token; re-evaluating the load condition associated with the network service in response to receiving the second request; and prioritizing the second request based on the unique token in response to the load condition indicating that the network service is available to receive the requests, wherein prioritizing the second request includes placing the second request into a priority queue, the priority queue separate from a standard queue used for requests received without an included unique token, and wherein the priority queue is prioritized over the standard queue; and in response to receiving a flow control indication after prioritizing the second request and prior to fulfilling the second request, removing at least the second request from the priority queue and returning the unique token to the client.

12. The computer program product of claim 11 , where prioritizing the second request based on the unique token includes sending the second request to the network service before a third request that was received prior to the second request.

13. The computer program product of claim 11 , wherein in response to determining that the network service is currently unavailable to receive the requests due to load, one or more rules are evaluated to determine one or more actions to be taken when the network service is currently unavailable, wherein the one or more actions include at least one of: reducing a rate that requests are sent to the network service for a period of time or blocking access to the network service for a period of time.

14. The computer program product of claim 11 , wherein the network service is a network service in the cloud-based network system.

15. The computer program product of claim 11 , wherein the network service is a customer-premise network service separate from the cloud-based network system.

16. A system, comprising: memory for storing data; and one or more processors operable to perform operations comprising: receiving, at a load controller of a cloud-based network system, a first request for a network service from a client, wherein the load controller is separate from the network service; evaluating a load condition associated with the network service, the load condition indicating an availability of the network service to receive requests, based at least in part on a determination of whether a flow control indication is received by the load controller from the network service indicating that the network service is currently unavailable to receive the requests due to load; returning a unique token associated with the first request to the client in response to the load condition indicating that the network service is not available to receive the requests, wherein the unique token is associated with a unique identifier and wherein an entry to a token database at the load controller associated with the unique token is entered, the entry at the token database including a set of statistics associated with the token including a number of times the unique token has been submitted to the load controller; after returning the unique token to the client, receiving a second request for the network service from the client, the second request including at least a portion of the first request and the unique token; re-evaluating the load condition associated with the network service in response to receiving the second request; prioritizing the second request based on the unique token in response to the load condition indicating that the network service is available to receive the requests, wherein prioritizing the second request includes placing the second request into a priority queue, the priority queue separate from a standard queue used for requests received without an included unique token, and wherein the priority queue is prioritized over the standard queue; and in response to receiving a flow control indication after prioritizing the second request and prior to fulfilling the second request, removing at least the second request from the priority queue and returning the unique token to the client.

17. The system of claim 16 , where prioritizing the second request based on the unique token includes sending the second request to the network service before a third request that was received prior to the second request.

18. The system of claim 16 , further comprising prioritizing the second request among at least one other request received with an included unique token in the priority queue.

19. The system of claim 16 , wherein prioritizing the second request based on the unique token includes accessing the set of statistics associated with the unique token included with the second request and comparing those statistics to statistics associated with at least one other request received with an included unique token, wherein the second request is prioritized over the at least one other request based on a higher number of times the unique token included with the second request has been submitted to the load controller as compared to the number of times the unique token including with the at least one other request has been submitted to the load controller.