Noisy Neighbor Throttling

Multitenant systems share resources between two or more tenants (in which each tenant has one or more clients, or users). Because tenants use the same shared resources, the activity of one tenant may negatively impact another tenant's use of the system. A noisy neighbor problem arises when one tenant's performance is degraded because of the activities of another tenant. The noisy neighbor problem also occurs even when each individual tenant may be consuming moderate amounts of the shared system's capacity, but the collective resource usage of many tenants results in a peak in overall usage that exceeds the shared system's capacity.

Common throttling algorithms for client requests include Token Bucket, Leaky Bucket, and Concurrency Limiter. Unfortunately, these have notable limitations, such as poor protection against overflow and denial of service (DoS) attacks, an inability to allocate idle resources, and an inability to focus throttling on the wort offending noisy neighbors. These algorithms require careful cost measurements and precise load estimates to be effective, and when a situation arises that violates the predictions, such an impactful external event or a historically low-demanding customer starting busy workloads, they either cause excessive or insufficient throttling.

For example, with these common throttling algorithms, even if the service is mostly idle and is able to accommodate more requests from a given low volume client, that client may still be throttled at low pre-defined limits, despite resources still being available. Or, if multiple high-volume clients are active simultaneously, outside predictions, the system will be overwhelmed. This may result in all clients being throttled, including well-behaved clients.

The disclosed examples are described in detail below with reference to the accompanying drawing figures listed below. The following summary is provided to illustrate some examples disclosed herein.

Solutions disclosed herein provide for more effective noisy neighbor throttling. Examples identify a trigger point at which to initiate throttling for a server; receive, at the server, requests from a plurality of clients in each time slot of a plurality of time slots; determine, during each time slot of the plurality of time slots, a total request count identifying a count of the requests accepted during the time slot; update a client list with a client request count identifying a count of the requests accepted, from each client of the plurality of clients, during the time slot; and based on at least determining that the total request count meets the trigger point during a first time slot: identify a first client as having a highest client request count of the client list (i.e., identify the first client as a noisy neighbor); and throttle the first client according to an acceptance rate for the first client.

Corresponding reference characters indicate corresponding parts throughout the drawings.

Improved solutions for noisy neighbor throttling (1) ensure that global capacity is not exceeded, (2) allow use of idle resources without a need to specify exceptions, and (3) focus throttling on misbehaving actors (“noise neighbors”) first, while avoiding throttling well-behaved ones. A trigger point is set for initiating throttling for a server, and requests are received from multiple clients. In each timeslot, a total request count and a client-specific request count is determined. The client-specific request counts are sorted in descending order in a list. Efficient single-swap sorting may be used for speed. If the total request count meets the trigger point during a timeslot, the client with the highest request count is readily-identifiable at the top of the sorted list, and that client is marked as noisy and throttled. If, while that client is still being throttled, the trigger is hit again and that client still tops the list, the throttling is enhanced. If, however, a different client is at the top of the list, both clients are subject to throttling.

Aspects of the disclosure solve multiple problems that are necessarily rooted in computer technology, and render use of computing platforms more reliable and efficient in highly common use cases, by providing the practical result of improving noisy neighbor throttling. For example, controlling network access by reducing the impact of noisy neighbors is a practical problem that is necessarily rooted in computer technology, and aspects of the disclosure provide improved in at least the three ways previously noted. This significantly improves the use of computers for network operations, such as by preserving performance for well-behaved clients. These advantageous results are accomplished, at least in part, by updating a client list of clients with a client request count identifying a count of requests accepted, from each client of a plurality of clients, during a time slot; identifying, upon a threshold violation, a first client as having a highest client request count of the client list; marking the first client as noisy; and throttling the first client according to an acceptance rate for the first client.

The various examples will be described in detail with reference to the accompanying drawings. Wherever preferable, the same reference numbers will be used throughout the drawings to refer to the same or like parts. References made throughout this disclosure relating to specific examples and implementations are provided solely for illustrative purposes but, unless indicated to the contrary, are not meant to limit all examples.

1 FIG. 100 102 104 106 140 830 112 102 114 104 116 106 110 112 132 122 102 114 134 124 104 116 136 126 106 illustrates an example architecturethat advantageously provides for improved noisy neighbor throttling. A set of clients, a client, a client, and a client, each transmits requests to a serverover a network. The requests include a requestfrom client, a requestfrom client, and a requestfrom client, and are collectively identified as requests. Requesthas a request taskand a client identifier (ID)that identifies client; requesthas a request taskand a client IDthat identifies client; and requesthas a request taskand a client IDthat identifies client.

132 136 102 106 140 140 Each of request tasks-indicates a task that one of clients-is requesting serverto perform. Although only three clients and three requests are illustrated, it should be understood that a far larger number of clients and requests may by typical. A request is a unit of work that may be accepted or rejected by server, and may be a network request, an HTTP request, and a local operation. Examples of the local operations include a read operation, a write operation, a key value update, and a cryptographic operation (e.g., encrypt or decrypt data). A client is an identifiable source of requests, such as subscriber, a user, an application, a tenant, an internet protocol (IP) address, a machine (physical or virtual), or another computing component. A client ID may be a subscriber ID for a multi-user subscriber, a user ID for a single user, an IP address associated with the request, a combination of any two or more of the subscriber ID, the user ID, and the IP address, or any other suitable identifier that associates a request with a particular client.

140 140 800 140 140 8 FIG. Servermay be a computing component, whether a single device or a cluster of computing resources (physical and/or virtual) that takes requests from clients and decides whether to accept or reject each request. Servermay be, use, or execute as a virtual machine (VM) on a computing device, which is shown in further detail in. If a request is accepted, serverallocates further resources (such as CPU, memory, disk and internal network) to process the request. If a request is rejected, serverpromptly stops working on it, preventing further resource allocation.

140 140 In order to preserve performance and resources for well-behaved clients, serverthrottles traffic (i.e., rejects requests) from misbehaving clients, which are also referred to as noisy neighbors. A noisy neighbor (misbehaving client) is a client that is source of an excessive number of requests to server, or that has comparatively high traffic (in terms of the number of requests) in relation to most other clients. A well-behaved client is one that is not a noisy neighbor, i.e., a client that transmits a small or moderate number of requests (i.e., a reasonable traffic load). Throttling is the rejection of requests, for a specific client, in response to that client transmitting too many requests in a particular unit of time, often measured in time slots that are measured on the order of milliseconds (mS).

Throttling may be implemented using an acceptance rate, which is a measure of the probability of an incoming request from a throttled client being accepted. For example, with an acceptance rate of 60%, six out of every ten incoming requests from a throttled client are accepted, and four out of every ten incoming requests from that throttled client are rejected. Selection, from among the incoming requests from the throttled client, for acceptance or rejection may be probabilistic (i.e., random) or deterministic according to some selection algorithm. Throttling may be specified for some duration, after which it is lifted.

140 142 110 144 144 150 152 154 156 150 300 400 160 144 504 502 504 5 FIG. Serverhas a request handlerfor handling and processing accepted ones of requests, and a throttling logicto handling throttling, when needed. Throttling logichas multiple parameters and data sets, as shown, which include a trigger point, an acceptance rate, a throttling duration, a trigger multiplierby which to increase trigger point, a client list, a throttled client list, and a total request count. Throttling logicmakes throttling decisions within time slots. A time slot, and a current time, which is within time slot, are shown. The use of time slots is described in further detail in relation to.

160 504 160 150 150 140 206 150 150 156 156 156 150 6 FIG. 2 2 FIGS.A andB 6 FIG. a Total request countis the sum of all requests accepted from all clients in the current time slot (e.g., time slot). When total request countmeets trigger point, throttling is applied to the noisiest client, which is identified as described in relation to. Trigger pointmay typically be set to 50% to 80% of the capacity of serverto handle requests without becoming overloaded. This is a server capacity, which is shown in. Trigger pointhas an initial value, but may be increased by applying trigger multiplieras a multiplier during certain periods of exceptionally high traffic, as described in relation to. Typical values for trigger multipliermay range from 1.0 (no increase) to 1.1. When trigger multiplieris 1.05, for example, the increase in trigger pointwill be 5%.

152 152 152 152 152 4 6 FIGS.and Acceptance ratemay have a value of 0.5 to 0.8, with 0.6 or 0.7 being typical. When a client is throttled, the acceptance rate for that client may be initially set to acceptance rate. However, if that client persists in being noisy, the acceptance rate for that persistently noisy client can be reduced, such as by multiplying that client's current acceptance rate by acceptance rateeach time that persistently noisy client is tagged again during the duration of its throttling. When acceptance rateis 0.7, for example, this can result in an applied acceptance rate, for a particular client, of 0.7 initially, then 0.49 (approximately 50%), and then 0.343, and so forth, reducing each time that client is tagged as being persistently noisy during the time it is being throttled. That is, each client will have its own acceptance rate, which may match acceptance rate, or be lower (in some scenarios). This is described in relation to.

154 154 300 400 3 FIG. 4 FIG. A throttling durationdefines the length of time for which throttling is applied to a client, initially, although this may be extended for persistently noisy clients. Throttling durationmay have a typical value of 0.5 and 2.0 times the duration of each time slot. Client listis shown in further detail in, and throttled client listis shown in further detail in. This solution causes that, at any given time, zero or more clients have the noisy neighbor mark. At specific times, the noisy neighbor mark is removed for specific clients, ending throttling and restoring request acceptance for those clients.

2 2 FIGS.A andB 2 FIG.A 2 FIG.B 2 2 FIGS.A andB 200 160 202 204 206 200 150 206 200 202 204 206 150 a a b illustrate a throttling scenario.shows a plotof total request countgraphed against a request count axisas a function of time, with a time axis. Server capacityis shown at the top of plot, with trigger pointat about 80% of server capacity.shows a plot, also with request count axis, time axis, server capacity, and trigger point.should be viewed together.

160 312 314 316 312 112 102 314 114 104 316 116 106 5 FIG. Total request countis the sum of a client request count, a client request count, and a client request count. Client request countis the count of all requests (e.g., request) accepted from clientduring the current time slot (see). Similarly, client request countis the count of all requests (e.g., request) accepted from clientduring the current time slot, and client request countis the count of all requests (e.g., request) accepted from clientduring the current time slot.

200 200 221 160 150 312 314 316 102 104 106 102 312 160 a b 2 FIG.A 2 FIG.B 3 FIG. Five notable events are shown in both plotsand. At a first event, total request countreaches (meets) trigger pointas shown in, and because client request countis the highest (i.e., higher than either client request countor client request count) clientis throttled. See. Clientand clientare both well-behaved clients. How clientis identified is described in relation to. Due to the throttling, client request countand total request countboth drop substantially.

222 160 150 221 102 312 102 102 312 160 223 154 102 312 102 2 FIG.A 2 FIG.B 6 FIG. At a second event, total request countagain reaches trigger pointas shown in, very shortly after event, while clientis still being throttled. Because client request countis again the highest (see), and clientis already being throttled, clientis even further throttled. This enhanced (further) throttling is described in relation to. Due to the throttling, client request countand total request countboth drop substantially. At a third event, throttling durationexpires, and clientis released from throttling. Client request countbegins climbing slowly, but clientis now a well-behaved client.

224 160 150 314 104 314 160 225 160 150 102 312 2 FIG.A 2 FIG.A 2 FIG.B At a fourth event, total request countagain reaches trigger pointas shown in. However, this time, client request countis the highest, so clientis throttled. Due to the throttling, client request countand total request countboth drop substantially. Shortly thereafter, though, at a fifth event, total request countagain reaches trigger pointas shown in. Again, it is client, as shown by client request countbeing the highest in.

102 102 152 222 106 2 2 FIGS.A andB This time, though, although clientis again selected for throttling, because the prior throttling had expired, clientis subject to only the initial throttling rate (i.e., acceptance rate), rather than the enhanced throttling that occurred at event. Throughout the time period shown in, clientremained a well-behaved client, and so was never throttled.

3 FIG. 300 300 300 300 302 102 122 312 304 104 124 314 306 106 126 316 illustrates further detail for client list. Client listhas an entry for active clients, which are clients from which a request was received during the current time slot. Client listis emptied at the beginning of each time slot (other than the initial time slot at start-up). As illustrated, client listhas three entries. An entryfor clienthas client IDand client request count. An entryfor clienthas client IDand client request count. An entryfor clienthas client IDand client request count.

300 310 300 312 314 316 300 102 221 222 225 104 224 2 2 FIGS.A andB Client listis sorted as the various client request counts are updated with incoming requests, so that a highest client request countis at the top of client list. That is, whichever of client request count, client request count, and client request countis the highest value will be at the top of client list. This enables rapid identification of whichever client is noisiest. For example, this scheme permitted rapid identification of clientat events,, and, and rapid identification of clientat event, in.

300 316 314 312 316 300 316 314 314 316 312 300 316 In some examples, client listis not fully or entirely sorted, but instead is only “mostly sorted” using single-swap to enhance speed. In single-swap sorting, only a single, recently-updated value, is swapped with a neighbor. For example, if client request count, which is shown at the bottom of the list is updated such that it exceeds both client request countand client request count, then client request countshould be at the top of client list. However, with a one-stage single-swap sort, client request countis compared with only client request count(the neighbor immediately above) and swapped with client request countbecause client request countis the higher value. In this scenario, client request countremains at the top of client list, and so client list is not yet fully sorted (with client request countat the top).

After one swap iteration, it is possible that an entry is not in its proper (fully sorted) position. So a second iteration (or stage) of the swap may be performed again at the same time, such as up to some fixed number of times. Some examples may use ten or fewer iterations of swaps. If after these swaps the entry is still not in a sorted position, the swap algorithm stops, leaving the entry in the incorrect position. This is by design, for speed. Because the swap runs multiple times, (e.g., with the various incoming requests) each incoming request moves an entry closer to its proper position. The likelihood of an entry being in the incorrect position when a client needs to be selected for throttling is diminishingly small.

300 300 5 FIG. Using a constant number of swap operations per request, the time complexity (CPU cost) of keeping client listsorted is O(1), which enables this approach to be used where high speeds and reduced overhead burden is critical. Additionally, because this scheme has constant time and is fully CPU-bound, a lock may be used (as described in relation to) to maintain synchronization (i.e., prevent a race condition between updating client listand determining the noisiest neighbor), without any operating system (OS) kernel interaction. This further enhances speed.

4 FIG. 2 2 FIGS.A andB 400 400 400 400 402 102 122 452 102 412 102 404 104 124 454 104 414 104 102 104 400 225 illustrates further detail for throttled client list. Throttled client listhas an entry for clients currently subject to throttling. Throttled client listis updated at the beginning of each time slot, by removing entries for clients whose time for throttling has ended (i.e., expired or concluded). As illustrated, throttled client listhas two entries. An entryfor clienthas client ID, an acceptance ratefor client, and an end timefor throttling client. An entryfor clienthas a client ID, an acceptance ratefor client, and an end timefor throttling client. Because entries are shown for both clientand client, this is the configuration of throttled client listthat could be expected immediately after eventof.

452 454 152 102 222 452 152 454 104 2 2 FIGS.A andB If each client had been throttled anew, both acceptance rateand acceptance ratewill equal acceptance rate. However, when clientis subject to enhanced throttling (e.g., such as at eventof), acceptance ratewill be lowered to below its initial value of acceptance rate, and also lower than acceptance ratefor client.

5 FIG. 6 FIG. 6 FIG. 500 504 504 500 502 504 504 504 504 504 504 504 504 504 504 504 504 502 504 d a b e c d d c e d d. illustrates an exemplary timelinewith a time slotand events that may occur during time slot. Timelineshows current timewithin time slot, rendering time slota current time slot. This is also a time slot, which is referenced in the description of. A second time slot, referenced in the description of, is the subsequent time slot, next time slot. A prior time slotpreceded current time slot. The roles of current time slot, prior time slot, and next time slotare relative, and current time slotmoves such that current timeremains within current time slot

506 508 506 504 300 160 150 5 FIG. Each time slot has a start timeand a stop time, and may be one mS to one minute. A time slot may be associated with a timestamp that marks its start time. During a single time slot, using time slotfor explanation, there may be multiple updates to client list, and checks of whether total request countmeets trigger point. In the example of, there are two iterations of each, although it should be understood that some examples may use different numbers of each operation.

504 502 160 312 314 300 160 150 312 316 d 6 FIG. A series of updates and sorts may occur during time slot(current time slot) and the other time slots, interlaced with decisions as to whether to throttle a client, as described below in relation to. These update total request countand also any of client request counts-for which requests are to be allocated. Client listis thus updated and sorted, during each time slot. A trigger point decision checks whether total request countmeets trigger point. A lock prevents state corruption from race conditions, such as interlaced sorting operations or interlaced increments of client request counts-. Various types of locks may be used, such as a spinlock, mutex lock, a thread critical section lock, or another mutual exclusion mechanism for multiprocessor or multithreaded systems. For example, a spinlock is a lock that causes a thread trying to acquire it to simply wait in a loop (“spin”) while repeatedly checking whether the lock is available. Since the thread remains active but is not performing a useful task, the use of such a lock is a form of busy waiting. Because they avoid overhead from OS process rescheduling or context switching, spinlocks are efficient if threads are likely to be blocked for only short periods.

6 FIG. 8 FIG. 600 100 600 800 600 144 144 150 140 152 154 156 144 shows a flowchartillustrating exemplary operations that may be performed by architecture. In some examples, operations described for flowchartare performed by computing deviceof. Flowchartdescribes a process that starts and finishes for throttling logicfor each incoming request. In periods in which no request is received, no state change happens for throttling logic. An earlier setup process had already identified trigger point, at which to initiate throttling for server, and also acceptance rate, throttling duration, and trigger multiplierfor use by throttling logic.

600 112 602 604 502 606 608 610 150 150 156 300 400 502 400 160 312 316 600 504 a a. Flowchartcommences with receiving requestin operation, engaging a lock in operation, and identifying current timein operation. A decision operationdetermines whether the time slot has changed to a new time slot. If so, operationperforms an update/resetting process. Trigger pointreturns to initial value(if it had been altered by trigger multiplier), client listis cleared, throttled client listis updated to remove clients no longer subject to throttling (i.e., current timeis past the end time for a client as identified in throttled client list), and total request countclient request counts-are all reset. Flowchartis described with the time slot being current time slot

612 122 112 400 614 502 412 102 616 452 102 616 618 112 452 Decision operationidentifies the associated client using client IDin requestand determines whether the request has been received from a client that is on throttled client list. If not, then there is no throttling, as shown by box. If so, then based on at least current timenot yet reaching an end time for throttling that client (e.g., end timefor client), operationthrottles that client according to the acceptance rate for that client (e.g., acceptance ratefor client). Operationincludes decision operation, which determines whether the current request (request) is accepted. This may be a probabilistic determination, such as using a random number with acceptance rateto determine acceptance.

618 112 620 112 618 112 102 112 112 312 102 160 112 600 642 If decision operationdetermines to reject request, operationrejects request. Operationmay cause the data packet carrying requestto be immediately dropped, or may cause an HTTP response to clientindicating that requestwas not accepted, or cause an error state to be associated with request. Neither client request countfor client, nor total request countis incremented for request, in this scenario. There is no triggering, and flowchartthen moves to operationto release the lock.

618 112 612 622 112 142 112 140 112 If, however, decision operationdetermines to accept request, or decision operationdetermined that the client was not throttled, then operationaccepts request. This may include sending a signal to request handlerthat requestshould be fulfilled (processed/executed), or set a flag that will be acted upon by server, indicating that requestshould be fulfilled.

624 300 504 312 300 310 504 300 300 300 504 626 160 110 504 312 316 a a a a Operationupdates client listwith the client request count accepted from each client during time slot, which in this specific example is incrementing client request count. This includes sorting at least a portion of client listby descending order of the client request counts, such that a client having highest client request countduring time slotmoves to the top of client list. Some examples use single-swap sorting, which does not entirely sort client list. In some examples, client listcomprises only clients from which a request is received in time slot. Operationdetermines total request countidentifying a count of requestsaccepted during time slot. This may be accomplished by summing all of client request counts-.

628 160 150 504 160 150 160 150 160 150 160 150 160 150 600 642 a Trigger point decision operationdetermines whether total request countmeets trigger pointduring time slot. Total request countmeets trigger pointwhen total request countequals or exceeds trigger point; total request countdoes not meet trigger pointwhen total request countis below trigger point. If total request countdoes not meet trigger point, flowchartmoves to operation.

160 150 504 630 310 300 102 102 504 632 400 634 638 a a If total request countdoes meet trigger pointduring time slot, operationidentifies which client has having highest client request countof client list. In this described example, it is client. Thus, clientcomprises a noisy neighbor or a misbehaving client during time slot. Operationupdates throttled client listusing operations-.

634 400 636 636 412 102 502 154 452 152 400 638 102 640 150 150 150 150 156 642 a Decision operationdetermines whether the client has already been throttled by examining throttled client listand identifying whether the client ID is already listed. If not, normal throttling is applied using operation. Operationsets end timefor throttling clientto current timeplus throttling duration, and sets acceptance rateto acceptance rate. If the client is already on throttled client list, operationextends the end time for throttling the clientand reduces the acceptance rate for that client (enhancing the throttling). Operationincreases trigger pointfrom initial valueof trigger point, such as by multiplying the current value of trigger pointby trigger multiplier. Operationreleases the lock.

7 FIG. 8 FIG. 700 100 700 800 700 702 704 shows a flowchartillustrating exemplary operations that may be performed by architecture. In some examples, operations described for flowchartare performed by computing deviceof. Flowchartcommences with operation, which includes identifying a trigger point at which to initiate throttling for a server. Operationincludes receiving, at the server, requests from a plurality of clients in each time slot of a plurality of time slots.

706 708 710 712 710 712 Operationincludes determining, during each time slot of the plurality of time slots, a total request count identifying a count of the requests accepted during the time slot. Operationincludes updating a client list with a client request count identifying a count of the requests accepted, from each client of the plurality of clients, during the time slot. Operationsandare both based on at least determining that the total request count meets the trigger point during a first time slot. Operationincludes identifying a first client as having a highest client request count of the client list. Operationincludes throttling the first client according to an acceptance rate for the first client.

An example system comprises: a processor; and a computer-readable medium storing instructions that are operative upon execution by the processor to: identify a trigger point at which to initiate throttling for a server; receive, at the server, requests from a plurality of clients in each time slot of a plurality of time slots; determine, during each time slot of the plurality of time slots, a total request count identifying a count of the requests accepted during the time slot; update a client list with a client request count identifying a count of the requests accepted, from each client of the plurality of clients, during the time slot; and based on at least determining that the total request count meets the trigger point during a first time slot: identify a first client as having a highest client request count of the client list; and throttle the first client according to an acceptance rate for the first client.

An example computer-implemented method comprises: identifying a trigger point at which to initiate throttling for a server; receiving, at the server, requests from a plurality of clients in each time slot of a plurality of time slots; determining, during each time slot of the plurality of time slots, a total request count identifying a count of the requests accepted during the time slot; updating a client list with a client request count identifying a count of the requests accepted, from each client of the plurality of clients, during the time slot; and based on at least determining that the total request count meets the trigger point during a first time slot: identifying a first client as having a highest client request count of the client list; and throttling the first client according to an acceptance rate for the first client.

One or more example computer storage devices have computer-executable instructions stored thereon, which, on execution by a computer, cause the computer to perform operations comprising: identifying a trigger point at which to initiate throttling for a server; receiving, at the server, requests from a plurality of clients in each time slot of a plurality of time slots; determining, during each time slot of the plurality of time slots, a total request count identifying a count of the requests accepted during the time slot; updating a client list with a client request count identifying a count of the requests accepted, from each client of the plurality of clients, during the time slot; and based on at least determining that the total request count meets the trigger point during a first time slot: identifying a first client as having a highest client request count of the client list; and throttling the first client according to an acceptance rate for the first client.

updating a throttled client list identifying each client being throttled, an acceptance rate for the client being throttled, and an end time for throttling the client being throttled; based on at least a current time not yet reaching the end time for throttling the first client, continuing to throttle the first client; based on at least the current time being after the end time for throttling the first client, ceasing to throttle the first client; determining that the total request again meets the trigger point; identifying the first client as again having the highest client request count of the client list; based on at least the current time not yet reaching the end time for throttling the first client, extending the end time for throttling the first client; based on at least the current time not yet reaching the end time for throttling the first client, reducing the acceptance rate for the first client; extending the end time for throttling the first client comprises resetting the end time to the current time plus the throttling duration; reducing the acceptance rate for the first client comprises multiplying the current acceptance rate for the first client by an initial value of the acceptance rate; based on at least the start time for a time slot being after the end time for throttling a client, removing the client from the throttled client list; identifying the trigger point comprises determining the trigger point based on a server capacity for accepting requests in a time slot; the trigger point is set at a value of 50% to 80% of the server capacity; the end time for throttling the first client is set to the current time plus a throttling duration; the throttling duration ranges between 0.5 and 2.0 times a duration of each time slot; the acceptance rate for the first client ranges between 0.5 and 0.8; based on at least determining that the total request count meets the trigger point, increasing the trigger point from an initial value of the trigger point; based on at least the first time slot ending, returning the trigger point to the initial value, clearing the client list, purging the throttled client list of clients no longer subject to throttling, resetting the total request count, and resetting the client request count for each client; based on at least the first time slot ending, clearing the client list; each request comprises a client ID and a request task; the client identifier comprises an ID selected from the list consisting of: a subscriber ID for a multi-user subscriber, a user ID for a single user, an IP address associated with the request, and a combination of any two or more of the subscriber ID, the user ID, and the IP address; the request task comprises a task selected from the list consisting of: a network request, an HTTP request, and a local operation; updating the client list comprises sorting at least a portion of the client list by descending order of the client request counts such that a client having a highest client request count during the time slot moves to a top of the client list; the sorting comprises single-swap sorting; the server comprises a cluster of computing resources; the local operation comprises a task selected from the list consisting of: a read operation, a write operation, a key value update, and a cryptographic operation; each time slot comprises 1.0 mS to 1.0 minute; based on at least receiving a request, identifying the client; based on at least accepting a request, incrementing the client request count for the identified client; identifying the client request count as a count of the requests accepted from the client during the current time slot; updating the client list does not require entirely sorting the client list; the client list comprises only clients from which a request is received in the current time slot; determining, during the first time slot, that the total request count meets the trigger point; the total request count meets the trigger point when the total request count equals or exceeds the trigger point; the total request count does not meet the trigger point when the total request count is below the trigger point; the first client comprises a noisy neighbor or a misbehaving client during the first time slot; the first client comprises a well-behaved client, not a noisy neighbor or a misbehaving client, prior to the first time slot; the throttled client list identifies each client being throttled using the client ID or another identifier uniquely associated with the client ID; the trigger point is increased within a single time slot by a trigger multiplier of 1.0 to 1.1; a lock prevents identifying a client as having a highest client request count using the client request count from a prior time slot. Alternatively, or in addition to the other examples described herein, examples include any combination of the following:

While the aspects of the disclosure have been described in terms of various examples with their associated operations, a person skilled in the art would appreciate that a combination of operations from any number of different examples is also within scope of the aspects of the disclosure.

8 FIG. 800 800 800 800 800 is a block diagram of an example computing device(e.g., a computer storage device) for implementing aspects disclosed herein, and is designated generally as computing device. In some examples, one or more computing devicesare provided for an on-premises computing solution. In some examples, one or more computing devicesare provided as a cloud computing solution. In some examples, a combination of on-premises and cloud computing solutions are used. Computing deviceis but one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the examples disclosed herein, whether used singly or as part of a larger set.

800 Neither should computing devicebe interpreted as having any dependency or requirement relating to any one or combination of components/modules illustrated. The examples disclosed herein may be described in the general context of computer code or machine-useable instructions, including computer-executable instructions such as program components, being executed by a computer or other machine, such as a personal data assistant or other handheld device. Generally, program components including routines, programs, objects, components, data structures, and the like, refer to code that performs particular tasks, or implement particular abstract data types. The disclosed examples may be practiced in a variety of system configurations, including personal computers, laptops, smart phones, mobile tablets, hand-held devices, consumer electronics, specialty computing devices, etc. The disclosed examples may also be practiced in distributed computing environments when tasks are performed by remote-processing devices that are linked through a communications network.

800 810 812 814 816 818 820 822 824 800 800 812 814 Computing deviceincludes a busthat directly or indirectly couples the following devices: computer storage memory(i.e., a computer-readable medium), one or more processors, one or more presentation components, input/output (I/O) ports, I/O components, a power supply, and a network component. While computing deviceis depicted as a seemingly single device, multiple computing devicesmay work together and share the depicted device resources. For example, memorymay be distributed across multiple devices, and processor(s)may be housed with different devices.

810 812 800 812 812 812 812 814 800 812 8 FIG. 8 FIG. a b b Busrepresents what may be one or more buses (such as an address bus, data bus, or a combination thereof). Although the various blocks ofare shown with lines for the sake of clarity, delineating various components may be accomplished with alternative representations. For example, a presentation component such as a display device is an I/O component in some examples, and some examples of processors have their own memory. Distinction is not made between such categories as “workstation,” “server,” “laptop,” “hand-held device,” etc., as all are contemplated within the scope ofand the references herein to a “computing device.” Memorymay take the form of the computer storage media referenced below and operatively provide storage of computer-readable instructions, data structures, program modules and other data for the computing device. In some examples, memorystores one or more of an operating system, a universal application platform, or other program modules and program data. Memoryis thus able to store and access dataand instructionsthat are executable by processorand configured to carry out the various operations disclosed herein. Thus, computing devicecomprises a computer storage device having computer-executable instructionsstored thereon.

812 812 800 812 800 800 812 800 800 812 8 FIG. In some examples, memoryincludes computer storage media. Memorymay include any quantity of memory associated with or accessible by the computing device. Memorymay be internal to the computing device(as shown in), external to the computing device(not shown), or both (not shown). Additionally, or alternatively, the memorymay be distributed across multiple computing devices, for example, in a virtualized environment in which instruction processing is carried out on multiple computing devices. For the purposes of this disclosure, “computer storage media,” “computer storage memory,” “memory,” and “memory devices” are synonymous terms for the memory, and none of these terms include carrier waves or propagating signaling.

814 812 820 814 800 800 814 814 800 800 816 800 818 800 820 820 Processor(s)may include any quantity of processing units that read data from various entities, such as memoryor I/O components. Specifically, processor(s)are programmed to execute computer-executable instructions for implementing aspects of the disclosure. The instructions may be performed by the processor, by multiple processors within the computing device, or by a processor external to the client computing device. In some examples, the processor(s)are programmed to execute instructions such as those illustrated in the flow charts discussed below and depicted in the accompanying drawings. Moreover, in some examples, the processor(s)represents an implementation of analog techniques to perform the operations described herein. For example, the operations may be performed by an analog client computing deviceand/or a digital client computing device. Presentation component(s)present data indications to a user or other device. Exemplary presentation components include a display device, speaker, printing component, vibrating component, etc. One skilled in the art will understand and appreciate that computer data may be presented in a number of ways, such as visually in a graphical user interface (GUI), audibly through speakers, wirelessly between computing devices, across a wired connection, or in other ways. I/O portsallow computing deviceto be logically coupled to other devices including I/O components, some of which may be built in. Example I/O componentsinclude, for example but without limitation, a microphone, joystick, game pad, satellite dish, scanner, printer, wireless device, etc.

800 824 824 800 824 824 826 826 828 830 826 826 a a Computing devicemay operate in a networked environment via the network componentusing logical connections to one or more remote computers. In some examples, the network componentincludes a network interface card and/or computer-executable instructions (e.g., a driver) for operating the network interface card. Communication between the computing deviceand other devices may occur using any protocol or mechanism over any wired or wireless connection. In some examples, network componentis operable to communicate data over public, private, or hybrid (public and private) using a transfer protocol, between devices wirelessly using short range communication technologies (e.g., near-field communication (NFC), Bluetooth™ branded communications, or the like), or a combination thereof. Network componentcommunicates over wireless communication linkand/or a wired communication linkto a remote resource(e.g., a cloud resource) across a computer network. Various different examples of communication linksandinclude a wireless connection, a wired connection, and/or a dedicated link, and in some examples, at least a portion is routed through the internet.

800 Although described in connection with an example computing device, examples of the disclosure are capable of implementation with numerous other general-purpose or special-purpose computing system environments, configurations, or devices. Examples of well-known computing systems, environments, and/or configurations that may be suitable for use with aspects of the disclosure include, but are not limited to, smart phones, mobile tablets, mobile computing devices, personal computers, server computers, hand-held or laptop devices, multiprocessor systems, gaming consoles, microprocessor-based systems, set top boxes, programmable consumer electronics, mobile telephones, mobile computing and/or communication devices in wearable or accessory form factors (e.g., watches, glasses, headsets, or earphones), network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, virtual reality (VR) devices, augmented reality (AR) devices, mixed reality devices, holographic device, and the like. Such systems or devices may accept input from the user in any way, including from input devices such as a keyboard or pointing device, via gesture input, proximity input (such as by hovering), and/or via voice input.

Examples of the disclosure may be described in the general context of computer-executable instructions, such as program modules, executed by one or more computers or other devices in software, firmware, hardware, or a combination thereof. The computer-executable instructions may be organized into one or more computer-executable components or modules. Generally, program modules include, but are not limited to, routines, programs, objects, components, and data structures that perform particular tasks or implement particular abstract data types. Aspects of the disclosure may be implemented with any number and organization of such components or modules. For example, aspects of the disclosure are not limited to the specific computer-executable instructions, or the specific components or modules illustrated in the figures and described herein. Other examples of the disclosure may include different computer-executable instructions or components having more or less functionality than illustrated and described herein. In examples involving a general-purpose computer, aspects of the disclosure transform the general-purpose computer into a special-purpose computing device when configured to execute the instructions described herein.

By way of example and not limitation, computer readable media comprise computer storage media and communication media. Computer storage media include volatile and nonvolatile, removable and non-removable memory implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules, or the like. Computer storage media are tangible and mutually exclusive to communication media. Computer storage media are implemented in hardware and exclude carrier waves and propagated signals. Computer storage media for purposes of this disclosure are not signals per se. Exemplary computer storage media include hard disks, flash drives, solid-state memory, phase change random-access memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that may be used to store information for access by a computing device. In contrast, communication media typically embody computer readable instructions, data structures, program modules, or the like in a modulated data signal such as a carrier wave or other transport mechanism and include any information delivery media.

The order of execution or performance of the operations in examples of the disclosure illustrated and described herein is not essential, and may be performed in different sequential manners in various examples. For example, it is contemplated that executing or performing a particular operation before, contemporaneously with, or after another operation is within the scope of aspects of the disclosure. When introducing elements of aspects of the disclosure or the examples thereof, the articles “a,” “an,” “the,” and “said” are intended to mean that there are one or more of the elements. The terms “comprising,” “including,” and “having” are intended to be inclusive and mean that there may be additional elements other than the listed elements. The term “exemplary” is intended to mean “an example of.” The phrase “one or more of the following: A, B, and C” means “at least one of A and/or at least one of B and/or at least one of C.”

Having described aspects of the disclosure in detail, it will be apparent that modifications and variations are possible without departing from the scope of aspects of the disclosure as defined in the appended claims. As various changes could be made in the above constructions, products, and methods without departing from the scope of aspects of the disclosure, it is intended that all matter contained in the above description and shown in the accompanying drawings shall be interpreted as illustrative and not in a limiting sense.