A packet transmission scheduler that may temporally smooth packet transmission over paths or connections to destinations by scheduling packets for transmission to destinations during transmit windows, and by limiting the amount of data that is scheduled for transmission to each destination in each transmit window. A transmit window limit and state information may be maintained for each destination and used in scheduling packets for the destination in the transmit windows. The scheduler may dynamically adjust the transmit window limits for the destinations according to performance feedback for the connections, allowing the packet transmission scheduler to determine optimal or near-optimal transmit window limits for connections so that packets can be sent to the destinations as quickly as possible at rates that the respective connections can handle without dropping packets or experiencing other problems such as long round trip times.
Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A device, comprising: one or more processors; and a memory comprising program instructions executable by at least one of the one or more processors to implement a packet transmission scheduler comprising: a packet scheduler module configured to: determine transmit window limits for connections over a network to a plurality of destinations, wherein a transmit window limit for a given connection indicates a maximum number of packets to send via the connection during a time interval; monitor one or more performance metrics of the connections to the plurality of destinations; adjust, in accordance with changes to the one or more performance metrics, a particular transmit window limit on network traffic for a particular destination of the plurality of destinations; and for the particular destination of the plurality of destinations: receive a plurality of packets for the particular destination from one or more sources, wherein the number of packets received for the particular destination is greater than the transmit window limit for the respective connection; and distribute the plurality of packets for the particular destination into two or more buckets corresponding to two or more transmit windows, wherein the transmit windows are time intervals that begin at respective start times, and wherein, to distribute the packets for the particular destination, the number of packets placed into each bucket is less than or equal to the transmit window limit for the respective connection; a packet sender module configured to: during a current transmit window, send packets from a bucket corresponding to the current transmit window to respective destinations via respective connections, wherein a number of packets sent to a destination during the current transmit window is no greater than the transmit window limit for the respective connection; determine that the start time for a next transmit window is reached and all of the packets in the bucket have been sent; and advance the current transmit window to a next bucket in response to said determining.
A device schedules network packets for transmission, smoothing traffic to multiple destinations. It determines a "transmit window limit" for each destination, representing the maximum packets allowed during a time interval. A packet scheduler monitors connection performance metrics and adjusts these limits dynamically. When the number of packets received for a destination exceeds its transmit window limit, the scheduler distributes them into buckets, each representing a transmit window. The number of packets in each bucket never exceeds the limit. A packet sender then transmits packets from the current window's bucket. The sender determines when the window is complete (start time of next window reached and all packets sent) and advances to the next bucket.
2. The device as recited in claim 1 , wherein the packet transmission scheduler is configured to maintain state information for each destination, wherein the state information for a given destination indicates a current bucket into which packets for the given destination are being placed and a count of packets for the given destination that have been placed into the current bucket.
The device, as described previously, maintains state information for each destination. This state information tracks the current bucket (transmit window) to which packets for that destination are being added, and a count of how many packets have already been placed into that current bucket. This allows the device to keep track of where it is in the packet scheduling process for each destination.
3. The device as recited in claim 1 , wherein, to distribute the packets for the particular destination into two or more buckets corresponding to two or more transmit windows, the packet scheduler module is configured to: place packets for the particular destination into a current bucket for the particular destination until the transmit window limit for the particular destination is reached, wherein the current bucket for the particular destination corresponds to a first transmit window; and advance the current bucket for the particular destination to a next bucket corresponding to a second transmit window.
To distribute packets into buckets for different transmit windows, the packet scheduler places packets for a destination into the current bucket until the transmit window limit for that destination is reached. The current bucket corresponds to the current transmit window. Once the limit is hit, the scheduler advances to the next bucket, which corresponds to the next transmit window, and begins placing subsequent packets into that bucket.
4. The device as recited in claim 1 , wherein, to distribute the packets for the particular destination into two or more buckets corresponding to two or more transmit windows, the packet scheduler module is configured to: receive, during the current transmit window, the plurality of packets for the particular destination; and place one or more of the plurality of packets into the bucket corresponding to the current transmit window for transmittal to the particular destination during the current transmit window, where at most the transmit window limit for the particular destination is placed into the bucket; and place remaining packets for the particular destination into one or more upcoming transmit windows.
The packet scheduler receives packets for a destination *during* the current transmit window. It places some of these received packets into the bucket corresponding to the current window, up to the transmit window limit. Any remaining packets that exceed this limit are placed into buckets corresponding to future, upcoming transmit windows for later delivery. This ensures that no more than the window limit is sent during each time interval.
5. The device as recited in claim 1 , wherein each bucket is a linked list of packets for one or more of the plurality of destinations, and wherein the packets in each bucket are transmitted to respective ones of the destinations during respective transmit windows.
Each bucket, used for holding packets for transmission, is implemented as a linked list of packets. These packets can be destined for one or more of the plurality of destinations. The packets within each specific bucket are then transmitted to their respective destinations during the transmit window associated with that bucket.
6. The device as recited in claim 1 , wherein, to adjust the particular transmit window limit on the network traffic for the particular destination of the plurality of destinations, the packet transmission scheduler is configured to: increase or decrease the particular transmit window limit on the network traffic for the particular destination according to said changes to the one or more performance metrics.
The device dynamically adjusts the transmit window limit for a specific destination based on changes in network performance metrics. The packet transmission scheduler increases the transmit window limit if the performance metrics improve (e.g., lower latency, less packet loss). Conversely, it decreases the limit if the metrics degrade, preventing congestion and dropped packets.
7. The device as recited in claim 1 , wherein the packet transmission scheduler is configured to: determine that, for a monitored connection to a given destination of the plurality of destinations, at least one performance metric is below a performance threshold, wherein network traffic to the given destination on the monitored connection is not currently limited according to the transmit window limits; and in response to said determining: determine an initial transmit window limit for the monitored connection; and begin placing packets for the given destination into the buckets corresponding to the transmit windows.
The packet scheduler monitors connections and determines when a performance metric falls below a defined threshold for a connection to a destination. If the connection's traffic isn't currently limited by transmit windows, the system reacts by: determining an initial transmit window limit for that connection; and beginning to place packets for that destination into buckets, thereby enabling traffic shaping for that previously un-limited connection.
8. The device as recited in claim 1 , wherein the plurality of destinations include one or more specific addresses on the network.
The plurality of destinations, for which packets are scheduled and transmitted, can include one or more specific, individual network addresses. This means the system can control traffic to particular servers or endpoints on the network.
9. The device as recited in claim 1 , wherein the plurality of destinations include ranges of addresses on the network.
The plurality of destinations can also include *ranges* of network addresses, not just single addresses. This allows the scheduler to manage traffic to entire subnets or groups of related destinations with a single transmit window limit.
10. The device as recited in claim 1 , wherein the device is a load balancer node among a plurality of load balancer nodes in a distributed load balancer system receiving the plurality of packets, wherein respective load balancer nodes of the plurality of load balancer nodes comprise a respective one or more processors and a respective memory comprising program instructions executable by the least one of the one or more processors to implement the packet transmission scheduler, and wherein the sources are server nodes in the load balancer system.
The device implementing the packet transmission scheduler is a load balancer node within a distributed load balancer system. This load balancer receives packets from server nodes. Each load balancer node contains processors and memory executing the packet transmission scheduler. This distributes the scheduling task across multiple nodes to handle high traffic loads.
11. A method, comprising: determining, by a packet scheduler module implemented on one or more devices on a network, transmit window limits for connections over an external network to a plurality of destinations, wherein a transmit window limit for a given connection indicates a maximum number of packets to send via the connection during a transmit window; obtaining, by the packet scheduler module, packets for the plurality of destinations from one or more sources; distributing, by the packet scheduler module, the packets for the destinations into a plurality of buckets corresponding to a plurality of transmit windows, wherein the transmit windows are time intervals scheduled to begin at respective transmit window start times, wherein distributing the packets for the destinations into the plurality of buckets comprises limiting a number of packets placed into each bucket for each destination according to the transmit window limit for the respective connection; transmitting, by a packet sender module implemented on the one or more devices, packets from a bucket corresponding to a current transmit window to respective destinations via connections over the external network; advancing the current transmit window to a next bucket after completing the current transmit window, wherein the current transmit window is completed when the start time for a next transmit window is reached and all of the packets in the bucket corresponding to the current transmit window have been transmitted; and adjusting, in accordance with changes to performance metrics of a particular connection of the connections to a particular destination of the plurality of destinations, a particular transmit window limit on network traffic for the particular destination of the plurality of destinations.
A method for scheduling network packet transmissions comprises: determining transmit window limits for network connections to multiple destinations; obtaining packets from one or more sources for these destinations; distributing these packets into buckets representing different transmit windows, limiting the number of packets per bucket according to each destination's transmit window limit; transmitting packets from the current window's bucket to destinations; advancing the current transmit window after the start time of the next window is reached and all packets in the bucket are sent; and adjusting a transmit window limit for a destination based on performance metrics changes for that connection.
12. The method as recited in claim 11 , further comprising maintaining state information for each destination, wherein the state information for a given destination indicates a current bucket into which packets for the given destination are being placed and a count of packets for the given destination that have been placed into the current bucket.
In addition to the previous method, the method also maintains state information for each destination. This state information tracks which bucket (transmit window) packets for that destination are currently being added to, and how many packets have already been placed in that current bucket, facilitating accurate packet distribution and scheduling.
13. The method as recited in claim 11 , wherein said distributing the packets for the destinations into a plurality of buckets corresponding to a plurality of transmit windows comprises, for each destination: placing packets for the destination into a current bucket for the destination until the transmit window limit for the destination is reached, wherein the current bucket for the destination corresponds to a first transmit window; and advancing the current bucket for the destination to a next bucket corresponding to a second transmit window.
The method described previously distributes packets into buckets by, for each destination: placing packets into the current bucket until the transmit window limit is reached. After the limit is reached, the current bucket is advanced to the next bucket, corresponding to the next transmit window, and subsequent packets are placed there.
14. The method as recited in claim 11 , wherein said distributing the packets for the destinations into a plurality of buckets corresponding to a plurality of transmit windows comprises, for a given destination: receiving, during the current transmit window, a plurality of packets for the destination; and placing one or more of the plurality of packets into the bucket corresponding to the current transmit window for transmittal to the destination during the current transmit window, where at most the transmit window limit for the destination is placed into the bucket; and placing remaining packets for the given destination into one or more upcoming transmit windows.
For a given destination, the method involves: receiving packets during the current transmit window; placing some of these packets (up to the transmit window limit) into the bucket for the current window for transmission during that window; and placing any remaining packets into buckets corresponding to future transmit windows. This limits the number of packets sent during any given transmit window interval.
15. The method as recited in claim 11 , wherein each bucket is a linked list of packets for one or more of the plurality of destinations, and wherein the packets in each bucket are transmitted to respective ones of the destinations during respective transmit windows.
In the previously described method, each bucket is structured as a linked list of packets destined for one or more of the destinations. These packets are then transmitted to their respective destinations during the time interval of the transmit window assigned to that particular bucket.
16. The method as recited in claim 11 , wherein said adjusting further comprises: increasing or decreasing the particular transmit window limit for the particular destination of the plurality of destinations according to said changes to the performance metrics.
The adjustment of the transmit window limit based on performance metrics in the described method involves either increasing or decreasing the limit. The limit is increased if the metrics indicate improved performance (e.g., less latency), or decreased if they indicate degraded performance (e.g., increased packet loss), for a specific destination.
17. The method as recited in claim 11 , further comprising: determining that, for a monitored connection to a given destination of the one or more destinations, at least one performance metric of the performance metrics is below a performance threshold, wherein network traffic to the given destination on the monitored connection is not currently limited according to the transmit window limits; and in response to determining that the monitored connection is performing below the performance threshold, begin scheduling packets for transmittal to the given destination during the transmit windows.
The method further comprises: detecting a scenario where a performance metric for a connection to a destination falls below a defined threshold, and where traffic to this destination is not currently regulated by transmit window limits. Upon detecting this, the method commences scheduling packets for transmission to that destination during defined transmit windows.
18. A non-transitory computer-readable storage medium storing program instructions that when executed on one or more computers cause the one or more computers to implement a packet transmission scheduler on a network configured to: determine transmit window limits for connections over an external network to a plurality of destinations, wherein a transmit window limit for a given connection indicates a maximum number of packets to send via the connection during a time interval; schedule packets for transmittal to the plurality of destinations during two or more transmit windows according to state information for a plurality of connections to the plurality of destinations, wherein the state information for a given connection indicates a transmit window into which packets for the respective destination are currently being scheduled and a packet count for the respective destination that indicates how many packets for the destination have been scheduled in the indicated transmit window; determine that the packet count for a given destination has reached a packet window limit for the respective connection; and in response to said determining, advance the transmit window for the given destination indicated in the state information to a next transmit window.
A non-transitory computer-readable storage medium stores instructions for a packet transmission scheduler. When executed, the scheduler determines transmit window limits for connections to destinations, schedules packets for transmission during transmit windows based on connection state information, the state information indicating a transmit window being scheduled into and a packet count for packets scheduled in that window. When the packet count for a destination reaches its limit, the scheduler advances the transmit window to the next transmit window.
19. The non-transitory computer-readable storage medium as recited in claim 18 , wherein the packet transmission scheduler is further configured to: send packets scheduled for transmittal during a current transmit window to respective destinations via respective connections over the external network; and advance to a next transmit window upon completing sending of the packets from the current transmit window.
In addition to the functions described previously, the packet transmission scheduler on the storage medium also sends packets scheduled for the current transmit window over the network to their respective destinations. It then advances to the next transmit window upon completing the sending of all the packets from the current transmit window's bucket.
20. The non-transitory computer-readable storage medium as recited in claim 18 , wherein the packet transmission scheduler is further configured to: initialize the transmit window limits for the connections according to one or more characteristics of the connections or of the respective destinations; and monitor one or more performance metrics for each of one or more of the connections; and modify the transmit window limit for at least one connection according to the monitored performance metrics for the at least one connection.
The packet transmission scheduler on the storage medium: initializes transmit window limits based on connection characteristics (e.g., bandwidth, latency) or destination characteristics; monitors performance metrics (e.g. packet loss, latency) for the connections; and dynamically adjusts the transmit window limit for a connection in response to monitored performance metrics, increasing the limit with good performance and decreasing it with poor performance.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
December 5, 2014
April 11, 2017
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.