Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A system for managing network-based content requests comprising: at least a first computing device hosting a virtual compute service providing virtual compute instances; and at least a second computing device hosting a block-based storage service, wherein the block-based storage service: obtains a request for content from a virtual compute instance of the virtual compute instances of the virtual compute service; determines that the content for which the request is obtained satisfies a content size threshold; obtains network information indicative of a measure of incast associated with the virtual compute instance from which the request for content is obtained; determines that the measure of incast satisfies an incast threshold; modifies a network latency range based at least in part on the measure of incast satisfying the incast threshold; selects a response latency from the modified network latency range, wherein: the selected response latency delays transmission of a first response from the block-based storage service, and the first response corresponds to a first portion of the content for which the request is obtained; and transmits the first response to the request for the content in accordance with the selected response latency.
The system manages network-based content requests in cloud computing environments to mitigate incast congestion. Incast occurs when multiple virtual compute instances simultaneously request data from a shared block-based storage service, overwhelming the network and degrading performance. The system includes a virtual compute service hosting virtual compute instances and a block-based storage service that processes content requests. When a request is received, the storage service checks if the requested content exceeds a size threshold and evaluates network incast metrics associated with the requesting instance. If incast exceeds a predefined threshold, the storage service adjusts a network latency range to stagger response transmissions. A response latency is selected from this modified range to delay sending the first portion of the requested content, reducing the likelihood of network congestion. Subsequent portions may be transmitted without delay or with reduced latency. This approach dynamically optimizes data delivery to prevent incast-related performance degradation while maintaining efficient storage access. The system is particularly useful in distributed computing environments where multiple instances frequently access shared storage resources.
2. The system as recited in claim 1 , wherein the block-based storage service modifies the content size threshold based at least in part on the measure of incast.
A distributed block-based storage system manages data storage and retrieval for multiple clients, where data is divided into fixed-size blocks. The system monitors network traffic patterns, particularly incast conditions where multiple clients simultaneously request data from the same storage node, leading to congestion and performance degradation. To mitigate this, the system dynamically adjusts a content size threshold, which determines the maximum size of data blocks that can be transferred in a single operation. By reducing the threshold during high incast conditions, the system limits the amount of data sent per request, reducing network congestion and improving overall performance. The adjustment is based on real-time measurements of incast levels, ensuring adaptive behavior in response to varying network conditions. This approach helps maintain efficient data transfer while preventing bottlenecks in the storage infrastructure. The system may also include mechanisms to distribute data across multiple storage nodes to further balance load and reduce incast-related issues.
3. The system as recited in claim 1 , wherein the block-based storage service further: determines that a second portion of the content for which the request is obtained does not satisfy the content size threshold; and transmits a second response to the request for the content without the selected response latency.
A system for managing content retrieval in a block-based storage service addresses the challenge of optimizing response times for content requests based on size. The storage service stores content in blocks and processes requests for content retrieval. When a request for content is received, the system evaluates whether the requested content meets a predefined size threshold. If the content exceeds this threshold, the system applies a selected response latency to the response, ensuring controlled delivery. However, if the content does not meet the threshold, the system bypasses the latency and transmits the response immediately. This approach optimizes performance by prioritizing smaller content requests while managing larger requests to prevent system overload. The system dynamically adjusts response handling based on content size, improving efficiency and user experience. The block-based storage service further includes mechanisms to store and retrieve content in discrete blocks, ensuring structured data management. The latency selection process may involve predefined rules or dynamic adjustments based on system load or network conditions. This system is particularly useful in environments where varying content sizes require flexible response strategies to maintain performance and reliability.
4. The system as recited in claim 1 , wherein to modify the network latency range based at least in part on the measure of incast indicated in the obtained network information, the block-based storage service: determines that the measure of incast indicated in the obtained network information satisfies the incast threshold; and increases the network latency range.
The system relates to optimizing network performance in block-based storage services, particularly addressing issues related to incast, where multiple data packets are sent to a single destination, causing congestion and increased latency. The system monitors network conditions, including incast levels, to dynamically adjust network latency ranges to mitigate performance degradation. When network information is obtained, the system evaluates the measure of incast against a predefined threshold. If the incast level exceeds this threshold, the system increases the network latency range to reduce congestion and improve data transfer efficiency. This adaptive adjustment helps maintain stable and reliable network performance in distributed storage environments. The system may also include additional components, such as a network monitoring module to collect real-time data and a latency adjustment module to implement changes based on the monitored conditions. The overall approach ensures that storage services can handle high traffic loads without significant latency spikes, enhancing overall system responsiveness.
5. The system as recited in claim 4 , wherein the block-based storage service further: determines that the incast threshold is no longer satisfied; and decreases the modified network latency range.
The system relates to optimizing network performance in block-based storage services, particularly addressing incast congestion where multiple storage nodes simultaneously send data to a requesting client, causing network latency spikes. The system monitors network conditions and dynamically adjusts the latency range to mitigate congestion. When the incast threshold is exceeded, the system increases the latency range to stagger data transmissions, reducing simultaneous traffic bursts. Once the incast condition is resolved, the system decreases the latency range to restore normal network efficiency. The block-based storage service includes a network latency adjustment module that tracks latency metrics and adjusts the latency range based on predefined thresholds. The system ensures balanced data delivery, preventing network overload while maintaining storage performance. This approach is particularly useful in distributed storage environments where synchronized data requests can degrade network performance. The dynamic adjustment mechanism adapts to real-time conditions, optimizing both throughput and latency.
6. The system as recited in claim 4 , wherein the block-based storage service further: determines that the incast threshold is no longer satisfied; and selects a second response latency from the modified network latency range without further modifying the response latency range.
A system for managing network traffic in a block-based storage service addresses the problem of incast congestion, where multiple servers send data to a single client simultaneously, causing network bottlenecks and degraded performance. The system monitors network conditions and dynamically adjusts response latencies to mitigate incast congestion. When the system detects that the incast threshold is exceeded, it modifies the network latency range to reduce the likelihood of congestion. Once the incast threshold is no longer satisfied, the system selects a second response latency from the previously modified network latency range without further adjusting the range. This ensures that the system can dynamically adapt to changing network conditions while maintaining efficient data transfer. The system includes components for monitoring network traffic, adjusting latency parameters, and selecting appropriate response latencies to optimize performance. The solution improves network efficiency and reduces latency-related disruptions in block-based storage environments.
7. The system as recited in claim 1 , wherein the block-based storage service further: obtains updated network information indicative of an updated measure of incast associated with the virtual compute instance from which the request for content is obtained; and exponentially increases the modified network latency range based at least in part on the obtained updated network information indicative of the updated measure of incast.
The system relates to managing network traffic in cloud computing environments, specifically addressing performance issues caused by incast congestion in block-based storage services. Incast occurs when multiple virtual compute instances simultaneously request data from a storage system, overwhelming the network and degrading performance. The system dynamically adjusts network latency to mitigate incast by monitoring network conditions and applying exponential backoff techniques. The system includes a block-based storage service that processes requests for content from virtual compute instances. It obtains network information indicating the measure of incast affecting a requesting instance, such as packet loss or latency spikes. Based on this data, the storage service modifies the network latency range for the instance to reduce the likelihood of incast. If the incast condition worsens, the system obtains updated network information and exponentially increases the modified latency range to further mitigate congestion. This adaptive approach ensures fair resource allocation and prevents network saturation while maintaining service reliability. The system may also include a network latency adjustment module to implement these changes dynamically.
8. The system as recited in claim 1 , wherein the block-based storage service further: obtains updated network information that indicates incast is not associated with the virtual compute instance from which the request for content is obtained; and linearly decreases the modified network latency range based at least in part on the updated network information.
A system for managing network latency in a block-based storage service involves dynamically adjusting latency ranges to optimize performance in distributed computing environments. The system monitors network conditions, including incast events where multiple data packets are sent to a single receiver, which can cause congestion and degrade performance. When incast is detected, the system modifies the network latency range to mitigate its impact. If subsequent network information indicates that incast is no longer present, the system gradually reduces the modified latency range back to its original state. This adaptive approach ensures efficient data retrieval while maintaining system stability. The system may also include features for processing read requests, determining latency ranges, and adjusting these ranges based on network conditions to prevent performance degradation. The dynamic adjustment mechanism helps balance load and reduce latency fluctuations, improving overall system responsiveness.
9. The system as recited in claim 1 , wherein the measure of incast indicated in the obtained network information includes a count of a number of network retransmissions to the virtual compute instance from which the request for content is obtained.
The system monitors network performance in virtualized computing environments to detect and mitigate incast issues, which occur when multiple data packets are sent to a single virtual compute instance simultaneously, overwhelming its processing capacity. The system obtains network information, including metrics related to incast conditions, to assess and address these performance bottlenecks. Specifically, the system measures incast by tracking the number of network retransmissions directed to the virtual compute instance from which a content request originates. Retransmissions are a key indicator of incast, as they often result from packet loss or congestion caused by excessive traffic. By analyzing retransmission counts, the system identifies incast events and can implement corrective actions, such as load balancing or traffic throttling, to maintain stable network performance. The system integrates with virtualized infrastructure to dynamically adjust network configurations based on real-time data, ensuring efficient resource utilization and minimizing disruptions. This approach helps prevent performance degradation in cloud and data center environments where virtual compute instances handle high volumes of concurrent requests.
10. The system as recited in claim 1 , wherein the measure of incast indicated in the obtained network information includes a queue length from a network component associated with the virtual compute instance from which the request for content is obtained.
A system monitors network performance in virtualized computing environments to detect and mitigate incast conditions, which occur when multiple data packets are sent to a single destination, overwhelming the receiver. The system obtains network information, including metrics that indicate incast, such as queue lengths from network components associated with virtual compute instances. By analyzing these metrics, the system identifies incast events, where excessive traffic congestion leads to packet loss or degraded performance. The system then adjusts network parameters, such as traffic shaping or load balancing, to reduce the impact of incast. The solution improves reliability and efficiency in data centers and cloud environments by dynamically responding to network congestion caused by high-volume, synchronized data flows. The system may also correlate incast metrics with other network performance indicators, such as latency or throughput, to provide a comprehensive view of network health. This approach helps prevent service disruptions and ensures smooth operation in distributed computing environments.
11. A computer-implemented method comprising: obtaining, at a storage server in a block-based storage service, a request for content from a compute instance, wherein the storage server hosts at least a portion of a storage volume; determining, by the storage server, that the content for which the request is obtained satisfies a content size threshold; obtaining, by the storage server, network information indicative of a measure of incast associated with the compute instance from which the request for content is obtained; determining, by the storage server, that the measure of incast satisfies an incast threshold; increasing, by the storage server, a response latency range; selecting, by the storage server, a response latency from the increased response latency range, wherein the selected response latency delays transmission of a first response from the storage server, and wherein the first response is associated with a first portion of the content for which the request is obtained; and transmitting, by the storage server, the first response to the request for content in accordance with the selected response latency.
This invention relates to a block-based storage service that mitigates incast congestion in distributed computing environments. The problem addressed is the performance degradation caused by incast, where multiple compute instances simultaneously request large amounts of data from a storage server, leading to network congestion and reduced throughput. The method involves a storage server receiving a request for content from a compute instance. If the requested content meets a predefined size threshold, the server evaluates network conditions by obtaining incast metrics associated with the requesting compute instance. If the incast level exceeds a specified threshold, the server increases the response latency range to stagger data transmission. A response latency is then selected from this expanded range, delaying the transmission of the first portion of the requested content. This delayed response helps distribute network traffic over time, reducing incast-related congestion. Subsequent portions of the content may be transmitted with adjusted latencies to further mitigate congestion. The method dynamically adapts to network conditions to improve overall system performance and reliability.
12. The computer-implemented method as recited in claim 11 further comprising: determining, by the storage server, that a second portion of the content for which the request is obtained does not satisfy the content size threshold; and transmitting, by the storage server, the second response to the request for content without the selected response latency.
The invention relates to optimizing content delivery in a storage system by dynamically adjusting response latency based on content size. The problem addressed is inefficient resource utilization when delivering content of varying sizes, where smaller content may not require the same latency optimizations as larger files. The method involves a storage server receiving a request for content and determining whether the requested content meets a predefined size threshold. If the content exceeds the threshold, the server selects a response latency to optimize delivery, such as reducing latency for large files to improve throughput. For content that does not meet the threshold, the server transmits the response without applying the selected latency, avoiding unnecessary processing overhead. This approach ensures efficient resource allocation by tailoring latency adjustments to content size, improving overall system performance. The method may also involve analyzing historical data to refine the size threshold and latency selection criteria, ensuring adaptive optimization over time. The system dynamically balances speed and resource usage, particularly in distributed storage environments where latency management is critical.
13. The computer-implemented method as recited in claim 11 further comprising: determining, by the storage server, that the incast threshold is no longer satisfied; and selecting, by the storage server, an updated response latency from the increased response latency range.
This invention relates to managing network traffic in storage systems, particularly addressing incast congestion where multiple clients request data simultaneously from a storage server, overwhelming the server's capacity and degrading performance. The method involves dynamically adjusting response latencies to mitigate incast congestion. Initially, the storage server detects that an incast threshold is satisfied, indicating excessive concurrent requests. In response, the server selects an increased response latency from a predefined range to stagger client requests, reducing the load on the server. Once the incast condition subsides, the server determines that the incast threshold is no longer satisfied and selects an updated response latency from the increased latency range, allowing the system to return to normal operation. The method ensures efficient data retrieval while preventing network congestion and performance degradation. The latency adjustments are automated, enabling real-time adaptation to varying network conditions. This approach is particularly useful in distributed storage environments where multiple clients access shared resources, ensuring stable and reliable data access. The invention improves system scalability and responsiveness by dynamically balancing load distribution.
14. The computer-implemented method as recited in claim 11 further comprising: determining, by the storage server, that updated network information indicative of an updated measure of incast associated with the compute instance from which the request for content is obtained indicates that the incast threshold is no longer satisfied; and decreasing, by the storage server, the increased response latency range.
The invention relates to managing network traffic and response latency in distributed storage systems, particularly addressing the problem of incast congestion where multiple compute instances request data simultaneously from a storage server, leading to network bottlenecks and degraded performance. The method involves monitoring network conditions, including incast levels, to dynamically adjust response latency ranges for data requests. When incast exceeds a predefined threshold, the storage server increases the response latency range to reduce the likelihood of congestion by staggering data transmissions. The system continuously evaluates network information, such as incast metrics, to determine when conditions improve. If the incast level falls below the threshold, the storage server decreases the response latency range, restoring normal or optimized data delivery timing. This adaptive approach ensures efficient resource utilization while maintaining performance under varying network loads. The method may also involve prioritizing requests based on urgency or other criteria to further mitigate congestion. The solution is particularly useful in cloud computing environments where multiple virtual machines or containers access shared storage resources.
15. The computer-implemented method as recited in claim 11 , wherein the obtained network information indicative of the measure of incast associated with the compute instance from which the request for content is obtained includes a count of a number of network retransmissions to the compute instance.
The invention relates to network performance monitoring in distributed computing environments, specifically addressing the problem of incast congestion, where multiple data packets are sent to a single compute instance, overwhelming its network interface and causing performance degradation. The method involves obtaining network information that quantifies incast-related issues, including a count of network retransmissions to the compute instance from which a content request originates. This retransmission count serves as an indicator of network congestion or packet loss, helping to diagnose and mitigate incast conditions. The method may also involve analyzing additional network metrics, such as packet loss rates or latency, to further assess the severity of incast. By monitoring these retransmissions, the system can dynamically adjust network traffic distribution, throttle requests, or implement load-balancing strategies to prevent or alleviate incast congestion. The approach is particularly useful in data centers, cloud computing, or high-performance computing environments where multiple nodes may simultaneously send data to a single destination, leading to network bottlenecks. The retransmission count provides a real-time, quantifiable measure of incast impact, enabling proactive network management and improved system reliability.
16. A non-transitory computer readable-medium including computer-executable instructions that, when executed by a system having at least one processor and memory, cause the system to at least: obtain a request for content from a computing device; determine a network characteristic associated with communication throughput to the computing device from which the request for content is obtained; modify a network latency range based at least in part on a measure of incast associated with the computing device from which the request for content is obtained, wherein the measure of incast is determined using a machine learning algorithm that is provided the determined network characteristic as input; select a response latency from the modified response latency range; and transmit a response to the request for content in accordance with the selected response latency.
This invention relates to optimizing content delivery over networks by dynamically adjusting response latency based on network conditions and incast behavior. The system obtains a content request from a computing device and evaluates network characteristics affecting communication throughput to that device. A machine learning algorithm analyzes these characteristics to determine a measure of incast (simultaneous data transmission from multiple sources to a single receiver) associated with the requesting device. The system then modifies a predefined network latency range using this incast measure and selects a specific response latency within this adjusted range. The content response is transmitted according to this selected latency, improving network efficiency and performance. The machine learning algorithm continuously refines its predictions by processing network characteristic inputs, enabling adaptive latency adjustments. This approach helps prevent network congestion and ensures timely content delivery by accounting for real-time network conditions and potential incast scenarios. The system operates on a computer-readable medium with executable instructions for a processor-based system, dynamically managing response times to optimize network performance.
17. The non-transitory computer-readable medium as recited in claim 16 , including further computer-executable instructions that, when executed by the system having at least one processor, cause the system to at least: determine that the content for which the request is obtained satisfies a content size threshold.
A system and method for managing digital content distribution involves a computer-readable medium storing executable instructions that, when run by a processor, perform operations to optimize content delivery. The system receives a request for content and evaluates whether the requested content meets a predefined size threshold. If the content exceeds this threshold, the system processes it differently than smaller content, such as by compressing, segmenting, or prioritizing delivery. The system may also assess network conditions, user device capabilities, or other factors to determine the most efficient way to transmit the content. This approach ensures that large files are handled in a manner that reduces latency, conserves bandwidth, and improves user experience. The system may further adjust delivery strategies based on real-time feedback or historical data to dynamically optimize performance. The invention is particularly useful in environments where large media files, software updates, or other substantial data transfers are common, ensuring efficient and reliable distribution.
18. The non-transitory computer-readable medium as recited in claim 16 , wherein the network latency range is modified based at least in part on an incast threshold.
A system and method for optimizing network performance in data transmission involves dynamically adjusting network latency ranges to prevent incast congestion. Incast occurs when multiple data packets are sent simultaneously to a single receiver, overwhelming the receiver and causing packet loss or delays. The system monitors network traffic to detect incast conditions, where the number of concurrent transmissions exceeds a predefined incast threshold. When such conditions are detected, the system modifies the network latency range to reduce the likelihood of incast by staggering packet transmissions. This adjustment may involve increasing the minimum latency or decreasing the maximum latency to spread out packet arrivals over time. The system may also incorporate additional factors, such as network load, bandwidth availability, or historical traffic patterns, to fine-tune the latency range. By dynamically adjusting latency, the system ensures efficient data transfer while minimizing congestion and improving overall network reliability. The method is implemented via a non-transitory computer-readable medium containing executable instructions for performing these operations.
19. The non-transitory computer-readable medium as recited in claim 16 , wherein the network characteristic associated with the communication throughput to the computing device from which the request for content is obtained, is based at least in part on at least one of a network characteristic measured by the system or an measure of a network characteristic obtained from another source.
This invention relates to optimizing content delivery in a networked computing environment by dynamically adjusting content based on network characteristics. The problem addressed is inefficient content delivery due to varying network conditions, which can degrade user experience. The system measures or obtains network characteristics, such as communication throughput, to determine the optimal content to deliver to a requesting computing device. These characteristics may be directly measured by the system or sourced from external data. The system then selects or modifies content based on these characteristics to ensure efficient and effective delivery. For example, if the network throughput is low, the system may deliver a lower-resolution version of the content or a compressed format to maintain performance. The invention improves content delivery by dynamically adapting to real-time network conditions, enhancing user experience and resource utilization. The system may also integrate network data from multiple sources, including direct measurements and third-party data, to refine its content selection process. This approach ensures that content delivery is optimized regardless of the network environment, whether it is a high-speed connection or a constrained bandwidth scenario.
20. The non-transitory computer-readable medium as recited in claim 16 , wherein the network characteristic associated with the communication throughput to the requesting computing device from which the request for content is received, is based at least in part on a weighted moving average of network characteristics over a defined time period.
This invention relates to optimizing content delivery over a network by dynamically adjusting communication throughput based on network performance metrics. The problem addressed is inefficient content delivery due to fluctuating network conditions, which can lead to slow load times, buffering, or poor user experience. The solution involves analyzing network characteristics associated with communication throughput to a requesting computing device and using this data to improve content delivery decisions. The system monitors network characteristics over time and calculates a weighted moving average of these characteristics over a defined time period. This weighted moving average provides a more accurate representation of recent network performance trends, allowing the system to adapt to changing conditions. The network characteristics may include metrics such as latency, bandwidth, packet loss, or other factors that influence communication throughput. By incorporating this weighted moving average, the system can make more informed decisions about how to deliver content, such as selecting optimal servers, adjusting data transfer rates, or prioritizing certain types of content. This approach ensures that content delivery is optimized for the current network state, improving efficiency and user experience. The use of a weighted moving average helps smooth out short-term fluctuations while still responding to longer-term trends, making the system more robust and adaptive.
Unknown
March 3, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.