Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A system for generating traffic statistics for a datacenter comprising: an end host in a datacenter, the end host comprising a memory, a processor, and a Network Interface Controller (NIC) to execute: summarizing traffic data at a level of a portion of a traffic flow between a computing node at the end host and a second computing node in the datacenter, and bundling the traffic data for the portion with source information for the traffic flow, and reporting the traffic data with source information to an analytic controller; and the analytic controller hosted by a computer in the datacenter comprising memory, a processor, and an NIC to: aggregating traffic data with bundled source information from multiple flow agents hosted by multiple end hosts in the datacenter to obtain aggregated traffic data; and retrieving, from a look up table, a list of links and nodes between the end host and the second computing node, the list of links and nodes between the end host and the second computing node corresponding to an enforced path between the end host and the second computing node, the list of links and nodes between the end host and the second computing node including two or more intermediate nodes; bundling the list of links and nodes between the end host and the second computing node with the aggregated traffic data; and provisioning the aggregated traffic data for traffic-statistic-producing parallel processing; maintaining a counter operable to indicate when a grouping time period has expired; combining traffic data for multiple traffic flows with a common source and destination within the grouping time period into a common tuple-traffic group, the common tuple-traffic group describable by common parameter values for the common source and destination within the source information used to transmit traffic flows belonging to the common tuple-traffic group, and combined traffic data summarizing the traffic scheduled for the common tuple-traffic group and comprising at least one of the traffic data summarizing the portion of the traffic flows and additional traffic data summarizing a second portion of a second traffic flow along the common tuple-traffic group; and generating a pre-activity file with the combined traffic data and corresponding source information for communication to the analytic controller.
A system for datacenter traffic analysis includes end hosts with agents that summarize traffic flow data (e.g., bytes, packets) between computing nodes. This agent bundles the summarized traffic data with source information and sends it to an analytic controller. The analytic controller aggregates this data from multiple hosts. It then retrieves a pre-defined path (list of links and intermediate nodes) between source and destination from a lookup table. This path data is combined with the aggregated traffic information and prepared for parallel processing to generate traffic statistics. To optimize, the system groups multiple traffic flows with the same source and destination within a time period into a "tuple-traffic group", and summarizes their combined traffic data into a pre-activity file before sending to the analytic controller.
2. The system of claim 1 , wherein the processor of at least one of the analytic controller and end host is further to execute: making formatting changes to the aggregated traffic data required for processing the data set by a data-mining application operable for performing coordinated, parallel-processing on the computing nodes in the cluster of computing nodes.
The system described for datacenter traffic analysis has an analytic controller or end host that reformats the aggregated traffic data. This reformatting prepares the data for a data-mining application, designed for coordinated, parallel processing across computing nodes. The application uses the reformatted data to derive traffic statistics through parallel processing, enabling efficient analysis of large datasets.
3. The system of claim 2 , wherein the cluster of computing nodes comprises a set of computing nodes within the datacenter.
Expanding on the datacenter traffic analysis system, the cluster of computing nodes performing parallel processing is simply a set of computing nodes within the datacenter. This specifies that the parallel processing infrastructure is integrated directly into the datacenter's existing resources, rather than relying on external systems.
4. The system of claim 2 , wherein the paths enforced on packets with common destinations and sources are enforced with respect to a logical overlay, allowing multiple different physical paths within a single path at a level of the logical overlay.
In the datacenter traffic analysis system, the enforced paths for packets with common destinations and sources are enforced at the level of a logical overlay network. This allows for multiple different physical paths to exist under a single logical path. This means the traffic analysis works at a higher level of abstraction, independent of the specific physical route a packet takes, making it more resilient to underlying network changes.
5. The system of claim 2 , wherein the processor of the analytic controller is further to execute aggregating the traffic data over a predetermined timeframe, the predetermined timeframe tuned to provide the data set: of sufficient size to detect patterns for traffic flows in the data center; and small enough to appropriately utilize the cluster to continuously provide relevant updates.
In the datacenter traffic analysis system, the analytic controller aggregates traffic data over a pre-determined timeframe. This timeframe is configured to be long enough to capture traffic flow patterns in the datacenter, but also short enough to allow the cluster of computing nodes to process the data and provide relevant updates continuously. The timeframe represents a trade-off between data volume and processing time.
6. The system of claim 1 , wherein the look up table comprises a plurality of lists, each list of the plurality of lists comprising at least one edge and at least two nodes traversed by traffic with a common destination and source.
In the datacenter traffic analysis system, the lookup table, which stores network paths, contains multiple lists. Each list represents a path for traffic with a specific source and destination and includes at least one network link (edge) and two intermediate nodes. This detailed path information allows the system to map traffic flows to specific network segments for analysis.
7. The system of claim 1 , the processor of the end host is further to execute: collecting traffic data about data transmitted from multiple virtual computing instances supported by the end host; and collating the traffic data to source information for the multiple virtual computing instances transmitting the traffic data.
In the datacenter traffic analysis system, the end host collects traffic data about data transmitted from multiple virtual computing instances. The host then associates (collates) the traffic data with source information for each of these virtual instances. This enables traffic analysis at the granularity of individual virtual machines or containers running on the host.
8. The system of claim 1 , the processor of the end host further to execute: providing a system call emulation in user space, emulating a kernel system call to send data on a socket buffer; and copying data metric information and copy source information from a send call to the system call emulation at the end host to transmit data within the datacenter over the a socket at the end host; providing data metric information and to copy source information from a send call at the end host to transmit data within the datacenter over the a socket at the end host.
In the datacenter traffic analysis system, the end host provides a system call emulation in user space that mimics a kernel system call to send data on a socket buffer. The end host copies both data metric information and source information from a user-space send call to this system call emulation when transmitting data. This approach allows the agent to capture traffic data without requiring modifications to the kernel, improving flexibility and reducing the risk of instability.
9. A method for calculating statistics about traffic within a datacenter, the method comprising: summarizing, by a flow agent at an end host in a datacenter, data transmissions across multiple packets at, at least one of, a traffic-flow level and a traffic-tuple level for packets transmitted from a common socket; communicate the data-transmission summaries with corresponding identification information to an analytic controller; aggregate data-transmission summaries and accompanying identification information from multiple flow agents at the analytic controller; store aggregated, data-transmission summaries at different nodes within a cluster of computing nodes in preparation for parallel processing; maintaining topology information for the datacenter at the analytic controller, the topology information including a plurality of enforced paths including a list of one or more edges and two or more intervening nodes enforced on packets with common destinations and sources by a connection based packet switching protocol implemented by the datacenter and to convert identification information for at least one of a traffic flow and a tuple-traffic group to a path through the datacenter, the path described by at least one edge and at least two nodes traversed; and pre-processing the aggregated data-transmission summaries by translating, with the topology information, the accompanying identification information into paths of edges and nodes and storing the paths of edges and nodes in the data transmission summaries; and uncover traffic statistics for the datacenter by performing data mining during parallel processing of the aggregated data-transmission summaries within the cluster of computing nodes; wherein the method further comprises, by the flow agent: maintaining a counter to indicate when a grouping time period has expired; combining traffic data for multiple traffic flows with a common source and destination within the grouping time period into a common tuple-traffic group, the common tuple-traffic group describable by common parameter values for the common source and destination within the source information used to transmit traffic flows belonging to the common tuple-traffic group; generating combined traffic data summarizing traffic scheduled for the common tuple-traffic group; and generating a pre-activity file with the combined traffic data and corresponding source information for communication to the analytic controller.
A method for calculating traffic statistics in a datacenter involves a flow agent summarizing data transmissions across multiple packets at an end host, at either the traffic flow or tuple level. The agent sends summaries and identification information to an analytic controller, which aggregates data from multiple agents and stores it across a computing cluster for parallel processing. The controller maintains datacenter topology, including enforced paths (edges and nodes) for traffic. It uses this to convert flow or tuple IDs into paths. The method preprocesses the data, translating IDs into paths and incorporating them into the summaries. Data mining on the aggregated summaries then uncovers traffic statistics. The agent groups flows with common sources/destinations within a time window into tuple-traffic groups, summarizes the combined traffic, generates a pre-activity file, and sends it to the controller.
10. The method of claim 9 , further comprising pre-processing the aggregated data-transmission summaries by removing sufficient data to accommodate parallel processing within a range of predetermined times by available computing resources in the cluster.
The method for calculating traffic statistics in a datacenter includes preprocessing the aggregated data-transmission summaries by removing sufficient data to ensure parallel processing completes within a predetermined time, given the available computing resources. This step is crucial for optimizing performance and preventing resource overload during the data mining process.
11. The method of claim 10 , further comprising combining multiple instances of traffic statistics, each instance of traffic statistics generated for an incremental period of time within the datacenter through parallel processing, the combined multiple instances of traffic statistics resulting in traffic statistics for the datacenter over an increased period of time while avoiding parallel processing of a larger data set corresponding the increased period of time.
The method of calculating datacenter traffic statistics can be improved by combining multiple instances of traffic statistics, where each instance is generated for a small time increment. This results in traffic statistics for a larger overall period, but avoids having to parallel-process a larger, single dataset corresponding to the larger period. This technique allows for incremental analysis and reduces the computational burden of analyzing large datasets at once.
12. The method of claim 9 , further comprising: maintaining a counter at the flow agent; referencing the counter to determine that a grouping period has elapsed; summarizing traffic scheduled for transmission during the grouping period within a report; and communicating the report to the analytic controller.
In the method for calculating traffic statistics in a datacenter, the flow agent maintains a counter to track time. When the counter indicates a grouping period has elapsed, the agent summarizes all traffic scheduled for transmission during that period into a report. This report is then communicated to the analytic controller, enabling time-based aggregation and analysis of traffic patterns.
13. The method of claim 9 , further comprising: adding at least one of a new traffic flow and a new tuple-traffic group to a list of at least one of traffic flows and tuple-traffic groups to be monitored when a new socket, used to define the at least one of the new traffic flow and the new tuple-traffic group, is created; and monitoring traffic scheduled for transmission from the new socket to collect traffic data summarizing data transmitted over the new socket.
The method for calculating traffic statistics in a datacenter involves dynamically adding new traffic flows or tuple-traffic groups to the list of items being monitored. This occurs when a new socket is created, which defines a new traffic flow or group. The system then monitors traffic scheduled for transmission from this new socket, collecting data to summarize traffic transmitted over it. This ensures that newly established connections are automatically included in the traffic analysis.
Unknown
September 5, 2017
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.