Graphical User Interface Representing Event Streams

This application is a continuation of U.S. patent application Ser. No. 18/629,889, filed on Apr. 8, 2024, which is a continuation of U.S. Pat. No. 12,028,208, filed on Jun. 6, 2023, which is a continuation of U.S. Pat. No. 11,716,248, filed on Jan. 18, 2022, which is a continuation of U.S. Pat. No. 11,245,581, filed on Sep. 17, 2019, which is a continuation of U.S. Pat. No. 10,462,004, filed on Apr. 29, 2015, which is a continuation-in-part of U.S. Pat. No. 10,360,196, filed on Jan. 30, 2015, which is a continuation-in-part of U.S. Pat. No. 10,127,273, filed on Apr. 15, 2014, which is also a continuation-in-part of U.S. Pat. No. 9,838,512, filed on Oct. 30, 2014. The subject matter of this application is also related to the subject matter in U.S. Pat. No. 10,700,950, filed on Apr. 29, 2015. The contents of each of the above-referenced applications are hereby incorporated by reference in their entirety.

The disclosed embodiments relate to techniques for processing network data. More specifically, the disclosed embodiments relate to techniques for providing visualizations of statistics associated with captured network data.

Large volumes of performance and log data may be captured as “events,” wherein each event includes a collection of performance data and/or diagnostic information that is generated by a computer system and is correlated with a specific point in time. Events can be derived from “time-series event data,” wherein time-series data comprises a sequence of data points (e.g., performance measurements from a computer system) that are associated with successive points in time and are typically spaced at uniform time intervals. More specifically, an event stream of time-series event data may be generated from wire data, such as network packets, captured by a number of remote capture agents deployed across a network. The remote capture agents may be installed on physical servers and/or virtual machines on the network. As a result, the remote capture agents may avert the need to deploy and connect physical hardware to network TAPS or SPAN ports, thus allowing users to configure and change their data capture configuration on-the-fly rather than in fixed formats.

Configuration or management of event streams generated from network packets captured by the remote capture agents may be performed through a configuration server and/or GUI. The configuration server and/or GUI may allow a user (e.g., an administrator) to specify a protocol used by network packets from which an event stream is created. Because such protocol-based capture and analysis of network data may result in the capture of multiple protocols in a large number of event streams, event stream information for the event streams may be grouped by one or more event stream attributes (e.g., protocol, application, category, event stream lifecycle) in the GUI. Such grouping(s) of the event stream information may facilitate analysis, understanding, and management of the event streams by the user.

The configuration server and/or GUI may also include a number of mechanisms or user-interface elements that further assist the user with management and use of the event streams. First, the configuration server and/or GUI may enable the generation of a set of statistics from an event stream without subsequently storing and processing the event stream by one or more components on a network. Alternatively, the GUI may enable the selective storage and/or processing of at least a portion of the event stream based on the statistics, a storage limit associated with the time-series event data, and/or user input through the GUI.

Second, the GUI may display the statistics and/or one or more graphs containing values from the statistics, along with a value of a statistic based on a position of a cursor over the graph(s). For example, the GUI may display a bar chart and/or pie chart of index volume for one or more event streams across a pre-specified time range. The GUI may also display the value of the index volume represented by a segment of the bar chart and/or a slice of the pie chart over which the cursor is positioned. The GUI may further highlight the segment and/or slice and dim other portions of the chart. Consequently, the configuration server and/or GUI may improve user understanding and/or decision-making related to partial or complete storage, indexing, and/or processing of event streams.

Thus, the disclosed embodiments provide a system that facilitates the processing of network data. During operation, the system causes for display a graphical user interface (GUI) for configuring the generation of time-series event data from network packets captured by one or more remote capture agents. Next, the system causes for display, in the GUI, a first set of user-interface elements containing a set of statistics associated with one or more event streams that comprise the time-series event data. The system then causes for display, in the GUI, one or more graphs comprising one or more values from the set of statistics. Finally, the system causes for display, in the GUI, a value of a statistic from the set of statistics based on a position of a cursor over the one or more graphs.

In some embodiments, the system also updates the set of statistics and the one or more graphs in real-time with the time-series event data from the one or more remote capture agents.

In some embodiments, the system also causes for display, in the GUI, a second set of user-interface elements for changing a view of the one or more graphs.

In some embodiments, changing the view of the one or more graphs includes at least one of:

In some embodiments, the system also sorts the set of statistics by an attribute associated with the set of statistics.

In some embodiments, the attribute is at least one of a name, a total number of events; a total incoming traffic, a total outgoing traffic, a total traffic, and an index volume.

In some embodiments, the one or more graphs include a pie chart of an index volume across the one or more event streams.

In some embodiments, the pie chart is updated with one or more values of an index volume of an event stream based on a position of a cursor over the pie chart.

In some embodiments, the one or more values include a percentage of a total index volume associated with the event stream and an amount of data associated with the index volume of the event stream.

In some embodiments, the set of statistics include at least one of a total number of events, a total incoming traffic, a total outgoing traffic, a total traffic, and an index volume.

In some embodiments, the one or more graphs include a bar chart of an index volume of the one or more event streams over time.

In some embodiments, a bar in the bar chart includes one or more segments representing the index volume of the one or more event streams over a time interval.

In some embodiments, the bar chart is updated with a value of an index volume of an event stream based on a position of a cursor over the bar chart.

In some embodiments, a portion of the bar chart is highlighted based on a position of a cursor over a legend of the bar chart.

In some embodiments the GUI further includes a second set of user-interface elements for managing the one or more event streams, and managing the one or more event streams includes enabling the generation of the set of statistics from an event stream without transmitting the event stream over a network for subsequent storage and processing of the event stream by one or more components on the network.

In some embodiments, managing the one or more event streams comprises adjusting an amount of capture of the one or more event streams based on the set of statistics.

In some embodiments, the system also causes for display a value of a statistic from the set of statistics based on a position of a cursor over a legend associated with the one or more graphs.

In some embodiments, the value of the statistic includes the index volume associated with the position of the cursor over the one or more graph.

In some embodiments, the value of the statistic is at least one of a percentage of a total index volume associated with the one or more event streams and an amount of data associated with the index volume of the one or more event streams.

In some embodiments, the system also changes an appearance of the one or more graphs based on the position of the cursor over the one or more graphs.

In some embodiments, changing the appearance of the one or more graphs includes highlighting a portion of a graph based on the position of the cursor over the graph.

In some embodiments, changing the appearance of the one or more graphs includes dimming a portion of a graph based on the position of the cursor over the graph.

In some embodiments, changing the appearance of the one or more graphs includes highlighting a first portion of a graph and dimming a second portion of the graph based on the position of the cursor over the graph.

In some embodiments, the system also changes an appearance of the one or more graphs based on the position of the cursor over a legend associated with the one or more graphs.

In some embodiments changing the appearance of the one or more graphs includes highlighting a portion of a graph based on the position of the cursor over the legend.

In some embodiments, changing the appearance of the one or more graphs includes dimming a portion of a graph based on the position of the cursor over the legend.

In some embodiments, changing the appearance of the one or more graphs includes highlighting a first portion of a graph and dimming a second portion of the graph based on the position of the cursor over the legend.

In some embodiments, causing for display, in the GUI, the value of the statistic includes at least one of:

In the figures, like reference numerals refer to the same figure elements.

The following description is presented to enable any person skilled in the art to make and use the embodiments, and is provided in the context of a particular application and its requirements. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the present disclosure. Thus, the present invention is not limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

The data structures and code described in this detailed description are typically stored on a computer-readable storage medium, which may be any device or medium that can store code and/or data for use by a computer system. The computer-readable storage medium includes, but is not limited to, volatile memory, non-volatile memory, magnetic and optical storage devices such as disk drives, magnetic tape, CDs (compact discs), DVDs (digital versatile discs or digital video discs), or other media capable of storing code and/or data now known or later developed.

The methods and processes described in the detailed description section can be embodied as code and/or data, which can be stored in a computer-readable storage medium as described above. When a computer system reads and executes the code and/or data stored on the computer-readable storage medium, the computer system performs the methods and processes embodied as data structures and code and stored within the computer-readable storage medium.

Furthermore, methods and processes described herein can be included in hardware modules or apparatus. These modules or apparatus may include, but are not limited to, an application-specific integrated circuit (ASIC) chip, a field-programmable gate array (FPGA), a dedicated or shared processor that executes a particular software module or a piece of code at a particular time, and/or other programmable-logic devices now known or later developed. When the hardware modules or apparatus are activated, they perform the methods and processes included within them.

Modern data centers often comprise thousands of host computer systems that operate collectively to service requests from even larger numbers of remote clients. During operation, these data centers generate significant volumes of performance data and diagnostic information that can be analyzed to quickly diagnose performance problems. In order to reduce the size of this performance data, the data is typically pre-processed prior to being stored based on anticipated data-analysis needs. For example, pre-specified data items can be extracted from the performance data and stored in a database to facilitate efficient retrieval and analysis at search time. However, the rest of the performance data is not saved and is essentially discarded during pre-processing. As storage capacity becomes progressively cheaper and more plentiful, there are fewer incentives to discard this performance data and many reasons to keep it.

This plentiful storage capacity is presently making it feasible to store massive quantities of minimally processed performance data at “ingestion time” for later retrieval and analysis at “search time.” Note that performing the analysis operations at search time provides greater flexibility because it enables an analyst to search all of the performance data, instead of searching pre-specified data items that were stored at ingestion time. This enables the analyst to investigate different aspects of the performance data instead of being confined to the pre-specified set of data items that was selected at ingestion time.

However, analyzing massive quantities of heterogeneous performance data at search time can be a challenging task. A data center may generate heterogeneous performance data from thousands of different components, which can collectively generate tremendous volumes of performance data that can be time-consuming to analyze. For example, this performance data can include data from system logs, network packet data, sensor data, and data generated by various applications. Also, the unstructured nature of much of this performance data can pose additional challenges because of the difficulty of applying semantic meaning to unstructured data, and the difficulty of indexing and querying unstructured data using traditional database systems.

These challenges can be addressed by using an event-based system, such as the SPLUNK® ENTERPRISE system produced by Splunk Inc. of San Francisco, California, to store and process performance data. The SPLUNK® ENTERPRISE system is the leading platform for providing real-time operational intelligence that enables organizations to collect, index, and harness machine-generated data from various websites, applications, servers, networks, and mobile devices that power their businesses. The SPLUNK® ENTERPRISE system is particularly useful for analyzing unstructured performance data, which is commonly found in system log files. Although many of the techniques described herein are explained with reference to the SPLUNK® ENTERPRISE system, the techniques are also applicable to other types of data server systems.

In the SPLUNK® ENTERPRISE system, performance data is stored as “events,” wherein each event comprises a collection of performance data and/or diagnostic information that is generated by a computer system and is correlated with a specific point in time. Events can be derived from “time-series data,” wherein time-series data comprises a sequence of data points (e.g., performance measurements from a computer system) that are associated with successive points in time and are typically spaced at uniform time intervals. Events can also be derived from “structured” or “unstructured” data. Structured data has a predefined format, wherein specific data items with specific data formats reside at predefined locations in the data. For example, structured data can include data items stored in fields in a database table. In contrast, unstructured data does not have a predefined format. This means that unstructured data can comprise various data items having different data types that can reside at different locations. For example, when the data source is an operating system log, an event can include one or more lines from the operating system log containing raw data that includes different types of performance and diagnostic information associated with a specific point in time. Examples of data sources from which an event may be derived include, but are not limited to: web servers; application servers; databases; firewalls; routers; operating systems; and software applications that execute on computer systems, mobile devices, and sensors. The data generated by such data sources can be produced in various forms including, for example and without limitation, server log files, activity log files, configuration files, messages, network packet data, performance measurements and sensor measurements. An event typically includes a timestamp that may be derived from the raw data in the event, or may be determined through interpolation between temporally proximate events having known timestamps.

The SPLUNK® ENTERPRISE system also facilitates using a flexible schema to specify how to extract information from the event data, wherein the flexible schema may be developed and redefined as needed. Note that a flexible schema may be applied to event data “on the fly,” when it is needed (e.g., at search time), rather than at ingestion time of the data as in traditional database systems. Because the schema is not applied to event data until it is needed (e.g., at search time), it is referred to as a “late-binding schema.”

During operation, the SPLUNK® ENTERPRISE system starts with raw data, which can include unstructured data, machine data, performance measurements or other time-series data, such as data obtained from weblogs, syslogs, or sensor readings. It divides this raw data into “portions,” and optionally transforms the data to produce timestamped events. The system stores the timestamped events in a data store, and enables a user to run queries against the data store to retrieve events that meet specified criteria, such as containing certain keywords or having specific values in defined fields. Note that the term “field” refers to a location in the event data containing a value for a specific data item.

As noted above, the SPLUNK® ENTERPRISE system facilitates using a late-binding schema while performing queries on events. A late-binding schema specifies “extraction rules” that are applied to data in the events to extract values for specific fields. More specifically, the extraction rules for a field can include one or more instructions that specify how to extract a value for the field from the event data. An extraction rule can generally include any type of instruction for extracting values from data in events. In some cases, an extraction rule comprises a regular expression, in which case the rule is referred to as a “regex rule.”

In contrast to a conventional schema for a database system, a late-binding schema is not defined at data ingestion time. Instead, the late-binding schema can be developed on an ongoing basis until the time at which a query is actually executed. This means that extraction rules for the fields in a query may be provided in the query itself, or may be located during execution of the query. Hence, as an analyst learns more about the data in the events, the analyst can continue to refine the late-binding schema by adding new fields, deleting fields, or changing the field extraction rules until the next time the schema is used by a query. Because the SPLUNK® ENTERPRISE system maintains the underlying raw data and provides a late-binding schema for searching the raw data, it enables an analyst to investigate questions that arise as the analyst learns more about the events.

In the SPLUNK® ENTERPRISE system, a field extractor may be configured to automatically generate extraction rules for certain fields in the events when the events are being created, indexed, or stored, or possibly at a later time. Alternatively, a user may manually define extraction rules for fields using a variety of techniques.

Also, a number of “default fields” that specify metadata about the events rather than data in the events themselves can be created automatically. For example, such default fields can specify: a timestamp for the event data; a host from which the event data originated; a source of the event data; and a source type for the event data. These default fields may be determined automatically when the events are created, indexed or stored.