Information Processing Apparatus, Information Processing Method, and Non-Transitory Computer Readable Medium

This application is based upon and claims the benefit of priority from the prior Japanese Patent Application No. 2024-088379, filed on May 30, 2024, the entire contents of which are incorporated herein by reference.

Embodiments described herein relate to an information processing apparatus, an information processing method, and a computer program.

In time-series waveform data such as patient's electrocardiogram data and pressure data of a propulsion apparatus of a spacecraft, in which a certain waveform pattern is repeated at predetermined periods, there exists a type of anomaly (partial anomaly) in which part of the waveform pattern collapses. By monitoring the time-series waveform data, it is possible to detect a period of the waveform pattern in which an anomaly has occurred. In this event, there is a demand to detect an interval from start to end of the occurrence of the anomaly in the waveform pattern in which the anomaly has occurred. In other words, there is a demand to detect a state (for example, normality or anomaly) of some intervals in the waveform pattern.

According to one embodiment, an information processing apparatus includes processing circuitry. The processing circuitry generates first distance information based on measurement waveform data and reference waveform data, the measurement waveform data including a plurality of first data points related to a measurement target that can transition between a first state and a second state, the reference waveform data including a plurality of second data points related to the first state, the first distance information including distances between the first data points and the second data points. The processing circuitry determines one or more corresponding second data points for each of the first data points based on the first distance information to generate first correspondence data in which the first data points are associated with the second data points. The processing circuitry

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings, the same reference numerals are assigned to the same or corresponding elements, and detailed description will be omitted as appropriate.

is a view for explaining outline of operation of an information processing apparatusaccording to Embodiment 1. Periodic time-series data measured from a certain measurement target is input to the information processing apparatus, for example. In the periodic time-series data, waveforms of almost identical patterns repeatedly appear. Examples of such time-series data include patient's electrocardiogram data, pressure data of a propulsion apparatus of a spacecraft, and sensing data from equipment that repeatedly performs a plurality of processes at certain periods.

In a case of the electrocardiogram data, the length of a waveform in a time direction included in one period expands and contracts according to a heart rate of the patient, but when the lengths of respective waveforms in the time direction are properly adjusted, the waveforms generally match. In a case of the pressure data of the propulsion apparatus, the length of a waveform in a time direction included in one period expands and contracts according to output of the propulsion apparatus, but when the lengths of respective waveforms in the time direction are properly adjusted, the waveforms generally match.

In a case of the electrocardiogram data, if the patient experiences arrhythmia, deformation occurs in part of the periodic waveform pattern. In a case of the pressure data of the propulsion apparatus, if knocking occurs in the propulsion apparatus, deformation occurs in part of the periodic waveform pattern. Further, in the equipment that performs a plurality of processes, in a case where an anomaly occurs in some processes, deformation occurs in a part corresponding to the processes in the waveform pattern. The information processing apparatusdetects partial deformation of such waveform patterns as a partial anomaly, and detects an interval in which the partial anomaly exists in the time-series data. In other words, the information processing apparatusdetects an interval in which a measurement target is in a second state (for example, an anomaly state) different from a first state (for example, regular state or a normal state). However, a state for which the interval is to be detected (the second state) is not limited to the anomaly state, and it is sufficient if the state for which the interval is to be detected is a state different from the regular state. For example, the state for which the interval is to be detected may be a state that is a precursor to an anomaly, or a state in which degradation has progressed significantly.

is a block diagram illustrating a configuration of the information processing apparatusaccording to Embodiment 1. The information processing apparatusincludes a data acquirer, a data storage, a first cost calculator, a first similarity calculator, a second cost calculator, a second similarity calculator, an anomaly detector, and an output device.

Some or all of the data acquirer, the data storage, the first cost calculator, the first similarity calculator, the second cost calculator, the second similarity calculator, the anomaly detector, and the output deviceconstitute a processor or processing circuitry, that performs processing according to the present embodiment.

In data acquirer, periodic time-series data measured by a sensor that senses a measurement target that is put into both the first state and the second state is input. The time-series data is sampled in a time direction at a predetermined sampling frequency and quantized in an amplitude direction with a predetermined number of bits. The data acquirersequentially extracts, from the periodic time-series data, measurement waveform data corresponding to each period of the periodic time-series data. Each piece of the measurement waveform data includes M data points (first data points) from the first to the M-th data points. The first of the first data points corresponds to the initial first data point, and the M-th of the first data points corresponds to the last first data point.

In the data storage, reference waveform data including M data points (second data points) from the first to the M-th data points, related to the first state (for example, a normal state), and predetermined setting values respectively corresponding to the M data points (details will be described later) are stored in advance. The first of the second data points in the reference waveform data corresponds to the initial second data point, and the M-th of the second data points corresponds to the last second data point. In Embodiment 1, a case will be described where the number of data points of the measurement waveform data is equal to the number of data points of the reference waveform data, but the number of data points of both may be different.

A method for creating the reference waveform data is not particularly limited. For example, the reference waveform data can be created by sequentially extracting, from the periodic time-series data measured from a measurement target in a normal state, test waveform data corresponding to one period of the periodic time-series data and obtaining a centroid waveform (barycenter) of the plurality of pieces of test waveform data. As the method for obtaining the centroid waveform, for example, asoft-DTW method and a DBA method can be used.

illustrates an example of normal measurement waveform data in a case where M=10 and an example of the reference waveform data. An upper part ofis the normal measurement waveform data, and a lower part ofis the reference waveform data. Comparing the two kinds of waveform data, they generally match.

illustrates an example of the measurement waveform data including a partial anomaly in a case where M=10, and an example of the reference waveform data. An upper part ofis the measurement waveform data including a partial anomaly, and a lower part ofis the reference waveform data. Comparing the two kinds of waveform data, there is a significant discrepancy in the sixth and seventh data points corresponding to the partial anomaly included in the measurement waveform data.

The first cost calculatorcalculates a distance between M data points (first data points) included in the measurement waveform data and M data points (second data points) included in the reference waveform data. More specifically, for each of the M first data points included in the measurement waveform data, a distance to each of the M second data points included in the reference waveform data is calculated. The first cost calculatorgenerates distance information (first distance information) that includes each distance between the first data point and the second data point based on the calculated distance.

Here, as the first distance information, a matrix (first distance matrix) having a size of M×M, in which the calculated distances are stored in the corresponding elements for each pair of the first data point and the second data point is generated. More specifically, the distance between the first data point and the second data point is defined as cost, and a matrix (first cost matrix D1) in which the cost is stored in the corresponding elements for each pair of the first data point and the second data point is calculated.

In general, a distance δ(x, y) between two data points x and y, that is, cost, can be defined as follows using γ as a parameter of a real number.

In Embodiment 1, a case where γ=1 in the above formula, that is, a Euclidean distance is considered as the cost. In this case, if the reference waveform data is set as {x1, x2, . . . xM} and the measurement waveform data is set as {y1, y2, . . . yM}, the first cost matrix D1 is calculated as follows.

illustrates two examples of the first cost matrix D1 (first distance information or first distance matrix). A left part ofis the first cost matrix D1 calculated from the normal measurement waveform data and the reference waveform data. A right part ofis the first cost matrix D1 calculated from the measurement waveform data including a partial anomaly and reference waveform data. The element in the i-th row and j-th column of the first cost matrix D1 is represented as an element (i, j). For example, the element in the sixth row and sixth column of the first cost matrix D1 in the right part ofis an element (,), and a value of the element (,) is 1.5. The value of the element (,) being 1.5 indicates that the distance (cost) between the sixth data point x1 of the reference waveform data and the sixth data point y6 of the measurement waveform data including a partial anomaly, is 1.5.

The first similarity calculatordetermines one or more corresponding second data points for each first data point based on the first cost matrix D1, and generates first correspondence data indicating correspondence between each first data point and the determined second data points. For example, the first similarity calculatorgenerates the first correspondence data by, for each first data point, searching for one or more corresponding second data points so that a sum of distances to the corresponding second data points is minimized, and all the second data points are associated with at least one first data point. Then, the first similarity calculatorcalculates a first evaluation value using the sum of distances between the corresponding first data points and the second data points included in the first correspondence data. The first evaluation value corresponds to the first similarity S1 between the measurement waveform data and the reference waveform data in the present embodiment, but may be a value other than similarity. The processing of the first similarity calculatorwill be described in more detail below.

In Embodiment 1, generation of the first correspondence data and calculation of the first similarity S1 are performed using dynamic time warping (DTW) from the first cost matrix D1. One of the characteristics of the DTW is its robustness to stretching and phase shift in a time direction. By using the DTW, even in a case where lengths of the measurement waveform data and the reference waveform data are different, or in a case where their phases are shifted, it is possible to calculate the similarity of two kinds of waveform data while taking these factors into account.

In detail, the first similarity calculatorcalculates a first DTW matrix V1 from the first cost matrix D1 according to the following formula.

In other words, starting from the element corresponding to a pair of the initial first data point and the initial second data point in the first cost matrix D1 (first distance matrix), tracing of each adjacent element from the starting point is sequentially repeated. From the starting point, a path that minimizes a sum of distances of the elements included in the path to elements other than the starting point is specified, and the sum of distances of the elements included in the specified path is calculated as a DTW value (first intermediate evaluation value). Then, by storing the DTW value calculated for each element in the corresponding element, a first DTW matrix V1 (first evaluation matrix) is generated. A value of the element corresponding to the pair of the initial first data point and the initial second data points in the first DTW matrix V1 may be the same as the value at the starting point of the first cost matrix D1.

A value of the end point (M, M) of the first DTW matrix V1 (the element corresponding to the pair of the last first data point and the last second data point) is the first similarity S1 (first evaluation value). When the first DTW matrix V1 is generated, correspondence between the first data point and the second data point indicated by each element included in the path traced from the starting point to the end point in the first cost matrix D1 corresponds to the above-described first correspondence data. Then, the sum of distances indicated by each correspondence included in the first correspondence data becomes the first similarity S1 (first evaluation value). As a difference between the measurement waveform data and the reference waveform data is smaller, the first similarity S1 becomes smaller, and in a case where both completely match, the first similarity S1 becomes zero. On the contrary, as the difference between the measurement waveform data and the reference waveform data is larger, the first similarity S1 becomes larger.

Note that in the present embodiment, the similarity (evaluation value) is defined such that as the similarity (evaluation value) is larger, the difference between the measurement waveform data and the reference waveform data is larger. However, the similarity (evaluation value) may also be defined such that as the similarity (evaluation value) is larger, the measurement waveform data becomes closer to the reference waveform data.

illustrates two examples of the first DTW matrix V1 (first evaluation matrix). A left part ofis the first DTW matrix V1 calculated from the normal measurement waveform data, and a value at the end point (,) is 0.6, and thus, the first similarity S1 is 0.6. A right part ofis the first DTW matrix V1 calculated from the measurement waveform data including a partial anomaly, and a value at the end point (,) is 1.3, and thus, the first similarity S1 is 1.3.

In each first DTW matrix V1 in, the elements enclosed by solid circles represent the path (referred to as an optimal path) that minimizes a sum of cost (distance) from the starting point (,) to the end point (,). This optimal path corresponds to the above-described first correspondence data. More specifically, the correspondence between the first data point and the second data point indicated by each element included in the optimal path corresponds to the first correspondence data. The first similarity S1 corresponds to the first evaluation value calculated from the first correspondence data.

The optimal path for the first DTW matrix V1 represents a maximum likelihood correspondence relationship between each data point included in the reference waveform data and each data point included in the measurement waveform data. For example, in a case of the first DTW matrix V1 in the right part of, the first to fourth data points of the reference waveform data (first data points) correspond to the first to fourth data points of the measurement waveform data (second data points), respectively. The fifth data point of the reference waveform data corresponds to two data points, namely the fifth and sixth data points of the measurement waveform data. The sixth to seventh data points of the reference waveform data correspond to the seventh to eighth data points of the measurement waveform data, respectively. The eighth to tenth data points of the reference waveform data correspond to the eighth to tenth data points of the measurement waveform data, respectively.

The second cost calculatorcalculates the second cost matrix D2 by replacing some of the elements in the first cost matrix D1 with setting values corresponding to the second data points for those elements. More specifically, the second cost calculatorselects at least one second data point from a plurality of second data points included in the reference waveform data as a target data point, and replaces a distance (cost) between the target data point and one or more first data points in the first cost matrix D1 with a setting value corresponding to the target data point. In Embodiment 1, the second cost calculatorselects all elements from the p-th row to the q-th row of the first cost matrix D1, and calculates the second cost matrix D2 by replacing the values of the respective elements with the setting values corresponding to the second data points for each row. The second cost matrix D2 corresponds to the second distance information or second distance matrix, obtained by replacing the distances (cost) between the target data point and one or more first data points in the first cost matrix D1 with setting values corresponding to the target data point.

In the data storagedescribed above, M setting values {r1, r2, . . . rM} corresponding to M data points (the second data points) included in the reference waveform data are stored in advance. The setting values have values according to an expected range of the distance to the first data point with which the second data point is associated in the processing of the first similarity calculatorin the measurement waveform data obtained in a case where the measurement target is in a normal state. For example, the setting values have values greater than or equal to an upper limit of the range.

is a view for explaining a method for generating or determining M setting values.

First, T pieces of test waveform data including M data points (third data points) related to the first state (for example, the normal state) are prepared. The method for creating the test waveform data is not particularly limited, and, for example, the test waveform data used to create the reference waveform data above can be reused as is.

Next, for each piece of the test waveform data, a maximum likelihood correspondence relationship between the M data points (second data points) included in the reference waveform data and the M data points (third data points) included in the test waveform data is determined by the DTW. In the example of, a data point x1 of the reference waveform data corresponds to a data point yof the test waveform data y(k), a data point x2 of the reference waveform data corresponds to data points yand yof the test waveform data y(k), a data point x3 of the reference waveform data corresponds to a data point yof the test data example y(k), and a data point x4 of the reference waveform data corresponds to a data point yof the test waveform data y(k). Similarly, a data point xM of the reference waveform data corresponds to a data point yM of the test waveform data y(k).

Next, an average mean (i) and a standard deviation std (i) of a distance δ(xi, y(k)) between the i-th data point xi of the reference waveform data and one or more data points corresponding to the i-th data point of the test waveform data y(k) is calculated.

Finally, for M setting values {r1, r2, . . . rM}, the i-th setting value ri is calculated by adding a certain margin, such as a times of the standard deviation, to an average distance (average cost) between the i-th data point of the reference waveform data and one or more data points corresponding to the i-th data point of the test waveform data. A value of a parameter a is, for example, preferably around 3.0.

A method for generating the M setting values {r1, r2, . . . rM} is not limited to the above method, and various modifications and adjustments can be made to the above method. Alternatively, a method that is completely different from the above-described method may be used.

The second cost calculatorsets an interval [p, q] defined by positive integers p and q that satisfy the relationship 1≤p≤q≤M, and for all positive integers i∈ [p, q], replaces a value of each element in the i-th row of the first cost matrix D1 with the i-th setting value ri. As a result, for each combination of p and q, the second cost matrix D2 (second distance information or second distance matrix) is calculated.

illustrates two examples of the second cost matrix D2 in a case where p=6 and q=7. In this case, each element of the sixth row of the first cost matrix D1 is replaced with the sixth setting value r6, and each element of the seventh row of the first cost matrix D1 is replaced with the seventh setting value r7. In the present embodiment, the values (cost) of all elements are replaced in row unit, but only the values (cost) of one or more specific elements may be replaced. For example, the values of the leftmost and rightmost elements of the elements in a certain row do not need to be replaced.

In Embodiment 1, M=10, and thus, for the interval [p, q], there are 45 patterns in a case where p<q (represented asC), andpatterns in a case where p=q, that is, a total of 55 patterns. The second cost calculatorcalculates 55 patterns of the second cost matrix D2 corresponding to these 55 intervals [p, q]. In other words, a plurality of second cost matrices D2 are calculated for one first cost matrix D1. Hereinafter, the second cost matrix D2, in which the values of the elements related to the interval [p, q] in the first cost matrix D1 are replaced with the setting values, will be denoted as “D2” as needed. Please note that this notation does not represent the element (p, q) of the second cost matrix D2.

The second similarity calculatorcalculates a plurality of second similarities S2 (second evaluation values) corresponding to a plurality of second cost matrices D2 calculated for a plurality of intervals [p, q] that satisfy the relationship 1≤p≤q≤M. It is only necessary that the method for calculating the second similarity S2 is performed using the DTW from the second cost matrix D2 in a similar manner to a case of the first similarity S1. While details have been as described above, description will be provided again.

Based on the second cost matrix D2, one or more corresponding second data points for each first data point are determined, and second correspondence data (optimal path) that indicates the correspondence between each first data point and the determined second data points is generated. For example, by searching for one or more second data points corresponding to each first data point so that a sum of distances to the corresponding second data points is minimized, and all the second data points are associated with at least one first data point, the second correspondence data is generated. Then, the second evaluation value is calculated as the second similarity S2 using the sum of distances between the corresponding first data points and second data points included in the second correspondence data.

In other words, starting from the element corresponding to a pair of the initial first data points and the initial second data point in the second cost matrix D2, tracing of each adjacent element from the starting point is sequentially repeated. From the starting point, the path that minimizes the sum of distances of the elements included in the path to elements other than the starting point is specified, and the sum of distances of the elements included in the specified path is calculated as a DTW value (second intermediate evaluation value). Then, by storing the DTW values calculated for each element in their corresponding elements, the second DTW matrix V2 (second evaluation matrix) is generated. The value of the element corresponding to the pair of the initial first data point and the initial second data point in the second DTW matrix V2 may be the same as the value of the starting point of the second cost matrix D2. The value of the end point (M, M) of the second DTW matrix V2 (corresponding to the pair of the last first data point and the last second data point) is the second similarity S2 (second evaluation value). When the second DTW matrix V2 is generated, the correspondence between the first data point and the second data point indicated by each element included in the path (optimal path) traced from the starting point to the end point in the second cost matrix D2 corresponds to the second correspondence data described above. Then, the sum of distances related to each correspondence included in the second correspondence data becomes the second similarity S2 (second evaluation value).

is a view illustrating an example of the second DTW matrix V2and the second similarity S2calculated corresponding to the second cost matrix D2related to the interval [,]. Two examples are illustrated: one for a normal waveform (left) and one for a waveform including a partial anomaly (right). A left part ofis an example of the second DTW matrix V2calculated from normal measurement waveform data, and a value of the end point (,) is 1.6, and thus, the second similarity S2(second evaluation value) is 1.6. A right part ofis an example of the second DTW matrix V2calculated from the measurement waveform data including a partial anomaly, and a value of the end point (,) is 1.0, and thus, the second similarity S2(second evaluation value) is 1.0.

The anomaly detectordetects the interval in which the measurement target is in the second state (in this case, an anomaly state) in the measurement waveform data, by evaluating the measurement waveform data based on the optimal path specified in the first DTW matrix V1 (first correspondence data) and the optimal path specified in the second DTW matrix V2 (second correspondence data). Based on the first similarity S1 (first evaluation value) and a plurality of second similarities S2 (second evaluation values), it is determined whether there is an interval in an anomaly state in the measurement waveform data. In a case where there is an interval in an anomaly state, the interval of the measurement waveform data corresponding to the interval [P, Q] related to the second similarity S2will be determined as the interval in an anomaly state. The interval of the measurement waveform data corresponding to the interval [P, Q] related to the second similarity S2includes the first data point corresponding to the target data point (second data point included in the interval [P, Q]) in the second correspondence data (optimal path), which is the interval of the measurement waveform data. The processing of the anomaly detectorwill be described in detail below.