Detecting Data Change Based on Adjusted Data Values

1. A method of detecting data change, comprising: separating observed data values into plural sets representative of respective distinct time windows; calculating, by at least one processor, predefined values representative of a temporal effect for the respective plural sets, wherein each of the predefined values is an aggregate value computed from the observed data values in the respective one of the sets; calculating, by the at least one processor, adjusted data values by removing impact of the predefined values such that the temporal effect is removed from respective observed data values in the respective plural sets, wherein the adjusted data values associated with each of the plural sets are calculated based on a difference between the observed data values within the corresponding set and the corresponding aggregate value; and detecting, by the at least one processor, the data change based on the adjusted data values.

2. The method of claim 1 , wherein detecting the data change comprises applying a change-point detection algorithm based on the adjusted data values.

3. The method of claim 1 , wherein the observed data values separated into the plural sets are part of a historical data set, the method further comprising: receiving an additional observed data value; calculating a further adjusted data value by subtracting one of the predefined values from the additional observed data value, wherein detecting the data change is further based on the further adjusted data value.

4. The method of claim 1 , wherein the observed data values comprise a time series of observed data values y t up to a time point N (t=1 to N), and wherein calculating the predefined values comprises calculating the predefined values based on the time series of observed data values y t from t=1 to t=N.

5. The method of claim 1 , further comprising determining whether any observed data value is an outlier by: setting an upper value and a lower value in a respective set of the plural sets; and determining that the observed data value is an outlier if the observed data value exceeds the upper value or is less than the lower value.

6. The method of claim 1 , wherein detecting the data change comprises detecting a systematic change.

7. The method of claim 1 , wherein detecting the data change identifies a change point at a lowest time granularity level provided by the observed data values.

8. The method of claim 1 , wherein separating the observed data values into the plural sets representative of the respective distinct time windows comprises separating the observed data values into the plural sets representative of periodically repeating time windows.

9. The method of claim 2 , wherein applying the change-point detection algorithm based on the adjusted data values comprises applying one of a cumulative sums (CUSUM) technique, a generalized likelihood ratio (GLR) technique, and a regression CUSUM technique.

10. The method of claim 3 , further comprising performing a quality determination of the additional observed data value.

11. The method of claim 7 , further comprising performing trend change detection at a higher aggregated time granularity level that is greater than the lowest time granularity level.

12. A method of detecting data change, comprising: separating observed data values into plural sets representative of respective time windows; calculating, by at least one processor, predefined values representative of a temporal effect for the respective plural sets; calculating, by the at least one processor, adjusted data values by removing impact of the predefined values such that the temporal effect is removed from respective observed data values in the respective plural sets; detecting, by the at least one processor, the data change based on the adjusted data values, wherein the observed data values comprise a time series of observed data values y t up to a time point N (t=1 to N), and wherein calculating the predefined values comprises calculating the predefined values based on the time series of observed data values y t from t=1 to t=N; and receiving observed data values y t after time point N (t≧N+1), wherein calculating the adjusted data values comprises calculating residual values r t , t=1 to t≧N+1, where r t =y t − y (k) , y (k) representing the predefined values, and k is a value from 1 to K, where K represents a number of the plural sets.

13. The method of claim 12 , wherein the plural sets of observed data values correspond to K time windows, wherein each y t corresponds to a respective time window k, k=1 to K, and wherein calculating each r t comprises selecting y (k) from among y (1) to y (k) to subtract from the corresponding y t .

14. The method of claim 13 , wherein the time windows represent periodically repeating time windows, and wherein receiving each observed data value y t comprises receiving a data value that falls in one of the periodically repeating time windows.

15. The method of claim 13 , wherein the plural time windows comprise the following time windows: Monday, Tuesday, Wednesday, Thursday, Friday, Saturday, Sunday, wherein 1 to K correspond to Monday to Sunday, and wherein each y t represents a data value occurring in one of Monday to Sunday.

16. A method of detecting data change, comprising: separating observed data values into plural sets representative of respective time windows; calculating, by at least one processor, predefined values representative of a temporal effect for the respective plural sets; calculating, by the at least one processor, adjusted data values by removing impact of the predefined values such that the temporal effect is removed from respective observed data values in the respective plural sets; detecting, by the at least one processor, the data change based on the adjusted data values; determining whether any observed data value is an outlier by: setting an upper value and a lower value in a respective set of the plural sets; determining that the observed data value is an outlier if the observed data value exceeds the upper value or is less than the lower value, wherein setting the upper value and lower value in the respective set comprises: computing an upper quantile and a lower quantile; computing an inter quantile that equals a difference between the upper quantile and lower quantile; determining the upper value as a maximum data value in the set that is no greater than (upper quantile+inter quantile*outlier range factor), wherein outlier range factor is a predefined constant; and computing the lower value as a minimum data value in the set that is no less than (lower quantile−inter quantile*outlier range factor).

17. A system comprising: at least one processor to: separate observed data values into plural subsets representing respective repeating time windows; calculate indications of a temporal effect on respective observed data values in the respective subsets, wherein each of the indications is an aggregate value computed from the observed data values in the respective one of the plural subsets; calculate adjusted data values by removing impact of the aggregate values such that the temporal effect is removed from respective observed data values in the respective plural subsets, wherein the adjusted data values associated with each of the plural subsets are calculated based on a difference between the observed data values within the corresponding subset and the corresponding aggregate value; and perform change point detection based on the adjusted data values.

18. The system of claim 17 , wherein the at least one processor is configured to further: receive at least one additional observed data value that corresponds to at least one of the plural subsets; and perform data quality determination of the at least one additional observed data value.

19. The system of claim 17 , wherein the repeating time windows represented by the respective plural subsets are periodically repeating time windows.

20. A method of detecting data change, comprising: separating observed data values into plural sets representative of respective distinct time windows; calculating, by at least one processor, predefined values representative of a temporal effect for the respective plural sets; calculating, by the at least one processor, adjusted data values by removing impact of the predefined values such that the temporal effect is removed from respective observed data values in the respective plural sets; and detecting, by the at least one processor, the data change based on the adjusted data values; wherein separating the observed data values into the plural sets representative of the respective distinct time windows comprises separating the observed data values into the plural sets representative of periodically repeating time windows, wherein calculating the adjusted data values for each of the plural sets comprises calculating the adjusted data values based on the observed data values and the predefined value of the corresponding set, and wherein the adjusted data values for each of the plural sets comprise adjusted data values equal to respective differences between the observed data values and the predefined value of the corresponding set.

21. A system comprising: at least one processor to: separate observed data values into plural subsets representing respective repeating time windows; calculate indications of a temporal effect on respective observed data values in the respective subsets; and perform change point detection by removing the temporal effect of the respective subsets from respective observed data values, wherein the temporal effect is removed from each of the subsets by calculating residual values that are based on the observed data values and the predefined value of the corresponding subset, and wherein the residual values for each of the subsets are equal to respective differences between the observed data values and the predefined value of the respective subset.