Efficient Transformer for Content-Aware Anomaly Detection in Event Sequences

1. A method for implementing a self-attentive encoder-decoder transformer framework for anomaly detection in event sequences, the method comprising: feeding event content information into a content-awareness layer to generate event representations; inputting, into an encoder, event sequences of two hierarchies to capture long-term and short-term patterns and to generate feature maps; adding, in the decoder, a special sequence token at a beginning of an input sequence under detection; during a training stage, applying a one-class objective to bound the decoded special sequence token with a reconstruction loss for sequence forecasting using the generated feature maps from the encoder; and during a testing stage, labeling any event representation whose decoded special sequence token lies outside a hypersphere as an anomaly.

2. The method of claim 1, wherein the decoder embeds the event sequences into a latent space where anomalies are distinguishable.

3. The method of claim 1, wherein the special sequence token represents event sequence status.

4. The method of claim 1, wherein the encoder includes attention blocks, 1-D convolutional filters with activation functions, and MaxPool layers to downsample the inputted event sequences of the two hierarchies.

5. The method of claim 1, wherein the decoder includes a masked self-attention layer to preserve an auto-regressive property.

6. The method of claim 1, wherein the decoder performs a one-time interference to predict all events.

7. The method of claim 1, wherein the input sequence under detection of the decoder includes padded zeroes inferred by a one forward procedure.

8. A non-transitory computer-readable storage medium comprising a computer-readable program for implementing a self-attentive encoder-decoder transformer framework for anomaly detection in event sequences, wherein the computer-readable program when executed on a computer causes the computer to perform the steps of: feeding event content information into a content-awareness layer to generate event representations; inputting, into an encoder, event sequences of two hierarchies to capture long-term and short-term patterns and to generate feature maps; adding, in the decoder, a special sequence token at a beginning of an input sequence under detection; during a training stage, applying a one-class objective to bound the decoded special sequence token with a reconstruction loss for sequence forecasting using the generated feature maps from the encoder; and during a testing stage, labeling any event representation whose decoded special sequence token lies outside a hypersphere as an anomaly.

9. The non-transitory computer-readable storage medium of claim 8, wherein the decoder embeds the event sequences into a latent space where anomalies are distinguishable.

10. The non-transitory computer-readable storage medium of claim 8, wherein the special sequence token represents event sequence status.

11. The non-transitory computer-readable storage medium of claim 8, wherein the encoder includes attention blocks, 1-D convolutional filters with activation functions, and MaxPool layers to downsample the inputted event sequences of the two hierarchies.

12. The non-transitory computer-readable storage medium of claim 8, wherein the decoder includes a masked self-attention layer to preserve an auto-regressive property.

13. The non-transitory computer-readable storage medium of claim 8, wherein the decoder performs a one-time interference to predict all events.

14. The non-transitory computer-readable storage medium of claim 8, wherein the input sequence under detection of the decoder includes padded zeroes inferred by a one forward procedure.

15. A system for implementing a self-attentive encoder-decoder transformer framework for anomaly detection in event sequences, the system comprising: a memory; and one or more processors in communication with the memory configured to: feed event content information into a content-awareness layer to generate event representations; input, into an encoder, event sequences of two hierarchies to capture long-term and short-term patterns and to generate feature maps; add, in the decoder, a special sequence token at a beginning of an input sequence under detection; during a training stage, apply a one-class objective to bound the decoded special sequence token with a reconstruction loss for sequence forecasting using the generated feature maps from the encoder; and during a testing stage, label any event representation whose decoded special sequence token lies outside a hypersphere as an anomaly.

16. The system of claim 15, wherein the decoder embeds the event sequences into a latent space where anomalies are distinguishable.

17. The system of claim 15, wherein the special sequence token represents event sequence status.

18. The system of claim 15, wherein the encoder includes attention blocks, 1-D convolutional filters with activation functions, and MaxPool layers to downsample the inputted event sequences of the two hierarchies.

19. The system of claim 15, wherein the decoder includes a masked self-attention layer to preserve an auto-regressive property.

20. The system of claim 15, wherein the decoder performs a one-time interference to predict all events.