Data Clustering for Machine Learning Model-Based Anomaly Prediction

Many industrial apparatuses are interconnected systems-of-systems having designs that are increasingly complicated and are susceptible to malfunction or failure for many different reasons. The larger a system-of-systems based apparatus is, the more difficult, costly and inefficient it can become to identify and troubleshoot the sources of anomalous apparatus operation. Conventional solutions for anticipating apparatus malfunctions and failures have relied upon the deep knowledge base of highly experienced system engineers, which, due to the heavy reliance of those solutions on the expertise of particular individuals, are brittle and ultimately untenable. Moreover, as the amount of monitoring data required to adequately profile the operating state of a system-of-systems based apparatus grows with the increasing complexity of such an apparatus, human review and interpretation of that data becomes ever more impracticable. Consequently, there is a need in the art for an automated solution for diagnosing the operational states of complex apparatuses and accurately predicting situations in which those apparatuses are likely to malfunction or fail.

The following description contains specific information pertaining to implementations in the present disclosure. One skilled in the art will recognize that the present disclosure may be implemented in a manner different from that specifically discussed herein. The drawings in the present application and their accompanying detailed description are directed to merely exemplary implementations. Unless noted otherwise, like or corresponding elements among the figures may be indicated by like or corresponding reference numerals. Moreover, the drawings and illustrations in the present application are generally not to scale, and are not intended to correspond to actual relative dimensions.

As stated above, many industrial apparatuses are interconnected systems-of-systems having designs that are increasingly complicated and are susceptible to malfunction or failure for many different reasons. The larger a system-of-systems based apparatus is, the more difficult, costly and inefficient it can become to identify and troubleshoot the sources of anomalous apparatus operation. Conventional solutions for anticipating apparatus malfunctions and failures have relied upon the deep knowledge base of highly experienced system engineers, which, due to the heavy reliance of those solutions on the expertise of particular individuals, are brittle and ultimately untenable. Moreover, and as also stated above, as the amount of monitoring data required to adequately profile the operating state of a system-of-systems based apparatus grows with the increasing complexity of such an apparatus, human review and interpretation of that data becomes ever more impracticable.

The present application discloses systems and methods for performing data clustering for machine learning (ML) model-based anomaly prediction that address and overcome the drawbacks and deficiencies in the conventional art by disclosing an automated solution for diagnosing the operational states of complex apparatuses and accurately predicting situations in which those apparatuses are likely to malfunction or fail. The present solution recognizes that even complex apparatuses are designed to follow an expected operating pattern that is programmed. Sensors may be outfitted on, in, or adjacent to apparatus components that allow for these operating patterns to be tracked through data.

The present data clustering for ML model-based anomaly prediction systems and methods advance the state-of-the-art by identifying and extracting data relevant to the operating state of an apparatus from a dataset, which may be or include a binary time series dataset including billions of data points, for example, processing that relevant data to generate performance data for the apparatus, and using a trained ML model to predict whether the apparatus is operating anomalously based on that performance data in real-time with respect to generation of the dataset. Furthermore, in some implementations, the present solution for performing data clustering for ML model-based anomaly prediction may include performing an intervention when an anomalous operating state of an apparatus is predicted. Such an intervention may include sounding an alarm at the apparatus, shutting down operation of the apparatus, or modifying the performance of the apparatus, to name a few examples.

It is noted that the present ML model-based anomaly prediction solution is distinct from dimensionality reduction approaches, such as Principle Component Analysis (PCA) for example, in which the original data is reduced into a smaller subset. By contrast to PCA, in which data is compressed for analysis, the anomaly prediction solution disclosed by the present application effectively expands collected data by clustering data points through the identification of meaningful correspondences between sensor data produced by different sensors, based on timing. As a result, the present anomaly prediction solution can provide better insight into the operational state of a physical apparatus than when a dimensionality reduction analytical framework such as PCA is applied.

It is further noted that due to the complexity of the apparatuses for which anomaly prediction is performed by the systems and methods disclosed herein, i.e., the number and interrelatedness of the components and subsystems comprised by each apparatus, reliance upon a trained ML model to accurately predict anomalous operation is essential. Thus, due to the complexity of the operation of the apparatuses subject to the present anomaly prediction solution, the present proactively predictive method is incapable of being performed as a mental process by a human mind, even with the aid of a general purpose computer. That is to say, while conventional attempts to predict anomalous operation by complex apparatuses may result in such predictions lagging the generation of performance data by hours, days or weeks, or even being performed as a diagnosis rather than prediction, in a forensic process post hoc to malfunction or failure of the system, the present ML model-based solution improves the technical field of mechanical troubleshooting and maintenance by advantageously enabling corrective interventions in real-time before apparatus failures or significant malfunctions occur. For example, a notification predicting anomalous operation by an apparatus may be output by the present system and according to the present method within up to ten seconds of receipt by the system of performance data for the apparatus, while a corrective intervention may be initiated within seconds or minutes of outputting the notification.

As defined in the present application, the expression “ML model” refers to a computational model for making predictions based on patterns learned from samples of data or training data. Various learning algorithms can be used to map correlations between input data and output data. These correlations form the computational model and can be used to make future predictions on new input data. Such a predictive model may include one or more logistic regression models, Bayesian models, artificial neural networks (NNs) such as Transformers, large-language models (LLMs), or multimodal foundation models, to name a few examples. In various implementations, ML models may be trained as classifiers and may be utilized to perform image processing, audio processing, natural-language processing, and other inferential analyses. A “deep neural network,” in the context of deep learning, may refer to a NN that utilizes multiple hidden layers between input and output layers, which may allow for learning based on features not explicitly defined in raw data. As used in the present application, a feature identified as a NN refers to a deep neural network.

It is further noted that the performance data provided by the systems and using the methods disclosed in the present application can be used to produce interpretable visualizations identifying where and when the operation of components of an apparatus are deviating from expectations. In addition, the present anomaly prediction solution may advantageously be implemented as automated systems and methods. As used in the present application, the terms “automation,” “automated” and “automating” refer to systems and processes that do not require the participation of a human system operator. Thus, the methods described in the present application may be performed under the control of hardware processing components of the disclosed systems.

1 FIG. 1 FIG. 100 100 102 104 106 108 106 110 112 112 130 114 shows exemplary systemfor performing data clustering for ML model-based anomaly prediction, according to one implementation. As shown in, systemincludes computing platformhaving hardware processor, memoryimplemented as a computer-readable non-transitory storage medium, and transceiver. According to the present exemplary implementation, memorystores software code, one or more trained ML models(hereinafter “trained ML model(s)”) and sensor databaseincluding a plurality of database entries each associating a respective subset of a plurality of sensors with one of plurality of apparatuses.

112 112 112 With respect to trained ML model(s), it is noted that in some implementations trained ML model(s)may include one or more statistical models based on Interquartile Range (IQR), shape analytics (e.g., kurtosis and/or wavelets), or auto-regression, to name a few examples. Alternatively, or in addition, trained ML model(s)may include one or more K-Means models, K-Nearest Neighbors (KNN) models, deep learning NNs (DNNs) such as Long Short-Term Memory (LSTM) models, Density-Based Spatial Clustering of Applications with Noise (DBSCAN) models, Gaussian Mixture models, Spectral clustering models, Time Series Transformer models, or any combination thereof.

1 FIG. 1 FIG. 100 114 124 134 134 124 120 122 100 114 140 150 132 140 100 126 114 128 136 112 138 112 136 160 100 114 160 136 As further shown in, systemis implemented within a use environment that includes plurality of apparatusesincluding apparatus, one or more sensors(hereinafter “sensor(s)”) used to monitor apparatus, and communication networkproviding network communication linkscommunicatively coupling systemwith plurality of apparatusesand portable deviceincluding display. Also shown inis system userutilizing portable deviceto interact with system, as well as global sensor dataincluding data generated by all active sensors of the plurality of sensors used to monitor plurality of apparatuses, clustered sensor data, performance dataprovided as input to trained ML model(s), predictionprovided by trained ML model(s)based on performance dataand notificationoutput by systemwhen an anomalous operating state of one or more of plurality of apparatusesis predicted. It is noted that notificationmay include a visual representation of performance dataon which the prediction of anomalous operation is based.

132 114 126 114 114 126 114 126 120 122 114 1 FIG. System usermay be an engineer or programmer tasked with evaluating the operational performance of one or more of plurality of apparatuses. Moreover, in some use cases, global sensor datamay describe a past operation of plurality of apparatuses, and may be utilized to perform a forensic analysis of a historical performance of one or more of plurality of apparatuses. However, in other use cases, global sensor datamay be received dynamically while one or more of plurality of apparatusesis/are in operation. In some implementations, as shown in, global sensor datamay be received via communication networkand network communication links. It is noted that in various implementations, plurality of apparatusesmay include one or more of a baggage claim carousel or carousels, an automated warehouse or warehouses, a theme park attraction or attractions, a vehicle assembly line machine or machines, an aviation system or systems, a HVAC system or systems, an engine or engines, manufacturing equipment, or a computer server or servers, to name a few examples.

134 114 114 114 114 114 114 114 134 134 The plurality of sensors, including sensor(s), used to monitor plurality of apparatusesmay include one or more sensors affixed to plurality of apparatuses, internal to plurality of apparatuses, situated adjacent to any of plurality of apparatuseswithin a venue housing any of plurality of apparatuses, or situated adjacent to any of plurality of apparatusesin an outdoor space occupied by any of plurality of apparatuses. That plurality of sensors including sensor(s)may include one or more cameras, one or more audio microphones, one or more temperature sensors, one or more pressure sensors, one or more vibration sensors, one or more chemical sensors, one or more timing devices, or any combination thereof. The plurality of sensors including sensor(s)may be configured to detect one or more of visual images, sounds, vibrations, heat, pressure, time duration, air quality or smells (aromas).

110 112 130 106 106 104 102 Although the present application refers to software code, trained ML model(s)and sensor databaseas being stored in memoryfor conceptual clarity, more generally, memorymay take the form of any computer-readable non-transitory storage medium. The expression “computer-readable non-transitory storage medium,” as defined in the present application, refers to any medium, excluding a carrier wave or other transitory signal, that provides instructions to hardware processorof computing platform. Thus, a computer-readable non-transitory storage medium may correspond to various types of media, such as volatile media and non-volatile media, for example. Volatile media may include dynamic memory, such as dynamic random access memory (dynamic RAM), while non-volatile memory may include optical, magnetic, or electrostatic storage devices. Common forms of computer-readable non-transitory storage media include, for example, internal and external hard drives, optical discs, RAM, programmable read-only memory (PROM), erasable PROM (EPROM) and FLASH memory.

100 106 Moreover, in some implementations, systemmay utilize a decentralized secure digital ledger in addition to memory. Examples of such decentralized secure digital ledgers may include a blockchain, hashgraph, directed acyclic graph (DAG), and Holochain® ledger, to name a few. In use cases in which the decentralized secure digital ledger is a blockchain ledger, it may be advantageous or desirable for the decentralized secure digital ledger to utilize a consensus mechanism having a proof-of-stake (PoS) protocol, rather than the more energy intensive proof-of-work (PoW) protocol.

1 FIG. 1 FIG. 110 112 130 106 100 102 104 106 100 110 112 130 100 112 130 100 106 130 100 120 122 It is further noted that althoughdepicts software code, trained ML model(s)and sensor databaseas being stored together in a single instance of memory, that representation is merely provided as an aid to conceptual clarity. More generally, systemmay include one or more computing platforms, such as computer servers for example, which may be co-located, or may form an interactively linked but distributed system, such as a cloud-based system, for instance. As a result, hardware processorand memorymay correspond to distributed processor and memory resources within system, and one or more of software code, trained ML model(s)and sensor databasemay be stored remotely from one another on the distributed memory resources of system. Furthermore, althoughdepicts trained ML model(s)and sensor databaseas components of systemstored in memory, in some implementations one or more of sensor databaseand trained ML model(s) may be remote resources accessible by systemvia communication networkand network communication links.

104 102 110 106 Hardware processormay include a plurality of hardware processing units, such as one or more central processing units, one or more graphics processing units, and one or more tensor processing units, one or more field-programmable gate arrays (FPGAs), custom hardware for machine learning training or inferencing, and an application programming interface (API) server, for example. By way of definition, as used in the present application, the terms “central processing unit” (CPU), “graphics processing unit” (GPU), and “tensor processing unit” (TPU) have their customary meaning in the art. That is to say, a CPU includes an Arithmetic Logic Unit (ALU) for carrying out the arithmetic and logical operations of computing platform, as well as a Control Unit (CU) for retrieving programs, such as software code, from memory, while a GPU may be implemented to reduce the processing overhead of the CPU by performing computationally intensive graphics or other processing tasks. A TPU is an application-specific integrated circuit (ASIC) configured specifically for artificial intelligence (AI) applications such as ML modeling.

108 100 108 108 Transceiverof systemmay be implemented as a wireless communication unit configured for use with one or more of a variety of wireless communication protocols. For example, transceivermay include a fourth generation (4G) wireless transceiver and/or a 5G wireless transceiver. In addition, or alternatively, transceivermay be configured for communications using one or more of Wireless Fidelity (Wi-Fi®), Worldwide Interoperability for Microwave Access (WiMAX®), Bluetooth®, Bluetooth® low energy (BLE), ZigBee®, radio-frequency identification (RFID), near-field communication (NFC), and 60 GHz wireless communications methods.

102 102 100 100 100 120 In some implementations, computing platformmay correspond to one or more web servers, accessible over a packet-switched network such as the Internet, for example. Alternatively, computing platformmay correspond to one or more computer servers supporting a private wide area network (WAN), local area network (LAN), or included in another type of limited distribution or private network. In addition, or alternatively, in some implementations, systemmay utilize a local area broadcast method, such as User Datagram Protocol (UDP) or Bluetooth®, for instance. Furthermore, in some implementations, systemmay be implemented virtually, such as in a data center. For example, in some implementations, systemmay be implemented in software, or as virtual machines. Moreover, in some implementations, communication networkmay be a high-speed network suitable for high performance computing (HPC), for example a 10 GigE network or an Infiniband network.

140 140 140 150 150 Portable devicemay take the form of a smartphone, or any other suitable portable computing system that implements data processing capabilities sufficient to provide a user interface, and implement the functionality attributed to portable deviceherein. For example, in other implementations, portable devicemay take the form of a tablet computer, laptop computer, or an augmented reality (AR) or virtual reality (VR) device, for example, providing display. Displaymay take the form of a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting diode (OLED) display, a quantum dot (QD) display, or any other suitable display screen that performs a physical transformation of signals to light.

2 FIG. 2 FIG. 240 240 244 248 250 246 210 212 212 230 shows a more detailed diagram of portable device, according to one implementation. As shown in, portable deviceincludes hardware processor, transceiver, displayand portable device memoryimplemented as a computer-readable non-transitory storage medium storing software code, one or more trained ML models(hereinafter “trained ML model(s)”) and sensor database.

2 FIG. 2 FIG. 240 201 214 224 234 234 224 220 222 240 214 226 214 228 236 212 238 212 236 260 240 214 As further shown in, portable deviceis utilized in use environmentthat includes plurality of apparatusesincluding apparatus, one or more sensors(hereinafter “sensor(s)”) used to monitor apparatus, and communication networkproviding network communication linkscommunicatively portable devicewith plurality of apparatuses. Also shown inare global sensor dataincluding data generated by all active sensors of the plurality of sensors used to monitor plurality of apparatuses, clustered sensor data, performance dataprovided as input to trained ML model(s), predictionprovided by trained ML model(s)based on performance dataand notificationoutput by portable devicewhen an anomalous operating state of one or more of plurality of apparatusesis predicted.

214 224 234 220 222 114 124 134 120 122 214 224 234 220 222 114 124 134 120 122 114 214 234 214 114 114 114 114 114 114 114 134 214 234 1 FIG. Plurality of apparatusesincluding apparatus, sensor(s)and communication networkproviding network communication linkscorrespond respectively in general to plurality of apparatusesincluding apparatus, sensor(s)and communication networkproviding network communication links, in. Consequently, plurality of apparatusesincluding apparatus, sensor(s)and communication networkproviding network communication linksmay share any of the characteristics attributed to respective plurality of apparatusesincluding apparatus, sensor(s)and communication networkproviding network communication links, and vice versa. That is to say, in various implementations, like plurality of apparatuses, plurality of apparatusesmay include one or more of a baggage claim carousel or carousels, an automated warehouse or warehouses, a theme park attraction or attractions, a vehicle assembly line machine or machines, an aviation system or systems, a HVAC system or systems, an engine or engines, manufacturing equipment, or a computer server or servers, to name a few examples, while the plurality of sensors, including sensor(s), used to monitor plurality of apparatusesmay include one or more sensors affixed to plurality of apparatuses, internal to plurality of apparatuses, situated adjacent to any of plurality of apparatuseswithin a venue housing any of plurality of apparatuses, or situated adjacent to any of plurality of apparatusesin an outdoor space occupied by any of plurality of apparatuses. Moreover, like plurality of sensorsincluding sensor(s), plurality of sensorsincluding sensor(s)may include one or more cameras, one or more audio microphones, one or more temperature sensors, one or more pressure sensors, one or more vibration sensors, one or more chemical sensors, one or more timing devices, or any combination thereof.

226 228 236 238 260 126 128 136 138 160 226 228 236 238 260 126 128 136 138 160 2 FIG. 1 FIG. In addition, global sensor data, clustered sensor data, performance data, predictionand notification, in, correspond respectively in general to global sensor data, clustered sensor data, performance data, predictionand notification, in. As a result, global sensor data, clustered sensor data, performance data, predictionand notificationmay share any of the characteristics attributed to respective global sensor data, clustered sensor data, performance data, predictionand notificationby the present disclosure, and vice versa.

240 250 140 150 240 250 140 150 140 240 250 150 250 140 244 248 246 210 212 230 1 FIG. 1 FIG. Portable deviceand displaycorrespond respectively in general to portable deviceand display, in. Thus, portable deviceand displaymay share any of the characteristics attributed to respective portable deviceand displayby the present disclosure, and vice versa. For example, like portable device, portable devicemay take the form of a smartphone, tablet computer, laptop computer, or an AR or VR device, for example, providing display. In addition, like display, displaymay take the form of an LCD, LED display, OLED display, or QD display. Moreover, although not shown in, portable devicemay include features corresponding respectively to hardware processor, transceiverand portable device memorystoring software code, trained ML model(s)and sensor database.

248 248 248 Transceivermay be implemented as a wireless communication unit configured for use with one or more of a variety of wireless communication protocols. For example, transceivermay include a 4G wireless transceiver and/or a 5G wireless transceiver. In addition, or alternatively, transceivermay be configured for communications using one or more of Wi-Fi®, WiMAX®, Bluetooth®, BLE, ZigBee®, RFID, NFC, and 60 GHz wireless communications methods.

244 240 Hardware processorof portable devicemay include multiple hardware processing units, such as one or more CPUs, one or more GPUs, one or more TPUs, and one or more FPGAs, as those features are defined above.

210 110 110 244 240 210 246 212 230 240 100 100 140 240 1 FIG. Software codecorresponds in general to software code, in, and can perform all of the operations attributed to software codeby the present disclosure. In other words, in implementations in which hardware processorof portable deviceexecutes software codestored locally in portable device memoryto utilize trained ML model(s)and sensor database, portable devicemay perform any of the actions attributed to systemby the present disclosure. Thus, in some implementations, systemmay be embodied in portable device/.

3 FIG. 3 FIG. 3 FIG. 3 FIG. 300 328 328 328 328 328 328 328 328 328 328 a b c d a d a d a d shows diagramshowing clustered sensor data,,and(hereinafter “clustered sensor data-”) for an exemplary cluster of sensor data used to perform ML model-based anomaly prediction for an apparatus, according to one implementation. According to the exemplary implementation shown in, the sensors producing clustered sensor data-are digital sensors and each of clustered sensor data-represents a respective binary time series generated by a single sensor. It is noted that althoughdepicts sensor data produced by digital sensors, in various implementations the present anomaly prediction solution may be adapted to use sensor data produced by analog sensors, or sensor data that is a combination of digital and analog sensor data. It is further noted that althoughdepicts sensor data produced by four different sensors, that representation is merely provided as an example. In various use cases, sensor data from any desired plurality of sensors may be clustered to perform the novel and inventive ML model-based anomaly prediction approach disclosed in the present application. For example, in other use cases, sensor data for as few as two sensors may be clustered to perform ML model-based anomaly prediction, sensor data for three sensors may be clustered to perform ML model-based anomaly prediction, or sensor data for more than four sensors may be clustered to perform ML model-based anomaly prediction.

328 328 a d Regarding the identification of what sensor data to include in clustered sensor data-and what sensor data to omit, that identification may be performed manually, i.e., sensors to be clustered could be identified by a system user, or may be performed in an automated process. When performed in an automated process, the identification of which sensors to cluster may be based on a moving window approach using different time intervals to identify sensor data that includes a rising edge or a falling edge within a predetermined time interval of a rising edge or falling edge of sensor data produced by another sensor. That process could be performed for a plurality of different time intervals, such as 0.1s, 0.5s, 1.0s, 10s, and 30s, for example, and a cluster based one on or more of rising edge to rising edge, rising edge to falling edge, falling edge to falling edge, and falling edge to rising edge could be determined. A plurality of clusters of sensors could be determined in this way.

328 328 128 228 328 328 328 328 328 328 328 328 328 328 328 a d a b c a c d a c d b c. 1 2 FIGS.and 3 FIG. Clustered sensor data-corresponds in general to clustered sensor data/in, and each of those corresponding features may share any of the characteristics attributed to any of those corresponding features by the present disclosure. According to the exemplary use case shown in, clustered sensor data,and(hereinafter “clustered sensor data-”) are generated by different sensors used to monitor the same apparatus, while clustered sensor datais generated by a sensor used to monitor a different apparatus, but is clustered with clustered sensor data-due to a timing correspondence by clustered sensor datawith clustered sensor dataand

1 2 3 FIGS.,and 328 328 134 234 124 224 328 114 214 362 328 364 362 364 328 362 328 a c d a a a b b b c c By way merely of example, and referring toin combination, let it be assumed that clustered sensor data-are generated by sensor(s)/used to monitor apparatus/in the form of a theme park ride, and that clustered sensor datais generated by a sensor used to monitor another of plurality of apparatuses/, which may be another theme park ride or any other type of apparatus. Let it further be assumed that falling edgeof clustered sensor datacorresponds to a command to close entry gates to a ride vehicle, while rising edgecorresponds to a command to open those gates. In addition, let it also be assumed that falling edgeand rising edgeof clustered sensor datacorrespond to the respective closing and opening of a first entry gate of the ride vehicle, and that falling edgeof clustered sensor datacorresponds to the closing of a last entry gate of the ride vehicle.

328 124 224 364 328 362 328 362 328 364 328 336 362 362 336 362 362 336 362 362 d d d c c d d b b ab a b ac a c bc b c 3 FIG. Although clustered sensor datamay be unrelated to the operation of apparatus/, it is noted that rising edgeof clustered sensor datacoincides with falling edgeof clustered sensor data, and that falling edgeof clustered sensor datacoincides with rising edgeof clustered sensor data. Also shown inare delta timebetween the command to close the entry gates to the ride vehicle, the command corresponding to falling edge, and falling edgeindicating closure of the first entry gate of the ride vehicle; delta timebetween the command to close entry gates to the ride vehicle corresponding to falling edgeand falling edgeindicating closure of the last entry gate of the ride vehicle; and delta timebetween falling edgeindicating closure of the first entry gate of the ride vehicle and falling edgeindicating closure of the last entry gate of the ride vehicle.

4 4 4 FIGS.A,B andC 3 FIG. 3 4 FIGS.andA 3 4 FIGS.andB 4 FIG.C 466 466 466 436 436 436 436 336 436 336 436 336 ab ac bc ab ab ac ac bc bc show exemplary visual representationA,B andC of respective performance data,andgenerated based on the cluster of clustered sensor data shown in, according to one implementation. Referring toin combination, performance datais the statistical distribution of variations in delta time(in milliseconds) over a plurality of commands to close entry gates to the ride vehicle and responsive first entry gate closures of the ride vehicle. Analogously, and referring toin combination, performance datais the statistical distribution of variations in delta timeover a plurality of commands to close entry gates to the ride vehicle and responsive last entry gate closures of the ride vehicle, while performance data, shown in, is the statistical distribution of variations in delta timebetween the first entry gate closure of the ride vehicle in response to the command to close entry gates to the ride vehicle and the last entry gate closure of the ride vehicle in response to the same command to close entry gates to the ride vehicle for a plurality of such closures.

436 436 436 136 236 336 336 336 436 436 436 436 436 436 136 236 362 328 364 328 364 328 362 328 124 224 ab ac bc ab ac bc ab ac bc ab ac bc c c d d b b d d 1 2 FIGS.and 3 FIG. 4 4 4 FIGS.A,B andC 3 FIG. 3 FIG. It is noted that performance data,and, collectively, correspond in general to performance data/, in, and also correspond respectively in general to delta times,and, in. It is further noted that the specific instantiations of performance data represented by performance data,and, inare provided merely as examples. In some use cases, in addition to, or as alternatives to one or more of performance data,and, performance data/may include variations in delta times between falling edgeof clustered sensor dataand rising edgeof clustered sensor data(shown to coincide in), variations in delta times between rising edgeof clustered sensor dataand falling edgeof clustered sensor data(also shown to coincide in), or any other metric determined to be relevant to predicting anomalous operation by apparatus/.

100 140 240 110 210 112 212 130 230 570 570 1 2 FIGS.and 5 FIG. 5 FIG. 5 FIG. The functionality of systemand portable device/, software code/, trained ML model(s)/and sensor database/, shown in, will be further described by reference to.shows flowchartpresenting an exemplary method for performing data clustering for ML model-based anomaly prediction, according to one implementation. With respect to the method outlined in, it is noted that certain details and features have been left out of flowchartin order not to obscure the discussion of the inventive features in the present application.

5 FIG. 1 2 FIGS.and 570 126 226 134 234 114 214 571 114 214 Referring to, with further reference to, flowchartincludes receiving global sensor data/generated by all active sensors of a plurality of sensors, including sensor(s)/, used to monitor plurality of apparatuses/(action). As noted above, in various implementations plurality of apparatuses/may include one or more of a baggage claim carousel or carousels, an automated warehouse or warehouses, a theme park attraction or attractions, a vehicle assembly line machine or machines, an aviation system or systems, a HVAC system or systems, an engine or engines, manufacturing equipment, or a computer server or servers, to name a few examples.

126 126 226 114 214 114 214 126 226 114 214 114 214 114 214 126 226 114 214 126 226 1 FIG. As noted above by reference to global sensor data, in, in some use cases, global sensor data/may describe a past operation of plurality of apparatuses/, and may be utilized to perform a forensic analysis of a historical performance of one or more of plurality of apparatuses/. However, in other use cases, global sensor data/may be received dynamically while one or more of plurality of apparatuses/is/are in operation and may be used to predict whether one or more of plurality of apparatuses/is in need of maintenance to correct anomalous operation by that/those one or more of plurality of apparatuses/. It is noted that global sensor data/may be or include a plurality of binary time series each generated by a respective one of the active sensors of the plurality of sensors used to monitor plurality of apparatuses/, and that global sensor data/may include tens of millions, hundreds of millions, or a billion or billions of data points

1 FIG. 2 FIG. 126 114 120 122 571 110 104 100 226 214 220 222 571 210 244 240 In some implementations, as shown in, global sensor datamay be received from active sensors of the plurality of sensors used to monitor plurality of apparatusesvia communication networkand network communication links, in action, by software code, executed by hardware processorof system. In other implementations, as shown in, global sensor datamay be received from active sensors of the plurality of sensors used to monitor plurality of apparatusesvia communication networkand network communication links, in action, by software code, executed by hardware processorof portable device.

134 234 114 214 114 214 114 214 114 214 114 114 114 134 234 As further noted above, the plurality of sensors, including sensor(s)/, used to monitor plurality of apparatuses/may include one or more digital and/or analog sensors affixed to plurality of apparatuses/, internal to plurality of apparatuses/, or situated adjacent to any of plurality of apparatuses/within a venue housing any of apparatuses, or situated adjacent to any of plurality of apparatusesin an outdoor space occupied by any of plurality of apparatuses. That plurality of digital and/or analog sensors including sensor(s)/may include one or more cameras, one or more audio microphones, one or more temperature sensors, one or more pressure sensors, one or more vibration sensors, one or more chemical sensors, one or more timing devices, or any combination thereof.

5 FIG. 1 2 3 FIGS.,and 570 114 214 124 224 126 226 124 224 130 230 328 328 114 214 124 224 124 224 134 234 572 328 328 126 572 110 104 100 328 328 226 572 210 244 240 a c a c a c Referring toin combination with, flowchartfurther includes, for a first apparatus of plurality of apparatuses/(the first apparatus hereinafter referred to as “apparatus/”), extracting from global sensor data/, based on a database entry for apparatus/obtained from sensor database/, first sensor data (hereinafter “clustered sensor data-”) generated by a first subset of the plurality of sensors used to monitor plurality of apparatuses/that are associated with apparatus/(the first subset of sensors associated with apparatus/hereinafter referred to as “sensor(s)/”) (action). In some implementations, clustered sensor data-may be extracted from global sensor data, in action, by software code, executed by hardware processorof system. However, in other implementations, clustered sensor data-may be extracted from global sensor data, in action, by software code, executed by hardware processorof portable device.

5 FIG. 1 2 3 FIGS.,and 3 FIG. 1 2 FIGS.and 570 124 224 126 226 328 114 214 134 234 134 234 573 328 328 328 128 228 364 328 362 328 362 328 364 328 d d a c d d c c d d b b. Continuing to refer toin combination with, flowchartfurther includes, for apparatus/, identifying, using global sensor data/, a first other sensor data (hereinafter “clustered sensor data”) generated by a sensor of the plurality of sensors used to monitor apparatuses/not included among sensor(s)/within a respective predetermined time interval of an expected timing of at least one sensor of sensor(s)/(action). As noted above by reference to, clustered sensor datais included with clustered sensor data-in clustered sensor data/indue to the timing relationship between rising edgeof clustered sensor datawith falling edgeof clustered sensor data, and the timing relationship between falling edgeof clustered sensor dataand rising edgeof clustered sensor data

3 FIG. 364 328 362 328 362 328 364 328 328 328 328 328 328 328 d d c c d d b b d a c d a c. It is noted that although thedepicts rising edgeof clustered sensor dataas coinciding with falling edgeof clustered sensor data, and falling edgeof clustered sensor dataas coinciding with rising edgeof clustered sensor data, those timing relationships are merely provided as examples. More generally, a relevant timing relationship between clustered sensor dataand one or more of clustered sensor data-may exist where a rising or falling edge of clustered sensor dataoccurs within a predetermined time interval preceding or following either a rising edge or a falling edge of any of clustered sensor data-

328 573 110 104 100 328 573 210 244 240 d d 3 FIG. In some implementations, clustered sensor datamay be identified, in action, by software code, executed by hardware processorof system. However, in other implementations, clustered sensor datamay be identified, in action, by software code, executed by hardware processorof portable device, as described above by reference to.

5 FIG. 1 2 3 4 FIGS.,,and 1 FIG. 2 FIG. 570 128 228 328 328 136 236 436 436 436 124 224 574 128 136 124 574 110 104 100 228 236 224 574 210 244 240 a d ab ac bc Referring toin combination with, flowchartfurther includes processing clustered sensor data//-to provide performance data//(,,) for apparatus/(action). In some implementations, as shown in, processing of clustered sensor datato provide performance datafor apparatus, in action, may be performed by software code, executed by hardware processorof system. However, as shown in, in other implementations, processing of clustered sensor datato provide performance datafor apparatus, in action, may be performed software code, executed by hardware processorof portable device.

130 230 328 328 134 234 128 228 328 328 574 136 236 436 436 436 328 328 336 336 336 328 328 134 234 328 328 328 574 336 336 336 364 328 362 328 362 328 364 328 a c a d ab ac bc a c ab ac bc a c d a c ab ac bc d d c c d d b b. 3 FIG. In some implementations, each of the plurality of database entries included in sensor database/may further identify an expected timing relationship for clustered sensor data-generated by sensor(s)/. In those implementations, the processing of clustered sensor data//-, in action, to provide performance data//(,,) may include determining a timing relationship among the data points included in clustered sensor data-(e.g., delta times,and), comparing the timing relationship among the data points included in clustered sensor data-with an expected timing relationship for data generated by sensor(s)/, and comparing a timing interval between the generation of clustered sensor dataand an expected timing of at least one sensor in clustered sensor data-with the respective predetermined time interval. That is to say, referring to the specific example shown in, in some implementations, actionmay include comparing delta times,andwith expected values for those time intervals, and comparing the timing of rising edgeof clustered sensor datawith the expected timing of falling edgeof clustered sensor data, and comparing the timing of falling edgeof clustered sensor datawith the expected timing of rising edgeof selected sensor content

5 FIG. 1 2 3 4 FIGS.,,and 1 FIG. 2 FIG. 570 112 212 136 236 436 436 436 124 224 575 112 212 136 236 436 436 436 138 238 575 110 104 100 112 138 124 575 210 244 240 212 238 224 ab ac bc ab ac bc Continuing to refer toin combination with, flowchartfurther includes predicting, using trained ML model(s)/and performance data//(,,), whether apparatus/is operating anomalously (action). It is noted that ML model(s)/may be trained to receive performance data//(,,) as input, and to provide prediction/as an output. In some implementations, as shown in, actionmay be performed by software code, executed by hardware processorof system, and using trained ML model(s)to provide predictionregarding anomalous operation by apparatus. Alternatively, and as shown in, in some implementations actionmay be performed by software code, executed by hardware processorof portable device, and using trained ML model(s)to provide predictionregarding anomalous operation by apparatus.

5 FIG. 1 2 4 FIGS.,and 1 FIG. 570 575 124 224 160 260 466 466 466 466 466 466 136 236 436 436 436 576 160 466 466 466 576 100 160 140 120 122 576 110 104 100 ab ac bc Referring toin combination with, flowchartfurther includes, outputting, when the predicting performed in actionidentifies apparatus/as operating anomalously, notification/including one or more of visual representationsA,B andC (hereinafter “visual representation(s)A/B/C”) of performance data//(,,) (action). In some implementations, as shown in, outputting notificationincluding visual representation(s)A/B/C, in action, may include transmitting, by system, notificationto portable devicevia communication networkand network communication links. In those implementations, actionmay be performed by software code, executed by hardware processorof system.

2 FIG. 260 466 466 466 576 260 466 466 466 250 240 576 260 250 576 210 244 240 However, in other implementations, as shown in, outputting notificationincluding visual representation(s)A/B/C, in actionmay include outputting notificationincluding visual representation(s)A/B/C to displayof portable device. In implementations in which actionincludes outputting notificationto display, actionmay be performed by software code, executed by hardware processorof portable device.

576 100 140 240 160 260 466 466 466 126 226 571 160 260 466 466 466 132 576 126 226 100 140 240 571 Whether actionis performed by systemor portable device/, in some implementations, notification/including visual representation(s)A/B/C may be output in real-time with respect to receiving global sensor data/in action. It is noted that, as defined for the purposes of the present application, the expression “real-time” refers to latency of a few seconds, such as up to ten seconds, or less. Thus, in some implementations, notification/including visual representation(s)A/B/C may be provided to system user, in action, within ten seconds or less of receipt of global sensor data/by systemor portable device/in action.

570 576 124 224 244 240 210 466 466 466 136 236 436 436 436 124 224 250 240 2 4 FIGS.and ab ac bc It is noted that, in some implementations, the method outlined by flowchartmay conclude with actiondescribed above for apparatus/. However, and referring toin combination, in other implementations, hardware processorof portable devicemay further execute software codeto render visual representation(s)A/B/C of performance data//(,,) for apparatus/on displayof portable device.

5 FIG. 571 572 573 574 575 571 575 571 575 576 571 576 571 575 571 576 With respect to the method outlined by, it is emphasized that actions,,,and(hereinafter “actions-”), or actions-and action(hereinafter “actions-”), may be performed in an automated process from which human involvement may be omitted. Moreover, such an automated process including actions-or actions-may be performed iteratively with a predetermined periodicity, such as a predetermined periodicity of less than one minute, for example.

572 573 574 575 572 575 572 575 576 124 224 572 575 572 575 576 114 214 126 226 571 104 100 110 244 240 210 572 575 572 575 576 114 214 124 224 126 226 In addition to performing actions,,and(hereinafter “actions-”), or actions-and action, for apparatus/, actions analogous to actions-or actions analogous to actions-andmay be performed for some or all other apparatuses included among plurality of apparatuses/using the same global sensor data/received in action. In other words, hardware processorof systemmay be configured to execute software code, or hardware processorof portable devicemay be configured to execute software code, to perform extracting, identifying, processing and predicting analogous to respective actions-, or to perform extracting, identifying, processing, predicting and outputting analogous to respective actions-andfor one, some, or all of plurality of apparatuses/, including apparatus/, subsequent to receiving global sensor data/.

112 212 112 212 571 576 100 140 240 132 104 100 110 112 244 140 240 210 212 112 212 It is noted that the operational performance of apparatuses may vary over the course of a day, i.e., apparatus performance at night may be different from apparatus performance during daylight hours, may vary seasonally, or may vary due to changes in environmental conditions such as temperature, humidity and the like. These variations in operational performance may result in a prediction of anomalous operation by an apparatus, despite that operational performance being normal in view of the prevailing circumstances. In order to reduce or eliminate false positive predictions of anomalous operation, it may be advantageous or desirable to use human review and correction or affirmation of predictions of anomalous operation to modify ML model(s)/, for example by using reinforcement learning to retrain ML model(s)/to more accurately identify which sensor data should be clustered when performing anomaly prediction. Thus, in some implementations, in addition to actions-, systemor portable device/may receive feedback data from usercorrecting or affirming a prediction of anomalous operation, and hardware processorof systemmay execute software codeto modify ML model(s), or hardware processorof portable device/may execute software codeto modify ML model(s), to improve the predictive performance of ML model(s)/over time.

571 575 571 576 104 100 110 244 140 240 210 114 214 112 212 100 140 240 100 140 240 100 240 Furthermore, in addition to actions-, or actions-, in some implementations, hardware processorof systemmay execute software code, or hardware processorof portable device/may execute software code, to perform an intervention when anomalous operation by one or more of plurality of apparatuses/is predicted by trained ML model(s)/. Such an intervention may include sounding an alarm, by systemor portable device/, at an apparatus operating anomalously, shutting down operation of an apparatus operating anomalously by systemor portable device/, or modifying the performance of an apparatus operating anomalously by systemor portable device, to name a few examples.

Thus, present application discloses systems and methods for performing data clustering for ML model-based anomaly prediction that address and overcome the drawbacks and deficiencies in the conventional art by disclosing an automated solution for diagnosing the operational states of complex apparatuses and accurately predicting situations in which those apparatuses are likely to malfunction or fail. The present data clustering for ML model-based anomaly prediction systems and methods advance the state-of-the-art by identifying and extracting data relevant to the operating state of an apparatus from a dataset, processing that relevant data to generate performance data for the apparatus, and using a trained ML model to predict whether the apparatus is operating anomalously based on that performance data. Furthermore, in some implementations, the present solution for performing data clustering for ML model-based anomaly prediction may include performing an intervention when the an anomalous operating state of an apparatus is predicted.

From the above description it is manifest that various techniques can be used for implementing the concepts described in the present application without departing from the scope of those concepts. Moreover, while the concepts have been described with specific reference to certain implementations, a person of ordinary skill in the art would recognize that changes can be made in form and detail without departing from the scope of those concepts. As such, the described implementations are to be considered in all respects as illustrative and not restrictive. It should also be understood that the present application is not limited to the particular implementations described herein, but many rearrangements, modifications, and substitutions are possible without departing from the scope of the present disclosure.