An anomaly detection system includes multiple sensors fixed within a user-selected environment to measure environmental conditions and wirelessly transmit data corresponding to the environmental conditions to a main device. The main device includes circuitry encoded with instructions from a trained machine learning model to identify one or more anomalies from the data received from the sensors. The main device transmits data related to the anomalies and the environmental conditions to a cloud storage via a plurality of wired and wireless connections. The cloud storage wirelessly relays the data to a user device. The user device, based on user's feedback, relays a feedback signal, via the cloud storage, to the trained machine learning model, where the feedback signal is incorporated into the trained machine learning model for the purpose of retraining.
Legal claims defining the scope of protection, as filed with the USPTO.
a plurality of sensors; a main device; and a cloud storage; wherein the plurality of sensors are fixed within a user-selected environment, wherein the plurality of sensors includes a gas sensor, a force sensor, a humidity sensor, a sound sensor, and a vibration sensor; the plurality of sensors being configured to measure a plurality of environmental conditions within the user-selected environment and wirelessly transmit the plurality of environmental conditions to the main device; wherein the main device includes circuitry encoded with instructions from a trained machine learning model to identify a plurality of one or more anomalies derived from the plurality of environmental conditions, wherein the main device is configured to transmit the plurality of one or more anomalies and the plurality of environmental conditions to the cloud storage via a plurality of wired and wireless connections; wherein the plurality of one or more anomalies and the plurality of environmental conditions are stored at the cloud storage, wherein the cloud storage is in wireless communication with an end user communications device and is configured to relay the plurality of one or more anomalies to the end user communications device, wherein the end user communications device is configured to communicate to a user the plurality of one or more anomalies, wherein the end user communications device is in wireless communication with the cloud storage and is configured to relay a feedback signal, via the cloud storage, to the trained machine learning model, wherein the circuitry of the main device includes instructions to incorporate the feedback signal into the trained machine learning model and to retrain the trained machine learning model with the feedback signal. . An anomaly detection system, comprising:
claim 1 a microcontroller; a chargeable battery; a low power consumption signal, wherein the low power consumption signal includes a wireless transmitter configured to transmit a low power signal to the cloud storage; and a sensing unit configured to measure each environmental condition of the plurality of environmental conditions; the sensing unit being electronically connected to the microcontroller, wherein the sensing unit and the microcontroller are each configured to wirelessly output each of the plurality of environmental conditions to the main device according to a time-step update. . The anomaly detection system of, wherein each of the plurality of sensors comprises:
claim 2 . The anomaly detection system of, wherein all of the sensors of the plurality of sensors are disposed in a common semi-enclosed space having an impermeable bottom and walls and an at least partially permeable top.
claim 2 wherein each sensor of the plurality of sensors is configured to wirelessly transmit the environmental condition to the trained machine learning model of the main device, the trained machine learning model identifying an anomaly corresponding to the environmental condition. . The anomaly detection system of, wherein each sensor of the plurality of sensors measures an environmental condition selected from the plurality of environmental conditions, wherein the plurality of environmental conditions are selected from a group consisting of a dust level, a pressure level, a gas level, a humidity level, a temperature level, a light level, a moisture level, a sound level, a vibration level, and a water level;
claim 1 a motherboard; a microprocessor; a memory unit; a local storage drive; a chargeable battery; a feature signal receiver; a wireless fidelity (WiFi) connection unit; a cellular connection unit; and an ethernet connection; wherein the feature signal receiver is configured to receive the plurality of environmental conditions from the plurality of sensors, wherein the local storage drive is configured to store the plurality of environmental conditions, wherein the main device connects to the cloud storage via the WiFi connection unit, via the cellular connection unit, and via the ethernet connection. . The anomaly detection system of, wherein the main device comprises:
claim 1 wherein the main device and cloud storage are connected to the end user communications device via a wide area network. . The anomaly detection system of, wherein the plurality of sensors and the main device are connected via a local communication network,
claim 1 wherein the access point is configured to scans and authenticates the new sensor upon detection of the broadcasting signal, and wherein the circuitry of the main device includes instructions to integrate the new sensor into the plurality of sensors. . The anomaly detection system of, wherein the main device includes an access point configured to detect a broadcasting signal output by a new sensor,
claim 1 . The anomaly detection system of, wherein a location of the main device is outside the user-selected environment.
claim 1 wherein the sensory range of the plurality of sensors is restricted by the main device to a boundary of the user-selected environment. . The anomaly detection system of, wherein the user-selected environment is defined by a distribution of the plurality of sensors and a sensory range of the plurality of sensors,
recording an environmental reading from a sensor according to a time-step, wherein the sensor is selected from the group consisting of a gas sensor, a force sensor, a humidity sensor, a sound sensor, and a vibration sensor; transmitting the environmental reading to a main device according to the time-step, wherein the main device accumulates and stores a plurality of environmental readings according to the time-step, triggering, via circuitry of the main device, a machine learning model when a count of the plurality of environmental readings reaches a threshold number of readings; identifying, via the machine learning model, an anomaly reading; categorizing, via the machine learning model, the anomaly reading into a positive anomaly reading, a negative anomaly reading, or an uncertain anomaly reading, wherein the anomaly reading is transmitted and stored in a cloud storage; transmitting, via a wide area network, the positive anomaly reading from the cloud storage to an end user communications device when the anomaly reading is the positive anomaly reading, wherein the end user communications device generates a first confirmation signal to confirm the positive anomaly reading, wherein the machine learning model is retrained on the plurality of environmental readings coupled with the first confirmation signal; returning the environmental reading to the main device from the cloud storage when the anomaly reading is the negative anomaly reading, wherein the machine learning model is retrained on the plurality of environmental readings coupled with the negative anomaly reading; transmitting, via the wide area network, the uncertainty anomaly reading to the end user communications device, wherein the end user communications device generates a second confirmation signal to confirm the anomaly reading as the positive anomaly reading or the negative anomaly reading; wherein the machine learning model is retrained on the plurality of environmental readings coupled with the second confirmation signal. . A method of anomaly detection, comprising:
claim 10 wherein the main device is configured to select one of the three detection models consisting of a k-means clustering algorithm, a support vector machine algorithm, and an isolation forest algorithm, according to an input signal received from the end user communications device. . The method of, wherein the machine learning model utilizes one of three detection models,
claim 10 wherein the default time range is modified by a first command signal, a second command signal, or a third command signal received by the end user communications device, the first command signal increasing the default time range, the second command signal maintaining the default time range, the third command signal decreasing the default time range. . The method of, wherein the time-step adheres to a default time range of five minutes,
claim 10 wherein the new sensor records a new environmental reading, wherein upon detection of the broadcasting signal the access point scans the new sensor and authenticates the new sensor, the main device establishing a wireless connection with the new sensor and integrating the new environmental reading into the machine learning model. . The method of, wherein the main device includes an access point configured to detect a broadcasting signal output by a new sensor,
claim 13 . The method of, wherein the access point detects the broadcasting signal in parallel with the machine learning model.
claim 10 . The method of, wherein the cloud storage is connected to the end user communications device by the wide area network.
claim 10 the plurality of environmental readings; the anomaly reading; and a user account configured to be accessible by the end user communications device; wherein the cloud storage relays a plurality of command signals from the user account to the main device. . The method of, wherein the cloud storage stores:
claim 10 . The method of, wherein the machine learning model is triggered when the count of the plurality of environmental readings reaches a threshold number of readings of 1,000.
claim 10 . The method of, wherein the main device comprises a Raspberry pi device containing circuitry to implement the machine learning model.
claim 10 . The method of, wherein the sensor comprises a temperature measuring device.
Complete technical specification and implementation details from the patent document.
The present application is a Continuation of U.S. application Ser. No. 18/311,955, now allowed, U.S. Pat. No. 12,131,622, having a filing date of May 4, 2023.
The inventor(s) gratefully acknowledge the support provided by Imam Abdulrahman bin Faisal University, Kingdom of Saudi Arabia.
The present disclosure relates to a remote monitoring and anomaly detection method and system and more particularly relates to system and method to generate anomaly reports for a user selected environment using machine learning based on inputs from sensors.
The “background” description provided herein is for the purpose of generally presenting the context of the disclosure. Work of the presently named inventors, to the extent it is described in this background section, as well as aspects of the description which may not otherwise qualify as prior art at the time of filing, are neither expressly nor impliedly admitted as prior art against the present invention.
Machine learning is a type of artificial intelligence that provides computers with the ability to learn without being explicitly programmed. Machine learning focuses on systems that can change when exposed to new data. Interest in machine learning is increasing due to a combination of advances in computing, big data management, and machine learning algorithms, among other things. Anomaly detection is a process of identifying outliers in the inputs for a region of interest (for example, offices, warehouse, shops, medicine storage plants, and the like). Example anomalies include, but are not limited to, a fire in the office, a failure in a mechanical part or a control system, high temperature in the medicine storage plants. Such anomalies may cause serious impacts including, but are not limited to, monetary losses, property damage, loss of life, etc.
Conventional systems and methods for anomaly detection suffer from one or more drawbacks hindering their adoption. Accordingly, it is one object of the present disclosure to provide a method and system to identify numerous anomalies in selected regions of interest in a smart way, verify if the reported anomalies are true and improve the detection model of the selected regions.
According to an exemplary embodiment of the present disclosure, anomaly detection system is disclosed. The anomaly detection system includes a plurality of sensors, a main device, and a cloud storage. The plurality of sensors are fixed within a user-selected environment and configured to measure a plurality of environmental conditions within the user-selected environment and wirelessly transmit the plurality of environmental conditions to the main device. The main device includes a circuitry encoded with instructions from a trained machine learning model to identify a plurality of one or more anomalies in the plurality of environmental conditions. The main device is configured to transmit the plurality of one or more anomalies and the plurality of environmental conditions to the cloud storage via a plurality of wired and wireless connections. The plurality of one or more anomalies and the plurality of environmental conditions are stored at the cloud storage and the cloud storage wirelessly relays the plurality of one or more anomalies to an end user communications device that communicates the plurality of one or more anomalies to a user. The end user communications device relays a feedback signal, via the cloud storage, to the trained machine learning model, so that the feedback signal is incorporated, via the main device, into the trained machine learning model for retraining based on the feedback signal.
According to another exemplary embodiment of the present disclosure, a method of anomaly detection is disclosed. The method includes recording an environmental reading from a sensor according to a time-step; transmitting the environmental reading to a main device according to the time-step, where the main device accumulates and stores a plurality of environmental readings according to the time-step; and triggering, via a circuity of the main device, a machine learning model when the plurality of environmental readings reaches a threshold number of readings. The method further includes identifying, via the machine learning model, an anomaly reading; categorizing, via the machine learning model, the anomaly reading into a positive anomaly reading, a negative anomaly reading, or an uncertain anomaly reading, where the anomaly reading is transmitted and stored in a cloud storage; and transmitting, via a wide area network, the positive anomaly reading from the cloud storage to an end user communications device when the anomaly reading is the positive anomaly reading.
The end user communications device generates a first confirmation signal to confirm the positive anomaly reading and the machine learning model is retrained on the plurality of environmental readings coupled with the first confirmation signal. The method further includes returning the environmental reading to the main device from the cloud storage when the anomaly reading is the negative anomaly reading, where the machine learning model is retrained on the plurality of environmental readings coupled with the negative anomaly reading. The method further includes transmitting, via the wide area network, the uncertainty anomaly reading to the end user communications device, where the end user communications device generates a second confirmation signal to confirm the anomaly reading as the positive anomaly reading or as the negative anomaly reading, and the machine learning model is retrained on the plurality of environmental readings coupled with the second confirmation signal.
These and other aspects of non-limiting embodiments of the present disclosure will become apparent to those skilled in the art upon review of the following description of specific non-limiting embodiments of the disclosure in conjunction with the accompanying drawings.
In the following description, it is understood that other embodiments may be utilized, and structural and operational changes may be made without departure from the scope of the present embodiments disclosed herein.
Reference will now be made in detail to specific embodiments or features, examples of which are illustrated in the accompanying drawings. Wherever possible, corresponding, or similar reference numbers will be used throughout the drawings to refer to the same or corresponding parts. Moreover, references to various elements described herein, are made collectively or individually when there may be more than one element of the same type. However, such references are merely exemplary in nature. It may be noted that any reference to elements in the singular may also be construed to relate to the plural and vice-versa without limiting the scope of the disclosure to the exact number or type of such elements unless set forth explicitly in the appended claims.
In the drawings, like reference numerals designate identical or corresponding parts throughout the several views. Furthermore, the terms “approximately,” “approximate,” “about,” and similar terms generally refer to ranges that include the identified value within a margin of 20%, 10%, or preferably 5%, and any values therebetween.
Aspects of the present disclosure are directed to enabling users to detect anomalies in a modern and smart way while maintaining user's privacy. The present disclosure provides system and method for anomaly detection, where multiple sensors are deployed in an environment that needs to be monitored. Data and readings of the surroundings from the sensors are collected and securely transmitted to a cloud computing analysis device. After sufficient recognition of the nature of the desired environment, anomalies are detected, and a trained machine learning module is improved over a time-step and through interaction with the user.
In various aspects of the disclosure, non-limiting definitions of one or more terms that will be used in the document are provided below.
The term “microcontroller” as used herein refers to a computer component adapted to control a system to achieve certain desired goals and objectives. For example, the microcontroller may refer to, be part of, or include: an Application Specific Integrated Circuit (ASIC); a digital, analog, or mixed analog/digital discrete circuit; a digital, analog, or mixed analog/digital integrated circuit; a combinational logic circuit; a field programmable gate array (FPGA); a processor circuit (shared, dedicated, or group) that executes code; a memory circuit (shared, dedicated, or group) that stores code executed by the processor circuit; other suitable hardware components that provide the described functionality; or a combination of some or all of the above, such as in a system-on-chip.
The term “sensor” refers, without limitation, to the component or region of a device which is configured to detect the presence or absence of a measurable parameter. For example, a camera sensor may be a camera configured to capture images or frames.
The term ‘machine learning’ refers to a method of data analysis that automates analytical model building. Machine learning is a branch of AI that uses statistical techniques to allow computer systems to learn from data without being explicitly programmed.
1 FIG. 100 100 100 102 104 104 102 104 102 106 108 110 112 114 115 116 118 120 122 102 104 illustrates an exemplary block diagram of an anomaly detection system(hereinafter referred to as “the system”), according to an embodiment of the present disclosure. The systemincludes a plurality of sensorsfixed within a user-selected environment. The user-selected environmentmay alternatively be referred to as a targeted area and includes a region of user's interest, such as, but not limited to, a warehouse, a food pantry, a patient room, or a space with indoor greenhouses. Each of the plurality of sensorsis configured to measure an environmental condition of the plurality of environmental conditions within the user-selected environment. In an embodiment, the plurality of environmental conditions may be selected from a group consisting of, but not limited to, a dust level, a pressure level, a gas level, a humidity level, a temperature level, a light level, a moisture level, a sound level, a vibration level, or a water level. Accordingly, the plurality of sensorsmay be, but not limited to, a dust sensor(for example, Sharp's GP2Y1010AU0F optical dust sensor) to sense the dust level, a pressure sensor(for example, Rp-s40-st pressure sensor) to sense the pressure level, a gas sensor(for example, MQ-2 gas sensor) to sense a presence of a gas and corresponding level thereof, a humidity sensor(for example, DHT11 humidity sensor) to sense the humidity level, a temperature sensor(for example, DHT11 temperature sensor) to sense the temperature level, a light sensor(for example, LM393 Light Sensor—LDR module) to sense the light level (such as, exposure of daylight), a moisture sensorto sense the moisture level, a sound sensor(for example, KY-038 sound sensor) to sense the sound level, a vibration sensor(for example, sw-420) vibration sensor) to sense the vibration level, and a water level sensor(for example, STSx0020 water level sensor) to sense the water level. Based on a requirement, one or more of these sensorsmay be located within the user-selected environmentto sense the corresponding parameters.
In a preferred embodiment of the present disclosure, a collection of particular environmental sensors are commonly located in a semi-enclosed space. In comparison to being disposed in the open without protection, certain sensors such as the dust sensor, the gas sensor, the humidity sensor, the sound sensor and the vibration sensor are preferably at least semi-isolated from the environment for purposes of providing a stable sensor output. In this respect it is preferable that the dust sensor, gas sensor, humidity, sound sensor and vibration sensor are located in a common semi-enclosed space. The semi-enclosed space is defined by an impermeable bottom that is preferably mounted on a wall or a structural member located in the region of the user's interest. The impermeable bottom is preferably made of a material that is thermally conductive but otherwise impermeable to transmission of gas. The impermeable bottom is preferably encircled along its perimeter with one or more walls defining impermeable vertically oriented walls extending from the impermeable bottom. Each of the aforementioned sensors is mounted on an extension that connects to a vertically oriented wall and extends into the semi-enclosed space parallel to the impermeable bottom. In this manner individual sensors can be better isolated from disruptive signal sources. The top of the semi-enclosed space is a partially permeable surface. Preferably, this surface is at least partially (at least 50% of the surface area) constructed of a metallic or thermally conductive material. A second minor portion (less than 50% of the surface area, preferably from 30 to 40% or 35 to 50%) is formed of a gas permeable thermoplastic membrane (e.g., having a thickness of 0.03 to 3 mm). The third portion (less than 30% of the surface area, preferably from 10 to 30% or about 20%) is formed of a mesh material having a pore size of from 1 mm to 10 mm, preferably 2-8 mm or about 5 mm.
100 124 126 128 130 132 134 136 138 140 142 102 124 124 124 104 124 104 102 124 104 The systemfurther includes a main devicethat houses various components including, but are not limited to, a motherboard, a microprocessor, a memory unit, a local storage drive, a chargeable battery, a feature signal receiver, a wireless fidelity (WiFi) connection unit, a cellular connection unit, and an ethernet connection. The sensorsare configured to wirelessly transmit values corresponding the plurality of environmental conditions to the main device. The main deviceincludes a circuitry encoded with instructions from a trained machine learning model to identify a plurality of one or more anomalies in the plurality of environmental conditions. In one embodiment, the main devicemay be located within the user-selected environment. In another embodiment, a location of the main devicemay be outside the user-selected environment. As used herein, the term “user-selected environment” may be defined by a distribution and a sensory range of the plurality of sensors, where the sensory range is restricted by the main deviceto a boundary of the user-selected environment.
102 144 146 148 150 150 150 144 150 144 124 102 124 Each sensorincludes a microcontroller, a chargeable battery, a low power consumption signal, and a sensing unit. In some embodiments, the sensing unitmay include a pressure measuring device. The sensing unitis electronically connected to the microcontrollerand configured to measure each environmental condition of the plurality of environmental conditions. The sensing unitand the microcontrollerare together configured to wirelessly output each of the plurality of environmental conditions to the main deviceaccording to a time-step update. As such, the plurality of sensorsare configured to transmit to the main device, signals corresponding to the plurality of environmental conditions.
124 136 102 132 132 124 132 102 124 102 124 At the main device, the feature signal receiverreceives the signals corresponding to the plurality of environmental conditions from the plurality of sensorsand values corresponding to each of the plurality of environmental conditions are stored in the local storage drive. In an example, the local storage drivemay be embodied as a hard drive or a solid-state drive (SSD) that is installed within the main device. A storage capacity of such local storage drivemay be determined based on type of inputs, for example digital inputs, received from the sensors. The trained machine learning model in the main deviceidentifies an anomaly corresponding to the environmental conditions. In some embodiments, the plurality of sensorsand the main devicemay be connected via a local communication network (not shown).
104 102 102 104 124 156 156 124 102 In one aspect, new sensor(s) may be included in the user-selected environment, where the new sensor(s) includes the components described hereinabove with respect to the sensors. For example, a soil sensor (for example, LM393 soil moisture sensor) may be added to the set of sensorsin the user-selected environment. The main devicemay include an access pointconfigured to detect a broadcasting signal output by the new sensor(s). Upon detection of the broadcasting signal, the access pointmay scan and authenticate the new sensor(s). Upon completion of the authentication, the main devicemay be configured to integrate the new sensor(s) into the plurality of sensors.
100 152 124 152 152 The systemalso includes a cloud storageand the main devicebeing configured to transmit the plurality of one or more anomalies and the plurality of environmental conditions to the cloud storagevia a plurality of wired and wireless connections. Although not illustrated in figures, in an embodiment, the cloud storagemay include a cloud service unit, a machine learning unit, a cloud data storage unit, and a processing unit.
126 128 130 124 132 152 138 140 142 124 152 148 102 152 148 The motherboard, the microprocessor, and the memory unitof the main deviceare configured to process and transfer the plurality of environmental conditions and the plurality of one or more anomalies from the local storage driveto the cloud storage. In an embodiment, the WiFi connection unit, the cellular connection unit, and the ethernet connectionaids the connection between the main deviceand the cloud storage. In some embodiments, the low power consumption signalof each sensormay include a wireless transmitter (not shown) configured to transmit a low power signal to the cloud storage. In another embodiment, the low power consumption signalmay comprise a light diode (not shown) emitting a light signal in the visible light spectrum.
152 152 154 154 152 154 154 124 152 154 154 152 124 154 154 Data corresponding to the plurality of one or more anomalies and the plurality of environmental conditions are stored at the cloud storage. Further, the cloud storageis configured to wirelessly relay the data corresponding to the plurality of one or more anomalies to an end user communications device. In an example, the end user communications devicemay be embodied as a handheld device (such as a smartphone) with a mobile application, or as a computer with a web application, configured to receive and process the data received from the cloud storage. In an embodiment, the data corresponding to the plurality of one or more anomalies and the plurality of environmental conditions may be displayed in a form of a graphical dashboard in the end user communications device. As such, the end user communications deviceis configured to communicate data corresponding the plurality of one or more anomalies, to the user. In an embodiment, the main deviceand cloud storagemay be connected to the end user communications devicevia a wide area network. Based on the received data, the end user communications deviceis configured to relay a feedback signal, via the cloud storage, to the trained machine learning model of the main device. Feedback communicated from the end user communications deviceis incorporated into the trained machine learning model for the purpose of retraining the trained machine learning model. With multiple such feedback signals from the end user communications device, the trained machine learning model may be capable of ascertaining authenticity of each of the plurality anomalies.
100 100 In some embodiments, a network unit (not shown) may be implemented as an interface to communicatively couple the components of the systemthrough various means including, but not limited to, standard telephone lines, Local Area network (LAN) or a Wide Area Network (WAN) links, broadband connections, wireless connections, or combination of any or all of the above. The network unit may use various communication protocols such as a TCP/IP, Ethernet, IEEE 802.11, WiFi, WiMAX, and such protocols. The network unit may include network interface card, a wireless network adapter, USB network adapter, modem, or any other elements enabling coupling the components of the systemwith any type of network capable of communication and performing the operations described herein.
2 FIG. 2 FIG. 1 FIG. 1 FIG. 100 144 150 144 150 144 144 150 144 150 102 150 124 102 102 112 112 144 102 144 102 124 illustrates a technical layout of the system, according to an aspect of the present disclosure.is described in conjunction with. The microcontroller(for example, a firebeetle Esp32 microcontroller) may be programmed with C++ using, for example, Arduino Integrated Development Environment (commonly referred to as Arduino IDE) to store data in form of global and local variables. As described in, the sensing unitis connected to the microcontrollerand “the sensing unit+the microcontroller” pair functions as a Message Queuing Telemetry Transport (MQTT) client. The microcontrollerprovides the WiFi connection to the sensing unitand, owing to such connection, the microcontrollermeasures readings of the parameter sensed by the sensing unitof corresponding sensor. The MQTT client is configured to publish the data (such as the measured value of the parameter sensed by the sensing unit) to a MQTT broker (such as the main device) relating to a parameter based on a type of the sensorand corresponding identification of the sensor. For example, the published data may be humidity [1], where the term humidity is the parameter sensed by the humidity sensorand “1” indicates the identification number of the humidity sensor. Along with such data, the values of humidity measured by the microcontrolleris also published. As described earlier, each sensorincludes a microcontroller (such as the microcontroller) configured to measure the corresponding parameter sensed by that sensor. As such, each microcontroller is configured to publish the respective data to the main device.
154 124 144 IF Exist: set next upload time; ELSE: Use default upload time Fetch publishing time: In an embodiment, a default publishing time period may be every 5 minutes. However, in some embodiments, the user may be allowed to change a publishing time period to any positive time value using the end user communications device. In some embodiments, each microcontroller may be configured to learn about the publishing time period from the main device. For example, the microcontroller (such as the microcontroller) may be configured to execute the below:
104 124 104 104 200 200 124 200 104 When a new sensor is added to the user-selected environment, the microcontroller of the new sensor transmits a MQTT packet to the MQTT broker (alternatively referred to as “the main device” in the present disclosure) to announce presence of the new sensor included in the user-selected environment. In some embodiments, information regarding the addition of the new sensor to the user-selected environmentmay be provided as an input by an auxiliary device(for example, an android phone) to the MQTT broker. In case of such addition of the new sensor, an access to the MQTT broker may be secured with user identification and password. For authentication purposes, in some embodiments, the auxiliary devicemay be paired with the main deviceusing, for example, service set identifier (SSID), user id, and password. Upon successful authentication, the auxiliary devicemay announce the addition of the new sensor to the user-selected environment.
124 102 102 200 102 156 102 102 104 102 156 102 102 In an embodiment, a Raspberry pi device may be implemented as the main deviceand may be programmed using, for example, Python scripting language and Bash scripting language. The Raspberry pi device may be configured to manage and control all the sensorsand perform anomaly detection. For such purpose, the Raspberry pi device may include three segments, namely the MQTT broker (that is configured to receive all messages from the clients, such as the sensorsand the auxiliary device), the MQTT client module, and an anomaly detection model. The Raspberry pi device may be configured to run the MQTT broker to gather all messages (such as data) from the clients. Data received from each of the sensorsmay be stored on a local server, for example MariaDB. In operation, the Raspberry pi device may be configured to actuate the access point, where the microcontrollers of all the sensorsmay connect with a static IP address. Information relating to the sensorspre-installed in the user-selected environmentmay be stored in the Raspberry pi device. As such, when the microcontrollers of such existing sensorsconnect with the access point, the Raspberry pi device may consider the data from such sensorsas, for example, known data. In such cases, the presence of the new sensor may be easily identified, and the Raspberry pi device may be configured to automatically initiate required procedure to add the new sensor to a list of sensorsand may allow the corresponding microcontroller of the new sensor to transmit data to the Raspberry pi device.
102 152 154 152 102 102 152 154 102 Further, using, for example, Paho MQTT Python client library, the Raspberry pi device may be configured to run the MQTT client module to subscribe to all parameters received from the sensors. When the new sensor connects for the first time, the Raspberry pi device may be configured to upload its identification and type (such as, “humidity [1]” as described earlier) to a cloud database (such as the cloud storage) and may set the anomaly detection model of the new sensor to clustering by default. In some embodiments, the Raspberry pi device may be configured to announce, to the user, addition of the new sensor by setting its status to “New” in the sensors' settings dashboard in the end user communications device. When the user accepts the addition of the new sensor, the Raspberry pi device may be configured to periodically transmit to the cloud storage, the data received from the new sensor. In cases where the user wishes to stop receiving data from a specific sensorfor a predefined time period, the user may be allowed to set a status of such sensorto “Rejected” and such selection may trigger a signal to the Raspberry pi device via the cloud storage. Upon receipt of such signal from the end user communications device, the Raspberry pi device may be configured to discard any MQTT packet (such as, data from the sensor) from such sensor until the status is set to “Accepted” by the user.
154 102 152 152 124 154 In cases where the addition of the new sensor was not initiated by the user, or if the user fails to identify the new added sensor, the end user communications devicemay allow the user to set the status of the new sensor to “Deleted” and the Raspberry pi device may be configured to discard all the incoming MQTT packet from the new sensor. Although the term “discard”, as used herein, refers to “reject as no longer useful”, in some embodiments, the data from such new sensors may be stored in a memory (not shown) of the Raspberry pi device and may be provided to the user as historical data when the status of the new sensor is set to “Accepted”. When the data corresponding to the accepted sensorsreach a predefined threshold, for example 1000 readings, the Raspberry pi device may be configured to run the anomaly detection model (set to a clustering model by default) on every new reading and trigger an alarm to the cloud storageregarding the reading associated with anomaly and corresponding timestamp of the anomaly. In an aspect, the cloud storage(for example, cloud real-time cloud server) may be configured to store all the data received by the Raspberry pi device, in form of JSON tree (otherwise described as “iTree”); store anomaly cases; store information corresponding to user accounts and preferences; connect the main devicewith the end user communications device; and process data and operations.
154 152 152 As described earlier, the end user communications devicemay be embodied as a handheld device (such as a smartphone) with a mobile application, or as a computer with a web application, configured to receive and process the data received from the cloud storage. In one aspect, the mobile application may be an android mobile application programmed using Java and configured to store data locally—using for example SQLite database. In another aspect, the web application may be programmed using one or more of HTML, JavaScript. NodeJs and configured to read and write data from the cloud storage.
156 100 102 154 156 154 156 156 102 102 The user may be allowed to create a user account in the access pointeither using the web application or the mobile application. When the systemis operated for the first time, the Raspberry pi device would require access to the internet and the user account to upload all the data relating to the sensors. The user may, via the end user communications device, connect to the access pointand share the details of the user account along with network credentials using “Set up new device” interface in the end user communications device(for example, the mobile application). Such connection with the access pointmay be protected by MQTT username and password which may be hardcoded in mobile application. As used here, the “mobile application” refers to a custom-made application for the mobile device that provides access to the access pointvia authentication of credentials. Post authentication, the user may be allowed to access and manage the data relating to the sensorsfrom anywhere, without the need to be connected to the network. In an embodiment, the data (including detected anomaly) relating to the sensorsmay be provided in form of dashboard (such as graphical dashboard) in the mobile application.
102 102 104 102 When the new sensors is added to the list of sensors, the details of the new sensor may be provided to the user in the mobile application and the user may be allowed to manage the status of the new sensor (such as Accept, Reject, Delete), set the anomaly detection model (for example, clustering, Support Vector Machine (SVM), Isolation Forest), and choose a representation name for the new sensor to be used thereafter. In some embodiments, more than one user-selected environment may be available for access by the user in the mobile application. When one or more sensorsneed to be relocated from one user-selected environment (such as the user-selected environment) to another user-selected environment, the user may be allowed to reset the user-selected environment based on the relocation. The user may also be allowed to delete all the data stored relating to the sensorbeing relocated, so that old data is not considered for anomaly detection.
The anomaly detection model is described below:
Clustering Model or K-Means Clustering:
1 2 n 1 2 k Given a set of observations (x, x, . . . , x), where each observation is a d-dimensional real vector, k-means clustering aims to partition the n observations into k (≤n) sets, S={S, S, . . . , S}, so as to minimize the within-cluster sum of squares (WCSS). The objective is to find:
i i where μis the mean of points in S. This is equivalent to minimizing the pairwise squared deviations of points in the same cluster:
The Equivalence can be Deduced from Identity:
Since the total variance is constant, the sum of squared deviations between points in different clusters (between-cluster sum of squares, BCSS) are maximized, which follows from the law of total variance.
Stage-1: a training dataset is used to build iTrees as described in previous sections; and Stage-2: each instance in the test set is passed through the iTrees built in the previous stage, and an “anomaly score” is assigned to the instance as described below. Anomaly detection with Isolation Forest is a process composed of two main stages:
Once all the instances in the test set have been assigned an anomaly score, the anomaly may be determined. For example, a data point where the anomaly score is greater than a predefined threshold may be marked as “anomaly”, based on a domain in which the analysis is being applied.
Computing the anomaly score of the data point is based on the observation that the structure of iTrees is equivalent to that of Binary Search Trees (BST), where a termination to an external node of the iTree corresponds to an unsuccessful search in the BST. As a consequence, an average may be estimated as:
where n is a testing data size, m is the size of a sample set, and H is the harmonic number, which may be estimated as:
where γ=0.5772156649 is the Euler-Mascheroni constant.
A value of c(m) represents an average of h(x) given m. As such, value of h(x) may be normalized, and the anomaly score may be estimated for a given instance x:
If s is close to 1, then x is very likely to be an anomaly; If s is smaller than 0.5, then x is likely to be a normal value; and If for a given sample all instances are assigned an anomaly score of around 0.5, then it is safe to assume that the sample doesn't have any anomaly.One-Class Support Vector Machines (SVM): where E(h(x)) is the average value of h(x) from a collection of iTrees. For any given instance x:
One-class SVM is a variation of the SVM that can be used in an unsupervised setting for anomaly detection. Unlike a regular supervised SVM, the one-class SVM does not have target labels for model training process. Instead, the one-class SVM learns a boundary for normal data points and identifies the data points outside the boundary as anomalies.
In an aspect, SVM based one-class classification (OCC) relies on identifying a smallest hypersphere (with a radius r, and a center c) consisting of all the data points. This method is referred to as Support Vector Data Description (SVDD) and may be defined in the following constrained optimization form:
However, the above formulation may be highly restrictive, and is sensitive to the presence of anomalies. Therefore, a flexible formulation, that allows for presence of anomalies is formulated as shown below:
i i 2 2 subject to, ∥φ(x)−c∥≤r+ζ∀i=1, 2, . . . , n.
From Karush-Kuhn-Tucker (KKT) optimality conditions:
1 where αthe solution to the following optimization problem:
subject to,
for all i=1, 2, . . . , n.
Both the one-class SVM and the isolation forest algorithms are capable of properly modeling multi-modal data sets. One-class SVM is sensitive to anomalies, making it more appropriate for novelty detection, when the training data is not contaminated with anomalies. Since the splits of the decision tree are chosen at random, the isolation forest is faster to train. In general, the SVMs are slow to train, especially with respect to the training set size. The K-Means Clustering algorithm requires the number of clusters (k) to be specified in advance, and may not handle noisy data and outliers, and hence may not be suitable to identify clusters with non-convex shapes.
100 Following are pseudocodes which may be implemented in the systemaccording to one embodiment:
144 150 For Microcontrollerand the Sensing Unit:
BEGIN Set topic name <- Sensor's name While True: Connect to the System Network If Connected: Set MQTT username=user Set MQTT password=pwd Connect to the System MQTT Broker Read Sensor data Publish MQTT to topic Fetch Sleep Time from database If sleep time set by user: Sleep for user's required time Else: Sleep for 5 minutes 124 For the Main Device(Raspberry Pi):(a) Connection:
Begin Connect to WIFI Set Access point name <- name Set Access point password <- pwd Start Access point Start MQTT Broker Connect to cloud Realtime database While True: Subscribe to MQTT sensor's topics Subscribe to MQTT Android mobile phone topic //used to handle sensors' data IF new reading arrives from microcontrollers: Fetch sensors state and anomaly_detection_model from cloud database IF sensor is approved by the user: Upload sensor's data Call Anomaly_detection (sensor's data, sensor's type, sensor's, nomaly_detection_model, timestamp) Else IF sensor is rejected by the user: Discard sensor's data Else IF sensor is connected to the first time: Upload sensor's type, id to the database and set its' status to NEW End IF //used to update cloud user or network data Else IF new reading arrives from Android mobile phone: Set cloud username <- received username Set network SSID <- received SSID Set network password<- received password Connect to new network End IF End (b) Anomaly Detection Model:
Procedure Anomaly_detection (sensor's data, sensor's type, sensor's, anomaly_detection_model,timestamp): Read sensor's data file located internally IF file exists: Add sensor's data to the file Fetch number of available readings IF available readings >250 Start anomaly detection model End IF //file does not exist (first reading of the sensor) Else: Create new File with the sensor's name and id Add sensor's data to the file End IF 154 For End User Communications Device:(a) Launch:
Begin Get network Access permission Start Main interface Main interface: IF signUp selected: Start signUp interface Else IF LogIn selected: Start LogIn interface Else IF Register new Device selected: Start Register_new_Device interface End (b) Sign-Up Interface:
Create cloud instance: Enter First name Enter Last name Enter Email Enter Password Repeat Password Check All fields are filled: IF all field filled properly: check if email already exist in cloud IF email already exist: ask user to log in or use different email Else: create new account in cloud End IF Else: promote user to re-enter missing data End IF End (c) Login Interface:
Enter Email Enter Password Connect to cloud IF Credentials correct: Start Dashboard interface Else: Ask user to check entered data End IF End (d) Register New Device Interface:
Select user id Enter Network SSID Enter Network Password Promote user to Connect to the System network Connect to MQTT broker IF connections success: Publish MQTT to Android topic Start Main interface End (e) Dashboard Interface:
Connect to cloud database: Select desired sensor Select number of readings Fetch sensor's data Display Realtime graph End (f) Anomalies Interface:
Connect to cloud database: Select desired sensor Select number of readings Fetch sensor's data anomalies Display Realtime graph End
3 FIG. 154 illustrates exemplary images of a sign-up webpage and a sign-in webpage of the web application of the end user communications device. Following are pseudocodes which may be implemented for the web application:
(a) Launch:
Begin Start Main interface Main interface: IF signUp selected: Start signUp page Else IF LogIn selected: Start LogIn page End IF End (b) Sign-Up Page:
Create cloud instance: Enter First name Enter Last name Enter Email Enter Password Repeat Password Check All fields are filled: IF all field filled properly: check if email already exist in cloud IF email already exist: ask user to log in or use different email Else: create new account in cloud End IF Else: promote user to re-enter missing data End IF End (c) Login Page:
Enter Email Enter Password Connect to cloud IF Credentials correct: Start Dashboard interface Else: Ask user to check entered data End IF End (d) Dashboard Page:
Connect to cloud database: Select desired sensor or all sensors Select number of readings Fetch sensors' data Display Realtime graphs End (e) Anomalies Page:
Connect to cloud database: Select desired sensor or all sensors Select number of readings Fetch sensors' data anomalies Display Realtime graphs End (f) Settings Page:
Connect to cloud database: Display all connected sensors to the system for each sensor: Enable users to set (sensors name, state, anomaly detection model, update time) End
4 FIG.A 4 FIG.E 4 FIG.A 4 FIG.B 4 FIG.E 102 106 106 108 throughillustrates portions of the dashboard of the web application depicting graph plots of measured values of parameters corresponding to the sensors. For example,illustrates graph plots corresponding to the readings of the dust sensorlocated in user's room. The readings are identified with user set name for the dust sensor, such as “Dust: My room dust” according to one example. Similarly, the pressure sensorlocated in a chair in the user room's is identified with user set name as “Force: Chair” according to another example. The other graphs plots ofthroughillustrates the names set by the user for respective sensors for the purpose of identification.
5 FIG. 6 FIG. 102 102 illustrates an exemplary settings interface where the user is allowed to control all the sensorsandillustrates an extension of the settings interface where the user is allowed to manage the anomaly detection model for each sensor.
7 FIG. 7 FIG. 102 124 112 102 illustrates an exemplary anomalies interface that indicates the presence of anomaly from the sensors. The image inshows the readings and timestamp of each reading received by the main devicefrom the humidity sensor. The user may be allowed to select the desired sensor to view the readings corresponding to that sensor. The value of each reading is also indicated on the dashboard. As described earlier, when the data point lies beyond the boundary or when the value of the reading is above a threshold value, the reading is considered as an anomaly and reposted to the user via the dashboard. In an embodiment, color coding or text of differing sizes may be followed to emphasis the value of the readings which are above the threshold value.
8 FIG. 102 102 illustrates a screenshot of cloud real-time database. Historical data of each sensormay be stored in the database (for example, MariaDB). The user may be allowed to access any of the historical data of each sensoras desired.
9 FIG. 10 FIG. 5 FIG. 11 FIG. 156 102 102 illustrates screenshot of android mobile application used to sign-up and sign-in to the user account and establish the connection with the access point.illustrates a screenshot of the android mobile application showing sensor setting interface. Similar to the web application as illustrated in, the user may be allowed to control all the sensorsvia the mobile application alternatively.illustrates a screenshot of the android mobile application showing sensor reading interface, where the user may be allowed to navigate between various sensors and obtain a visual statistics of the corresponding readings. Additionally, through this interface, the user may be allowed to assign the anomaly detection model for each sensor.
12 FIG. 12 FIG. 100 104 120 112 116 102 124 152 154 illustrates a schematic diagram of the systemincluding smart features. As illustrated in, the user-selected environmentmay include, but not limited to, a washing-dryer machine, an oven, a refrigerator, an audio device, a motion sensor, a light sensor, a water level sensor as the smart features. Each of the washing-dryer machine, the oven, the refrigerator, and the audio device may include a sensor. In one example, the washing-dryer machine may include the vibration sensorto sense the level of vibration during operation thereof, the refrigerator may include the humidity sensorto sense the amount of humidity in the air entering the refrigerator when the door of the refrigerator is opened, the oven may include the moisture sensorto sense the amount of moisture in the food placed therein, and the like. Each of these sensorsmay transmit the data to the main devicewhich is further shared with the user via the cloud storageand the end user communications device. As such, anomaly in functioning of any of the smart features may be determined and the user may be allowed to take appropriate action to remedy the anomaly.
13 FIG. 100 102 104 124 124 102 152 124 154 154 152 124 illustrates a schematic diagram of the systemequipped with various means of connectivity. In an embodiment, the sensorsin the user-selected environmentmay transmit the data to the main devicevia one of, but are not limited to, WiFi, Bluetooth, USB, or other known wired or wireless connections. The main devicemay be configured to communicate with the sensorsvia one of, but are not limited to, 5G technology, Bluetooth, WiFi, or LAN. The cloud storagemay be configured to establish a connection with the main deviceand the end user communications devicevia an antenna tower or a base transceiver station. The end user communications devicemay be configured to establish the connectivity with the cloud storageand the main devicevia one of, but are not limited to, 2G technology, 3G technology, 4G technology, 5G technology, LAN, or WiFi.
14 FIG. 100 100 1402 1404 1402 1404 100 1404 1402 104 102 112 114 116 120 154 1404 154 102 124 154 is a schematic diagram illustrating implementation of the system. In an aspect, the systemmay be configured to connect multiple domainswith corresponding applications. The domainsmay include, but are not limited to, artificial intelligence, healthcare, clinical studies, monetary module, green power, safety modules, and the like. The applicationsmay include, but are not limited to, bank, hospital, house, industries, offices, shops, warehouse, and the like. The systemmay help connect the applicationwith a required domainto identify anomalies. For example, the user-selected environmentmay be a medicine storage unit. Multiple sensors, such as the humidity sensor, the temperature sensor, the moisture sensor, and the vibration sensormay be located in the medicine storage unit. One or more end user communications devicesmay be located in a related application, such as the hospital. The end user communications devicesat the hospital may receive data from each of the sensorslocated in the medicine storage unit and the presence of anomaly may be communicated by the main deviceto the end user communications devices.
15 FIG. 100 104 1500 104 1502 1504 1506 1508 1510 1512 1514 1516 102 124 102 124 102 124 152 124 154 152 124 1500 is an exemplary schematic diagram of the systemimplemented for monitoring COVID-19 vaccine. The user-selected environmentmay be an enclosed spacewhere the COVID-19 vaccine is stored. The user-selected environmentmay include the humidity sensorto sense amount of humidity in the enclosed space, a light sensorto sense amount of light in the enclosed space, a temperature sensorto sense the temperature of the enclosed space, an object sensorto sense presence of objects others than the COVID-19 vaccine boxes in the enclosed space, a water sensor, a weight sensor, a color sensor, and a sound sensor. Each of these sensorstransmits the data to the main devicebased on a time-step update (alternatively referred to as the publishing time period in the present disclosure). Based on the data received from the sensors, the trained machine learning model in the main deviceis configured to determine anomalies with respect to the parameters corresponding to the sensors. For example, the main devicemay check for presence of anomalies with respect to, but are not limited to, temperature, weight, color, and darkness. Further, the cloud storagemay communicate the data from the main deviceto the end user communications device, along with terms or phrases corresponding to each data, such that the user may easily understand. For example, the terms or phrases may include “No light”, “Glass sound”, “Watercolor”, or “Temperature always −70 C”. The anomaly report from the cloud storagemay emphasize on the anomalies, such as “Temperature NOT [−70 C]”. Certain anomalies may be considered as normal by the user and may accordingly be communicated to the main deviceas a feedback signal to retrain the trained machine learning model. For other reported anomalies, the user may visit the enclosed spaceto rectify the anomaly.
16 FIG. 100 1600 104 1600 102 1602 1604 1606 1608 1610 1612 1614 1616 124 1600 1600 1600 1600 1600 152 124 154 124 1600 is an exemplary schematic diagram of the systemimplemented for monitoring a warehouse. The user-selected environmentin this case is the warehousethat stores goods and includes multiple sensors, such as a humidity sensor, a light sensor, a temperature sensor, an object sensor, a water sensor, a weight sensor, a color sensor, and a sound sensor. These sensors are configured to transmit data to the main device, where the data includes readings corresponding to, but are not limited to, temperature in the warehouse, activity of people in the warehouse, movement of machines in the warehouse, movement of goods in the warehouse, and sounds in the warehouse. The cloud storagemay associate each data with terms or phrases, such as “Usually quite every Sunday”, “Light is always ON”, and “Temperature 25 C”. The report from the main devicealong with these terms and phrases are communicated to the user via the end user communications device. In case of presence of an anomaly, the same may be indicated to the user. For example, the anomalies may be communicated as “temperature NOT normal”, “Dark”, or “Small object moved”. Certain anomalies may be considered as normal by the user and may accordingly be communicated to the main deviceas a feedback signal to retrain the trained machine learning model. For other reported anomalies, the user may visit the warehouseto rectify the anomaly.
17 FIG. 100 1700 104 1700 1700 1702 1704 1706 1708 1710 1712 1714 1716 124 1700 1700 1700 1700 152 124 154 124 1700 is an exemplary schematic diagram of the systemimplemented for monitoring a food pantry). The user-selected environmentin this case is the food pantrythat stores food and ingredients required to prepare the food. The food pantryincludes multiple sensors, such as a humidity sensor, a light sensor, a temperature sensor, an object sensor, a water sensor, a weight sensor, a color sensor, and a sound sensor. These sensors are configured to transmit data to the main device, where the data includes readings corresponding to, but are not limited to, temperature in the food pantry, humidity in the food pantry, concentration of oxygen against concentration of carbon-dioxide in the food pantry, and amount of light in the food pantry. The cloud storagemay associate each data with terms or phrases, such as “usually dry”, “usually dark”, “temperature 5 C”, or “Person size movement”. The report from the main devicealong with these terms and phrases are communicated to the user via the end user communications device. In case of presence of the anomaly, the same may be indicated to the user. For example, the anomalies may be communicated as “Very humid”, “Very bright for few hours”, or “Mouse size object moved”. Certain anomalies may be considered as normal by the user and may accordingly be communicated to the main deviceas a feedback signal to retrain the trained machine learning model. For other reported anomalies, the user may visit the food pantryto rectify the anomaly.
18 FIG. 100 1800 104 1800 1800 1800 1800 1802 1804 1806 1808 1810 1812 1814 1816 124 1800 1800 1800 1800 152 124 154 124 1800 is an exemplary schematic diagram of the systemimplemented for monitoring a patient room. The user-selected environmentin this case is the patient roomthat accommodates multiple patients. The patient roommay also be located in the residence of the user and may be used to accommodate family members of the user. Alternatively, the patient roommay be an intensive care unit in a hospital. The patient roommay include multiple sensors, such as a humidity sensor, a light sensor, a temperature sensor, an object sensor, a water sensor, a weight sensor, a color sensor, and a sound sensor. These sensors are configured to transmit data to the main device, where the data includes readings corresponding to, but are not limited to, temperature in the patient room, humidity in the patient room, carbon-dioxide level in the patient room, light pattern in the patient room, and bed weight pressure. The cloud storagemay associate each data with terms or phrases, such as “usually dry and cool”, “CO2=400 ppm”, “temperature 25 C”, or “Person size movement”. The report from the main devicealong with these terms and phrases are communicated to the user via the end user communications device. In case of presence of the anomaly, the same may be indicated to the user. For example, the anomalies may be communicated as “Very hot/humid”, “CO2>600 ppm”, or “Empty bed for hours”. Certain anomalies may be considered as normal by the user and may accordingly be communicated to the main deviceas a feedback signal to retrain the trained machine learning model. For other reported anomalies, the user may visit the patient roomto rectify the anomaly.
19 FIG. 100 1900 104 1900 1900 1902 1904 1906 1908 1910 1912 1914 1916 124 1900 1900 1900 1900 1900 152 124 154 124 1900 is an exemplary schematic diagram of the systemimplemented for monitoring an indoor greenhouse. The user-selected environmentin this case is the indoor greenhousethat houses multiple plants. The indoor greenhousemay include multiple sensors, such as a humidity sensor, a light sensor, a temperature sensor, an object sensor, a water sensor, a weight sensor, a color sensor, and a sound sensor. These sensors are configured to transmit data to the main device, where the data includes readings corresponding to, but are not limited to, temperature in the indoor greenhouse, soil humidity in the indoor greenhouse, carbon-dioxide level in the indoor greenhouse, brightness in the indoor greenhouse, and object motion in the indoor greenhouse. The cloud storagemay associate each data with terms or phrases, such as “wet soil”, “CO2=700 ppm”, “temperature 15 C to 35 C”, or “Person size movement”. The report from the main devicealong with these terms and phrases are communicated to the user via the end user communications device. In case of presence of the anomaly, the same may be indicated to the user. For example, the anomalies may be communicated as “Very hot/dry”, “CO2<200 ppm”, or “Animal size motion”. Certain anomalies may be considered as normal by the user and may accordingly be communicated to the main deviceas a feedback signal to retrain the trained machine learning model. For other reported anomalies, the user may visit the indoor greenhouseto rectify the anomaly.
20 FIG. 1 FIG. 2000 100 2002 124 124 126 128 134 138 is a flowchart of a methodof installing and actuating the system. At step, the main deviceis plugged into a power source. As described with respect to, the main deviceincludes the motherboard, the microprocessor, the chargeable battery, the WiFi connection unitwhich require power supply for operation thereof.
2004 102 104 102 104 At step, the sensorsare mounted within the user-selected environment. In an aspect, required and appropriate sensorsmay be mounted within the user-selected environment.
2006 124 124 102 At step, the service of the main devicemay be activated, such that the main device, such as the Raspberry pi, may fetch details from the sensors.
2008 200 At step, the user logs-in to the auxiliary device) and provides login credentials to be authenticated at the MQTT broker.
2010 102 At step, the user creates a profile in the MQTT broker and maps the sensorsto the profile.
2012 154 At step, the user accesses the mobile application or the web application via the end user communications device.
2014 102 124 At step, the user is allowed to manage and control the sensorsmapped to the profile and interacts with the main devicein response to anomaly reports.
21 FIG. 2100 104 2102 104 104 104 is a flowchart of a methodof adding new sensors to the set of sensors in the user-selected environment. At step, the new sensor (alternatively referred to as “feature” in the present disclosure) is selected for the user-selected environment. In an aspect, a sensor being moved from one user-selected environmentto another user-selected environmentmay be treated as the new sensor in the destination.
2104 104 At step, the new sensor is mounted at a desired location determined by the user in the user-selected environment.
2106 124 124 104 154 124 102 104 At step, the main devicereceives the data from the new sensor and determines the details thereof. The main devicecommunicates the details of the new sensor and the corresponding user-selected environmentto the user via the end user communications device. Once the user accepts the addition of the new sensor, the main deviceauthenticates the new sensor and includes the new sensor to the list of sensorsin the user-selected environment.
2108 124 At step, the user is allowed to configure and manage the new sensor once the new sensor is authenticated by the main deviceand added to the profile. In an aspect, the user may assign the anomaly detection model to the new sensor.
2110 156 124 At step, the new sensor is allowed to transmit the data to the access pointof the main device.
22 FIG. 2200 2202 124 104 is a flowchart of a methodof authenticating the new sensor. At step, the main devicescans the details of the new sensor included in the user-selected environment, based on the data packets received from the new sensor.
2204 124 156 152 At step, the main deviceestablishes a connection between the access pointand the new sensor. Details of the new sensor is communicated to the user via the cloud storage.
2206 124 At step, the main devicechecks whether the new sensor was authenticated by the user. For example, the user may click on “Accept” option in the dashboard of the mobile application to authenticate the new sensor.
2208 124 102 104 2210 124 102 If the new sensor is authenticated, at step, the main deviceadds the new sensor to the list of sensorsalready present in the user-selected environmentand provides an acknowledgement, at step, to the user regarding the addition of the new sensor. If the addition of the new sensor is rejected by the user, the main deviceprovides the acknowledgement of the removal of the new sensor from the list of sensors.
23 FIG. 2300 102 2302 124 102 102 124 102 is a flowchart of a methodof detecting an anomaly corresponding to a feature. For each sensor, at step, the main devicereads a status of the sensor. At any time, as desired by the user, the status of the sensormay be changed from ‘accepted’ to ‘rejected’ or ‘deleted’. As such, the main devicechecks the status of each sensor.
2304 124 102 At step, the main deviceprocessed the data received from the sensor.
2306 124 102 At step, the main devicedetermines the presence of anomaly in the data received from the sensor.
124 2308 124 154 152 If the main devicedetects an anomaly, at step, the main deviceprovides an alert to the user on the end user communications devicevia the cloud storage.
2310 2312 124 At step, the user is allowed to confirm if such anomaly report is true or if the readings are normal. In both the cases of confirmation from the user and rejection from the user in response to the alerted anomaly, at step, the trained machine learning model in the main deviceis retrained.
124 2306 124 2308 124 If the main device, at step, is uncertain of determining certain data points as anomaly, the main devicenotifies such data to the user and requests the user, at step, to confirm if such data should be considered as anomaly. Based on the feedback from the user, the trained machine learning model in the main deviceis retrained.
124 2306 124 2312 If the main device, at step, does not detect any anomaly from the readings, the main device, at step, trains the trained machine learning model based on the readings.
Once the device is implemented in the environment, it starts to collect data value from sensors for specific duration (interval time) until it reaches an accuracy threshold. The duration value varies depending on the sensor, environment, or user preferences (e.g., 5 mins, 1 hr, 5 secs . . . etc.). For example, at every interval time it will read the values from sensors. Then, the collected values will be stored in a cloud storage until it reaches the accuracy threshold (for example 10,000 values). The interval time and the threshold can be adjusted based on the user need. These values are then used to feed an unsupervised machine learning algorithm to build a model capable of recognizing the normality and predicting the anomalies. Once the model is self-built, it is used with an expert domain to validate the prediction given by the machine learning algorithm model prior to feed it back to the machine learning model. The values are stored in the database, with the validated output generated by the expert domain. Following that, the model is retrained using the newly stored values in order to improve its performance in the future by exposing it to up-to-date anomalies.
24 FIG. 25 FIG. 102 124 is a symbolic representation of components of the sensorandis a symbolic representation of components of the main device.
26 FIG.A 26 FIG.B 102 124 102 102 104 102 is an exemplary illustration of a graphic user interface (GUI) of the android mobile application depicting monitoring of data from sensorsandis an illustration of the GUI of the android mobile application depicting options made available to the user to interact with the main device. In some embodiments, as described earlier, the data of each sensorand anomaly reports may be made available to the user in form of the dashboard. The dashboard allows the user an overview showing a total number of anomalies detected, a total number of data point processed, and/or an anomaly rate for the processed data. This will allow the user to determine if there are significant problems with the sensorsor the user-selected environment. For example, if 100 data points out of 100 data packets received from the sensorsindicate anomalous behavior, then there is likely an issue that needs urgent addressing by the user.
27 FIG. 2700 2700 124 is a flowchart of a methodof anomaly detection, according to an aspect of the present disclosure. Some or all of the steps (or other processes described herein, or variations, and/or combinations thereof) herein may be performed under a control of one or more computer systems or controller configured with executable instructions and may be implemented as code (for example, executable instructions, one or more computer programs, or one or more applications) executing collectively on one or more processors, by hardware or combinations thereof. The code may be stored on a computer-readable storage medium, for example, in form of a computer program including instructions executable by the one or more processors. The computer-readable storage medium is non-transitory. In some embodiments, one or more (or all) of the steps of the methodmay be performed by the main devicewith the aid of the trained machine learning model.
As used herein, the term “controller” may refer to any computing device, such as a computer, a laptop, a desktop, a cloud server, or the like. As will be appreciated by a person skilled in the art, the controller may include a processor, a machine learning module, and a storage. The processor may be any logic circuitry, such as an integrated circuit (IC) that receives (or fetches) instructions from a memory and processes the instructions and generates output. In some embodiments, the processor may be a microprocessor unit or a microcontroller unit. Some examples of microprocessor unit may include Intel processor (manufactured by Intel Corporation of Mountain View, California), Advanced Micro Devices processor (manufactured by Advanced Micro Devices of Sunnyvale, California), Snapdragon processor (manufactured by Qualcomm, San Diego, California, United States) and such processors. In some embodiments, the processor may be high performance processor configured to handle process intensive instructions associated with machine learning (ML) and artificial intelligence (AI). Examples of high-performance processor include AMD Rayzen (manufactured by Advanced Micro Devices of Sunnyvale, California), and Intel Core i9-i9) (manufactured by Intel Corporation of Mountain View, California). The processor may be a single core processor or a multi-core processor that is capable of handling instruction level parallelism and thread level parallelism and having different levels of cache.
As used herein, the term “memory” refers to a data storage unit having one or more memory chips capable of storing data and allowing any storage location to be directly accessed by the processing circuitry. The memory may be a Dynamic Random-Access Memory (DRAM) or any variants. In some embodiments, the memory may be a volatile memory or a non-volatile memory. The processing circuitry may communicate with the memory through IC interconnects (not shown).
128 124 124 124 128 1 FIG. As used herein, the phrase “trained machine learning model” (or machine learning module) may include a suitable hardware, set of instructions, or combination of hardware and instructions to support and enable the microprocessor(see) of the main deviceto learn and improve from experience without being explicitly programmed. Although the trained machine learning model is described as a part of the main device, in some implementations, the trained machine learning model may be external to the main deviceand communicatively coupled to the microprocessor.
2700 2702 2700 144 102 150 1 FIG. The methodis described in conjunction with. At step, the methodincludes recording an environmental reading from a sensor according to a time-step. In an embodiment, the microcontrollerin the sensorrecords the readings from the sensing unit.
2704 2700 144 156 124 124 At step, the methodincludes transmitting the environmental reading to a main device according to the time-step. In an embodiment, the microcontrollermay transmit the values of the readings to the access pointof the main device, where the main deviceaccumulates and stores multiple readings according to the time-step.
2706 2700 124 102 128 124 At step, the methodincludes triggering, via a circuity of the main device, a machine learning model when the plurality of environmental readings reaches a threshold number of readings. In an embodiment, when the readings gathered from the sensorreaches the threshold number of readings, for example 1000 readings, the microprocessorof the main devicemay trigger the trained machine learning model.
2708 2700 At step, the methodincludes identifying, via the machine learning model, an anomaly reading. Upon such triggering, the trained machine learning model may identify the anomaly reading. In an embodiment, the anomaly detection model may be used. The anomaly detection model may be one of Clustering, SVM, or Isolation Forest.
2710 2700 152 At step, the methodincludes categorizing, via the machine learning model, the anomaly reading into a positive anomaly reading, a negative anomaly reading, or an uncertain anomaly reading. In an embodiment, trained machine learning model may categorize the anomaly reading into the positive reading indicating an anomaly, the negative reading indicating a normal state, or the uncertain reading which requires the user to confirm the anomaly. The anomaly readings may be transmitted and stored in the cloud storage.
2712 2700 152 154 At step, the methodincludes transmitting, via a wide area network, the positive anomaly reading from the cloud storageto the end user communications devicewhen the anomaly reading is the positive anomaly reading.
2714 2700 154 1900 At step, the methodincludes generating, by the end user communications device, a first confirmation signal to confirm the positive anomaly reading. In an embodiment, the user may be allowed to confirm the positive anomaly reading as an anomaly. For example, movement of a store keeper in the indoor greenhousemay be reported as the positive anomaly reading, and the user may reject such readings as anomaly since the user is aware of such movements.
2716 2700 At step, the methodincludes retraining the machine learning model on the plurality of environmental readings coupled with the first confirmation signal. Based on the feedback from the user, the machine learning model may be retained to consider such movements as normal, so that similar data points in future are not reported as anomalies to the user.
2718 2700 124 152 At step, the methodincludes returning the environmental reading to the main devicefrom the cloud storagewhen the anomaly reading is the negative anomaly reading.
2720 2700 At step, the methodincludes retraining the machine learning model on the plurality of environmental readings coupled with the negative anomaly reading.
2722 2700 154 At step, the methodincludes transmitting, via the wide area network, the uncertainty anomaly reading to the end user communications device.
2724 2700 154 At step, the methodincludes generating, by the end user communications device, a second confirmation signal to confirm the uncertainty anomaly reading as the positive anomaly reading or the negative anomaly reading.
2726 2700 At step, the methodincludes retraining the machine learning model on the plurality of environmental readings coupled with the second confirmation signal.
100 2700 100 2700 To this end, the present disclosure provides systemand methodto detect anomalies in an adaptable way, while maintaining user privacy. The present disclosure helps users who are reluctant to install surveillance cameras that may be a cause for users' annoyance or apprehension about the security breach. Besides detecting anomalies as soon as they occur, the systemand methodof the present disclosure are effective in revealing the facts of disasters when they occur, through the data that is collected about the surroundings in that instance.
As used herein, the terms “a” and “an” and the like carry the meaning of “one or more.”
Numerous modifications and variations of the present invention are possible in light of the above teachings. It is, therefore, to be understood that, within the scope of the appended claims, the invention may be practiced otherwise than as specifically described herein.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
September 20, 2024
June 30, 2026
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.