Method of analyzing eye tracking data for estimating user's cognitive and emotional level of consumption of visual information. A training machine learning model is trained using a data set containing gaze information of known training users, their known cognitive levels and their EEG signal measurements. A calibrating machine learning model is trained using a data set of calibrating visual information displayed to a user, calibrating gaze tracks of that user, calibrating actions data of that user, and calibrating session data related to the session environment. The device displays to that user a target visual information and records target eye tracking data of that user in response to consuming the target information. The recorded target eye tracking data is calibrated via the calibrating machine learning model. The calibrated target eye tracking data is fed into the training machine learning model, which estimates the cognitive levels of consumption of the target visual information of that user.
Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
2. The method of claim 1, further comprises recording one of a neural signals, bio-electric signals, captures of facial expressions and any of biometrical measurements of the user in response to the user consuming one of the calibration visual information and the target visual information.
4. The method of claim 3, wherein the training data set includes one of a neural signals, bio-electric signals, captures of facial expressions and any of biometrical measurements of a training user in response to consuming the training visual information.
5. The method of claim 1, wherein the display includes one of a smartphone display, a computer display, a tablet display, a television display, a projector display, a paper, a whiteboard, a blackboard, an interactive display and a display configured for displaying one of a calibrated visual information and a target visual information.
6. The method of claim 1, wherein the one of the target cognitive level and the target emotional level include data related to one of attention, cognitive concentration, cognitive load, cognitive fatigue, stress level, emotional state, emotional strain, reaction time, fatigue, mental effort and engagement.
7. The method of claim 1, the target session data of the user is augmented with any one measurement related to one of durations of fixations, measurements related to amplitudes of saccades, ratios of measurements related to durations of fixations and measurements related to the amplitudes of saccades, estimates of long and extra-long fixations of durations of fixations, measurements related to electroencephalography asymmetry, measurements related to spectral characteristics of the electroencephalography alpha rhythm, measurements related to lower alpha range suppression level, data related to electroencephalography, fatigue index, data related to alpha electroencephalography index, data related to intrusive saccades, data related to orthodromic saccades, data related to and regressive saccades, measurements related to blinking activity, measurements related to changes in skin characteristics, for example, electrical conductivity, data related to Baevsky index, data related to Kaplan index, data related to optically obtained plethysmograms, measurements related to micromimic of a face data, measurements related to saccade latency, measurements related to respiratory activity, measurements related to electroencephalography alpha rhythm desynchronization, and data related to electroencephalography ratio index.
This invention relates to enhancing user session data with physiological and behavioral measurements to improve analysis of cognitive and physiological states. The method augments target session data with various metrics derived from eye-tracking, electroencephalography (EEG), skin conductivity, facial micromimic, respiratory activity, and other physiological signals. Key measurements include fixation durations, saccade amplitudes, EEG asymmetry, alpha rhythm spectral characteristics, fatigue indices, blink activity, and plethysmographic data. Additional metrics cover saccade types (intrusive, orthodromic, regressive), latency, and skin conductivity changes. The system also incorporates indices like Baevsky and Kaplan, which assess stress and cognitive load. By combining these diverse physiological signals, the method provides a comprehensive assessment of user states, enabling applications in fatigue monitoring, cognitive workload analysis, and user experience optimization. The approach leverages multimodal data fusion to enhance the accuracy and depth of insights derived from user interactions.
8. The method of claim 1, further including recording during one of a calibration session and a target session at least one measurements related to one of durations of fixations, measurements related to amplitudes of saccades, ratios of the measurements related to durations of fixations and measurements related to the amplitudes of saccades, estimates of long and extra-long fixations of the durations of fixations, measurements related to electroencephalography asymmetry, measurements related to spectral characteristics of the electroencephalography alpha rhythm, measurements related to lower alpha range suppression level, data related to electroencephalography fatigue index, data related to alpha electroencephalography index, data related to intrusive saccades, data related to orthodromic saccades, data related to and regressive saccades, measurements related to blinking activity, measurements related to changes in skin characteristics, data related to Baevsky index, data related to Kaplan index, data related to optically obtained plethysmograms, measurements related to facial expressions data, measurements related to saccade latency, measurements related to respiratory activity, measurements related to electroencephalography alpha rhythm desynchronization, and data related to electroencephalography ratio index.
This invention relates to a method for monitoring cognitive and physiological states by recording various biometric and behavioral measurements during calibration or target sessions. The method involves capturing data related to eye movements, including fixation durations, saccade amplitudes, ratios of fixation durations to saccade amplitudes, and estimates of long and extra-long fixations. It also records electroencephalography (EEG) data, such as asymmetry, spectral characteristics of the alpha rhythm, lower alpha range suppression levels, fatigue index, alpha index, and alpha rhythm desynchronization. Additional measurements include intrusive, orthodromic, and regressive saccades, blinking activity, skin characteristic changes, Baevsky and Kaplan indices, optically obtained plethysmograms, facial expressions, saccade latency, respiratory activity, and EEG ratio index. The method provides a comprehensive approach to assessing cognitive load, fatigue, attention, and other physiological states by analyzing these diverse biometric and behavioral parameters. This can be applied in fields such as human-computer interaction, driver monitoring, medical diagnostics, and cognitive research.
9. The method of claim 1, further including recording data by electroencephalography applications, devices and hardware.
10. The method of claim 1, wherein training the calibrating machine learning model includes one of applying independent component analysis, training a computer neural network, applying a stacked autoencoder model, and applying a distributed regression trees model.
11. The method of claim 3, wherein one of the training machine learning model and the calibrating machine learning model includes one of regression algorithms, regularization algorithms, decision tree algorithms, bayesian algorithms, clustering algorithms, association rule learning algorithms, artificial neural network algorithms, deep learning algorithms, dimensionality reduction algorithms, ensemble algorithms, and computer vision algorithms.
This method evaluates a user's consumption of visual information (displayed on various mediums like smartphones, computers, or even paper) by analyzing their eye tracking data and various bioresponse data. This includes training a machine learning model using a dataset of a training user's bio-electric signals, neural signals, facial expressions, or other biometric measurements collected while they consume specific training visual information. This model is developed to evaluate cognitive levels (such as attention, cognitive load, or fatigue) or emotional levels (like stress or emotional state) related to the visual information. For either this **training machine learning model** or a separate **calibrating machine learning model** (used to adapt the system for a specific user), the system can implement a variety of algorithms. These include: regression algorithms, regularization algorithms, decision tree algorithms, Bayesian algorithms, clustering algorithms, association rule learning algorithms, artificial neural network algorithms, deep learning algorithms, dimensionality reduction algorithms, ensemble algorithms, and computer vision algorithms. ERROR (embedding): Error: Failed to save embedding: Could not find the 'embedding' column of 'patent_claims' in the schema cache
12. The method of claim 1, wherein one of the training machine learning model and calibrating machine learning model include a convolutional neural network, a fully-connected neural network operatively arranged within a network architecture.
13. The method of claim 1, wherein the target data set includes raw data from one of an analogue and a digital signal generated by one of an eye tracking device, an electroencephalography device and a bioelectric signal measurement device.
This invention relates to processing raw data from physiological monitoring devices to improve data quality and usability. The method involves receiving raw data from an eye tracking device, an electroencephalography (EEG) device, or a bioelectric signal measurement device, where the data may be in either analog or digital form. The raw data is then processed to remove noise and artifacts, enhancing signal clarity. The processed data is subsequently analyzed to extract meaningful patterns or features, which can be used for applications such as medical diagnostics, user interface control, or cognitive state monitoring. The method ensures that the raw data, whether from eye movement tracking, brainwave activity, or other bioelectric signals, is accurately captured and refined for reliable analysis. By addressing the challenges of signal distortion and interference, this approach enables more precise and actionable insights from physiological measurements. The technique is particularly useful in environments where signal integrity is critical, such as clinical settings or human-computer interaction systems.
14. The method of claim 13, wherein the raw data is transformed into frame-level feature representations by a signal transformation algorithm, the frame-level feature representations being a tensor having a time axis and a frequency axis that are input into a convolutional neural network and generate segment-level representations by a pooling algorithm.
15. The method of claim 14, wherein the segment-level representations are associated with representations of graze tracks, and the segment-level representations and representations of gaze tracks are input into a recurrent neural network, the recurrent neural network transforms the segment-level representations and representations of gaze tracks into frame-level estimates associated with cognitive level estimates and emotional level estimates for a given time interval.
This invention relates to analyzing human cognitive and emotional states using gaze tracking and neural networks. The problem addressed is the need for accurate, real-time assessment of cognitive and emotional responses from visual attention data, such as eye movements and gaze patterns. The method involves processing gaze track data to generate segment-level representations, which are then combined with representations of gaze tracks. These combined inputs are fed into a recurrent neural network (RNN), which processes the data sequentially over time. The RNN transforms the segment-level representations and gaze track data into frame-level estimates, providing both cognitive and emotional state assessments for specific time intervals. The system leverages temporal dependencies in gaze behavior to improve the accuracy of state estimation. The approach integrates gaze tracking with advanced machine learning to enable dynamic, context-aware analysis of mental and emotional states, useful in applications like user experience research, mental health monitoring, and adaptive human-computer interaction. The method ensures that both cognitive and emotional dimensions are captured simultaneously, offering a comprehensive understanding of a person's response to visual stimuli.
16. The method of claim 15, wherein the recurrent neural network has a set of parameters, each parameter being derived by iteratively applying a set of that recurrent neural network functions, the neural network functions include an optimizing a loss function by of one of a mean squared error and a cross-entropy function, such that the recurrent neural network applies weights to input data, the loss function is calculated, the loss function is optimized by changing the weights within the recurrent neural network, the recurrent neural network repeats the process until a predetermined threshold is reached.
17. The method of claim 1, wherein estimating one of a target cognitive and target emotional levels of consumption of the target visual information of the user at several target gaze tracks from applying of the training machine learning model to the calibrated target data set further includes estimating an independent measurement tensor for one of an attention, a cognitive concentration, a cognitive load, a cognitive fatigue, a stress level, an emotional state, an emotional strain, a reaction time, a fatigue, a mental effort and an engagement.
18. The method of claim 1, wherein the training machine learning model includes a recurrent neural network within its architecture, the recurrent neural network is a bi-directional long short-term memory network, and input data for the recurrent neural network includes tensors of representations that are generated from gaze tracks and from segment-level representations that are derived from processing electroencephalography data by a combination of a convolutional neural network and pooling algorithms applied to a number of frame representations generated by the convolutional neural network for a given time segment.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
July 28, 2020
October 11, 2022
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.