Classroom teaching cognitive load measurement system

1. A classroom teaching cognitive load measurement system, comprising: a task completion feature collecting module comprising a first computer, a second computer and a data processing center for recording an answer response time and a correct answer rate of a student when completing a task; a cognitive load self-rating collecting module for receiving a cognitive load self-rating scale input by the student when completing the task indicating a degree of mental effort and a task subjective difficulty of the student by the cognitive load self-rating scale; an expression and attention feature collecting module comprising a plurality of cameras and a first processor, for collecting a student classroom performance video to obtain a face region through a face detection; performing a face recognition on a region containing faces to complete an expression recognition and a head pose estimation; counting a smiley face duration of the student according to a result of the expression recognition and counting a watching duration of the student according to a result of the head pose estimation; a second processor configured to form a characteristic vector comprising six indexes including the answer response time, the correct answer rate, the degree of mental effort, the task subjective difficulty, the smiley face duration and the watching duration; and a third processor configured to input the characteristic vector to a trained classifier for a cognitive load rating to identify a classroom cognitive load level of the student.

2. The classroom teaching cognitive load measurement system of claim 1 , wherein the expression and attention feature collecting module comprises: a monitoring data collecting module comprising the plurality of cameras for capturing a scene image in a classroom; the first processor, configured to: detect the face region from the scene image; and perform the face recognition, the expression recognition and the head pose estimation by using models pre-trained.

3. The classroom teaching cognitive load measurement system of claim 2 , wherein the model for the face recognition is trained by: (1) using a visual geometry group (vgg)-face face database, and selecting a VGG model to train the model for the face recognition for a first time; and (2) optimizing the model for the face recognition obtained from the training step (1) by using a face image of the student in the classroom.

4. The classroom teaching cognitive load measurement system of claim 3 , wherein the model for the expression recognition and the head pose estimation is trained by: (3) a network initialization, wherein a VGG network is constructed and parameters in the VGG network are initialized by parameters of the model for the face recognition trained in the step (2); a first fully connected layer (FC1) in the VGG network is replaced with a latent variable analysis (LVA) layer; a second fully connected layer (FC2) is replaced with two marginal Fisher analysis layers: MFA_S layer and an MFA_H layer, which are two branches of an output of the LVA layer; the MFA_S layer and the MFA_H layer are then connected to a third fully connected layer (FC3) respectively, and the two branches are used in a smiley face recognition and the head pose estimation, respectively; (4) an LVA layer training, including: performing a hidden variable modeling t=Us+Vh+ t +ε on an observed quantity output by a convolution layer, t being the observed quantity output by the convolution layer, s and h representing a smiley face feature hidden variable and a head pose feature hidden variable respectively, t representing a mean, ε being a Gaussian noise, σ being a variance, U and V being transformation matrixes; outputting samples to the model for the smiley face recognition and head pose estimation, and solving a model parameter θ={U,V, t ,σ} through a maximized likelihood function L LVA =Σ i lnp θ (t i ,h j ,s k ), wherein a logarithmic function lnp θ ( ) indicates a likelihood of an i-th sample t i for having a head pose hidden variable h j with a head pose belonging to a j-th class and an expression hidden variable s k with an expression belonging to a k-th class; calculating outputs of the two branches of the LVA layer according to trained parameters, the output of the branch of the expression recognition being x s =W s t+b s , wherein W s =U T Σ −1 t ; the output of the branch of the head pose being x h =W h t+b h , wherein W h =V T Σ −1 , b h =V T Σ −1 t , and a superscript T represents a transposition; (5) MFA layers training, including: based on a MFA_S layer having an input being the output X s of the branch of the expression recognition and an output being y s =P s T x s , finding an eigenvector P s that enables an optimization arg ⁢ ⁢ max P s ⁢ ⁢ tr ⁡ ( P s T ⁢ S b ⁢ P s ) tr ⁡ ( P s T ⁢ S w ⁢ P s ) by minimizing an intraclass distance and maximizing an interclass distance, wherein S b is the interclass distance, and S W is the intraclass distance; solving the optimization is equivalent to solve an eigenvalue decomposition: S b p s =(S w +λI)p s with λ being an eigenvalue, P s being the eigenvector, I being a unit matrix; training an MFA_H layer in the same way with an output being the output X h of the branch of the head pose, an output being y h =P h T x h , and P h being the eigenvector; and (6) fixing parameters of the LVA layer and the MFA layers, fine-tuning parameters of other network layers to achieve minimizing a total loss function output by the two branches of the LVA layer, so as to obtain the model for the expression recognition and the head pose estimation.

6. The classroom teaching cognitive load measurement system of claim 1 , wherein the first processor adopts from a hierarchical cascading AdaBoost, a Hidden Markov Model (HMM) or a Support Vector Machine (SVM), and a feature used for a detection is selected from a Haar feature, a Sobel feature or a sparse feature.

7. The classroom teaching cognitive load measurement system of claim 1 , wherein the classifier adopted by the third processor is selected from a self-organizing neural network, an Elman neural network or a support vector machine.

8. The classroom teaching cognitive load measurement system of claim 2 , wherein the first processor adopts from a hierarchical cascading AdaBoost, a Hidden Markov Model (HMM) or a Support Vector Machine (SVM), and a feature used for a detection is selected from a Haar feature, a Sobel feature or a sparse feature.

9. The classroom teaching cognitive load measurement system of claim 3 , wherein the first processor adopts from a hierarchical cascading AdaBoost, a Hidden Markov Model (HMM) or a Support Vector Machine (SVM), and a feature used for a detection is selected from a Haar feature, a Sobel feature or a sparse feature.

10. The classroom teaching cognitive load measurement system of claim 4 , wherein the first processor adopts from a hierarchical cascading AdaBoost, a Hidden Markov Model (HMM) or a Support Vector Machine (SVM), and a feature used for a detection is selected from a Haar feature, a Sobel feature or a sparse feature.

11. The classroom teaching cognitive load measurement system of claim 5 , wherein the first processor adopts from a hierarchical cascading AdaBoost, a Hidden Markov Model (HMM) or a Support Vector Machine (SVM), and a feature used for a detection is selected from a Haar feature, a Sobel feature or a sparse feature.

12. The classroom teaching cognitive load measurement system of claim 2 , wherein the classifier adopted by the third processor is selected from a self-organizing neural network, an Elman neural network or a support vector machine.

13. The classroom teaching cognitive load measurement system of claim 3 , wherein the classifier adopted by the third processor is selected from a self-organizing neural network, an Elman neural network or a support vector machine.

14. The classroom teaching cognitive load measurement system of claim 4 , wherein the classifier adopted by the third processor is selected from a self-organizing neural network, an Elman neural network or a support vector machine.

15. The classroom teaching cognitive load measurement system of claim 5 , wherein the classifier adopted by the third processor is selected from a self-organizing neural network, an Elman neural network or a support vector machine.

16. A method for classroom teaching cognitive load measurement, the method comprising: recording an answer response time and a correct answer rate of a student when completing a task; receiving a cognitive load self-rating scale input by the student when completing the task indicating a degree of mental effort and a task subjective difficulty of the student by the cognitive load self-rating scale; collecting a student classroom performance video to obtain a face region through a face detection; performing a face recognition on a region containing faces to complete an expression recognition and a head pose estimation; counting a smiley face duration of the student according to a result of the expression recognition and counting a watching duration of the student according to a result of the head pose estimation; forming a characteristic vector comprising six indexes including the answer response time, the correct answer rate, the degree of mental effort, the task subjective difficulty, the smiley face duration and the watching duration and inputting the characteristic vector to a trained classifier for a cognitive load rating to identify a classroom cognitive load level of the student.