Method and System for Thermal Infrared Facial Recognition

1. A method for thermal infrared facial recognition, characterized by comprising: determining a thermal infrared facial image to be recognized; inputting the thermal infrared facial image to be recognized into a trained three-sense component extractor to position eyes, a nose, and a mouth of a face in the image, and to perform regional positioning of a left eye, a right eye, the nose, and the mouth, so as to constitute a corresponding key region group, wherein the key region group further comprises coordinates of a centroid of a mouth region, coordinates of a centroid of a left eye region, and coordinates of a centroid of a right eye region; performing affine transformation on the thermal infrared facial image to be recognized using a difference between a key region group of a standard face and the key region group of the face to be recognized, so as to obtain a thermal infrared image after face alignment calibration; inputting the thermal infrared image after face alignment calibration into a trained facial feature extraction to extract corresponding facial features; and inputting the extracted facial features into a trained classifier to recognize information of an owner.

3. The method for thermal infrared facial recognition according to claim 2 , characterized in that a centroid of the nose region is: { X nose _ = 1 Nose total ⁢ ∑ j ⁢ x j , ∀ ( x j , y j ) ∈ Area Nose Y nose _ = 1 Nose total ⁢ ∑ j ⁢ y j , ∀ ( x j , y j ) ∈ Area Nose where, ( X Nose , Y nose ) represents coordinates of the centroid of the nose region; and Nose total represents a total number of pixels within the nose region, x j represents a width coordinate of a j th pixel in the nose region, and y j represents a height coordinate of the j th pixel in the nose region; and the centroid of the mouth region is: ⁢ { X Mouth _ = 1 Mouth total ⁢ ∑ k ⁢ x k , ∀ ( x k , y k ) ∈ Area Mouth Y Mout _ = 1 Mouth total ⁢ ∑ k ⁢ y k , ∀ ( x k , y k ) ∈ Area Mouth where, ( X mouth , Y mouth ) represents coordinates of the centroid of the mouth region; and Mouth total represents a total number of pixels within the mouth region, x k represents a width coordinate of a k th pixel in the mouth region, and y k represents a height coordinate of the k th pixel in the mouth region.

4. The method for thermal infrared facial recognition according to claim 3 , characterized in that the eye region is divided into the left eye region and the right eye region by adopting a connecting line of the centroid of the mouth region and the centroid of the nose region, and the specific division is as follows: Area Leye = { C ⁡ ( u , v ) = [ RGB Eyes ] ❘ 0 ≤ u ≤ ( X Mouth _ - X Nose _ ) * ( v - Y nose _ ) Y Mout _ - Y nose _ + X nose _ , 0 ≤ v ≤ H } Area Reye = { C ⁡ ( u , v ) = [ RGB Eyes ] ❘ ( X Mou _ - X Nose _ ) * ( v - Y nose _ ) Y Mout _ - X nose _ + Y nose _ ≤ u ≤ W , 0 ≤ v ≤ H } where, Area Leye represents the left eye region, and Area Reye represents the right eye region; and the coordinates of the centroid of the left eye region and the coordinates of the centroid of the right eye region are respectively: { X Leye _ = 1 Leye total ⁢ ∑ l ⁢ x l , ∀ ( x l , y l ) ∈ Area Leye Y Leye _ = 1 Leye total ⁢ ∑ l ⁢ y l , ∀ ( x l , y l ) ∈ Area Leye ⁢ ⁢ { X Reye _ = 1 Reye total ⁢ ∑ r ⁢ x r , ∀ ( x r , y r ) ∈ Area Reye Y Reye _ = 1 Reye total ⁢ ∑ r ⁢ y r , ∀ ( x r , y r ) ∈ Area Reye where, ( X Leye , Y Leye ) is the coordinates of the centroid of the left eye region, X Leye and Y Leye are respectively a width value and a height value of the left eye region, Leye total represents a sum of pixels in the left eye region, x l represents a width coordinate of an l th pixel in the left eye region, and y l represents a height coordinate of the l th pixel in the left eye region; and ( X Reye , Y Reye ) is the coordinates of the centroid of the right eye region, X Reye and Y Reye are respectively a width value and a height value of the right eye region, Reye total represents a sum of pixels in the right eye region, x r represents a width coordinate of a r th pixel in the right eye region, and y r represents a height coordinate of the r th pixel in the right eye region.

5. The method for thermal infrared facial recognition according to claim 3 , characterized by comprising: obtaining a thermal infrared image of a standard face through superposing and averaging pixels of a plurality of thermal infrared frontal facial images of a plurality of people, wherein the connecting line of the centroid of the left eye region and the centroid of the right eye region of the standard face is parallel to an X-axis of the image plane rectangular coordinate system, and the connecting line of the centroid of the nose region and the centroid of the mouth region of the standard face is parallel to a Y-axis of the image plane rectangular coordinate system; inputting the thermal infrared image of the standard face into the trained three-sense component extractor to obtain the key region group of the standard face, comprising the coordinates of the centroid of the mouth region, the coordinates of the centroid of the left eye region, and the coordinates of the centroid of the right eye region; performing affine transformation on the thermal infrared facial image to be recognized according to a difference between the key region group of the standard face and the key region group of the face to be recognized, specifically comprising: determining coordinates of the key region group of the standard face: X ¯ std , p = ⁢ 1 Q ⁢ ∑ s = 1 Q ⁢ ⁢ X ¯ s , p , p ∈ ( Mouth , Leye , Reye ) Y ¯ std , p = ⁢ 1 Q ⁢ ∑ s = 1 Q ⁢ ⁢ Y ¯ s , p , p ∈ ( Mouth , Leye , Reye ) where X std,p and Y std,p are respectively a width coordinate and a height coordinate of a centroid in a p region of the standard face, p regions are respectively a mouth Mouth region, a left eye Leye region, and a right eye Reye region, X s,p represents a width coordinate of the centroid in the p region of an s th frontal facial thermal infrared image, Y s,p represents a height coordinate of the centroid in the p region of the s th thermal infrared frontal facial image, and Q represents a total number of thermal infrared frontal facial images; determining an affine transformation matrix through a following formula: [ X ¯ std , p Y ¯ s ⁢ t ⁢ d , p 1 ] = [ a b c d e f 0 0 1 ] ⁡ [ x p y p 1 ] where, (x p , y p ) is respectively a width coordinate and a height coordinate of the centroid in the p region of the thermal infrared facial image to be recognized before face alignment, [ a b c d e f 0 0 1 ] is the affine transformation matrix, and a, b, c, d, e, and f are all parameters in the affine transformation matrix; and performing the affine transformation on the thermal infrared facial image to be recognized using the obtained affine transformation matrix to obtain the thermal infrared image after face alignment calibration.

11. A system for thermal infrared facial recognition, characterized by comprising: an image determination unit, configured to determine a thermal infrared facial image to be recognized; a region group construction unit, configured to input the thermal infrared facial image to be recognized into a trained three-sense component extractor to position eyes, a nose, and a mouth of a face in the image, and to perform regional positioning of a left eye, a right eye, the nose, and the mouth, so as to constitute a corresponding key region group, wherein the key region group further comprises coordinates of a centroid of a mouth region, coordinates of a centroid of a left eye region, and coordinates of a centroid of a right eye region; an image alignment unit, configured to perform affine transformation on the thermal infrared facial image to be recognized according to a difference between a key region group of a standard face and the key region group of the face to be recognized, so as to obtain a thermal infrared image after face alignment calibration; a feature extraction unit, for inputting the thermal infrared image after face alignment calibration into a trained facial feature extraction to extract corresponding facial features; and an image recognition unit, configured to input the extracted facial features into a trained classifier to recognize information of an owner.

13. The system for thermal infrared facial recognition according to claim 12 , characterized by further comprising: a standard face determination unit, configured to obtain a standard thermal infrared facial image through superposing and averaging pixels of a plurality of thermal infrared frontal facial images of a plurality of people, wherein a connecting line of the centroid of the left eye region and the centroid of the right eye region of the standard face is parallel to an X-axis of the image plane rectangular coordinate system, and a connecting line of a centroid of the nose region and a centroid of the mouth region of the standard face is parallel to a Y-axis of the image plane rectangular coordinate system; the region group construction unit, further configured to input a thermal infrared image of the standard face into the trained three-sense component extractor to obtain a key region group of the standard face, comprising coordinates of the centroid of the mouth region, coordinates of the centroid of the left eye region, and coordinates of the centroid of the right eye region; and the image alignment unit, configured to determine coordinates of the key region group of the standard face through following formulae: X ¯ std , p = ⁢ 1 Q ⁢ ∑ s = 1 Q ⁢ ⁢ X ¯ s , p , p ∈ ( Mouth , Leye , Reye ) ; Y ¯ std , p = ⁢ 1 Q ⁢ ∑ s = 1 Q ⁢ ⁢ Y ¯ s , p , p ∈ ( Mouth , Leye , Reye ) ; where, X std,p and Y std,p are respectively a width coordinate and a height coordinate of a centroid in a p region of the standard face, p regions are respectively a mouth Mouth region, a left eye Leye region, and a right eye Reye region, X s,p represents a width coordinate of the centroid in the p region of an s th frontal facial thermal infrared image, Y s,p represents a height coordinate of the centroid in the p region of the s th thermal infrared frontal facial image, and Q represents a total number of thermal infrared frontal facial images; determine an affine transformation matrix through a following formula: [ X ¯ s ⁢ t ⁢ d ′ ⁢ p Y ¯ s ⁢ t ⁢ d , p 1 ] = [ a b c d e f 0 0 1 ] ⁡ [ x p y p 1 ] ; where, (x p , y p ) is respectively a width coordinate and a height coordinate of the centroid in the p region of the thermal infrared facial image to be recognized before face alignment, [ a b c d e f 0 0 1 ] is the affine transformation matrix, and a, b, c, d, e, and f are all parameters in the affine transformation matrix; and perform affine transformation on the thermal infrared facial image to be recognized according to the obtained affine transformation matrix to obtain the thermal infrared image after face alignment calibration.

14. The system for thermal infrared facial recognition according to claim 11 , characterized in that the feature extraction unit is configured to construct a neural network for facial feature recognition to form a corresponding facial feature extraction, wherein the facial feature extraction enables a degree of similarity of features of thermal infrared facial images of a same person to be high and a degree of similarity of features of thermal infrared facial images of different people to be low; a loss function L of the facial feature extraction may adopt two types below: a first loss function is: L=∥feature−HoC∥ 2 −∥feature−HeC∥ 2 ; where, feature is a feature of an input thermal infrared image extracted through the facial feature extraction, HoC is a category center feature of a category of the input thermal infrared image, and HeC is a category average feature of a certain category in which the input thermal infrared image is divided into; and for a wrongly classified thermal infrared image, a distance between the feature extracted from the thermal infrared image and a category center thereof is decreased while a distance between the feature and a wrongly classified category center is increased; and a second loss function is: L=∥feature−HoC∥ 2 ; where, feature is the feature of the input thermal infrared image extracted by the facial feature extraction, and HoC is a feature vector of a category center of the input thermal infrared image; and for the wrongly classified thermal infrared image, only the distance between the feature extracted from the thermal infrared image and the category center thereof is decreased.

15. The system for thermal infrared facial recognition according to claim 12 , characterized in that the feature extraction unit is configured to construct a neural network for facial feature recognition to form a corresponding facial feature extraction, wherein the facial feature extraction enables a degree of similarity of features of thermal infrared facial images of a same person to be high and a degree of similarity of features of thermal infrared facial images of different people to be low; a loss function L of the facial feature extraction may adopt two types below: a first loss function is: L=∥feature−HoC∥ 2 −∥feature−HeC∥ 2 ; where, feature is a feature of an input thermal infrared image extracted through the facial feature extraction, HoC is a category center feature of a category of the input thermal infrared image, and HeC is a category average feature of a certain category in which the input thermal infrared image is divided into; and for a wrongly classified thermal infrared image, a distance between the feature extracted from the thermal infrared image and a category center thereof is decreased while a distance between the feature and a wrongly classified category center is increased; and a second loss function is: L=∥feature−HoC∥ 2 ; where, feature is the feature of the input thermal infrared image extracted by the facial feature extraction, and HoC is a feature vector of a category center of the input thermal infrared image; and for the wrongly classified thermal infrared image, only the distance between the feature extracted from the thermal infrared image and the category center thereof is decreased.

16. The system for thermal infrared facial recognition according to claim 13 , characterized in that the feature extraction unit is configured to construct a neural network for facial feature recognition to form a corresponding facial feature extraction, wherein the facial feature extraction enables a degree of similarity of features of thermal infrared facial images of a same person to be high and a degree of similarity of features of thermal infrared facial images of different people to be low; a loss function L of the facial feature extraction may adopt two types below: a first loss function is: L=∥feature−HoC∥ 2 −∥feature−HeC∥ 2 ; where, feature is a feature of an input thermal infrared image extracted through the facial feature extraction, HoC is a category center feature of a category of the input thermal infrared image, and HeC is a category average feature of a certain category in which the input thermal infrared image is divided into; and for a wrongly classified thermal infrared image, a distance between the feature extracted from the thermal infrared image and a category center thereof is decreased while a distance between the feature and a wrongly classified category center is increased; and a second loss function is: L=∥feature−HoC∥ 2 ; where, feature is the feature of the input thermal infrared image extracted by the facial feature extraction, and HoC is a feature vector of a category center of the input thermal infrared image; and for the wrongly classified thermal infrared image, only the distance between the feature extracted from the thermal infrared image and the category center thereof is decreased.