A human machine interface (HMI), a method of interacting with an HMI and a corresponding computer program product facilitate user interaction with an HMI. In a method, a face of a person attempting to interact with the HMI is detected and a gesture made by the person relative to the HMI is also detected. The method analyzes information regarding the face of the person that has been detected to determine whether the person is wearing personal protection equipment. In an instance in which the person is determined to not be wearing personal protection equipment, the method suspends a response to the gesture made by the person. However, in an instance in which the person is determined to be wearing personal protection equipment, the method analyzes the gesture made by the person relative to the HMI and converts the gesture to a corresponding command to the system associated with the HMI.
Legal claims defining the scope of protection, as filed with the USPTO.
2. A method according to claim 1 wherein the personal protection equipment comprises a face mask.
3. A method according to claim 1 wherein analyzing information regarding the face of the first person comprises estimating point cloud data representative of the face of the first person to determine whether the first person is wearing the personal protection equipment.
4. A method according to claim 1 wherein detecting the face of the first person attempting to interact with the HMI and detecting the gesture made by the first person relative to the HMI comprises detecting the face of the first person based upon signals received from a first sensor and detecting the gesture made by the first person relative to the HMI based upon signals received from a second sensor, different than the first sensor.
5. A method according to claim 4 wherein the first sensor comprises a near infrared (NIR) sensor and the second sensor comprises an electro-optic (EO) sensor.
6. A method according to claim 1 wherein detecting the face of the first person attempting to interact with the HMI, analyzing information regarding the face of the first person that has been detected to determine whether the first person is wearing personal protection equipment, analyzing the gesture made by the first person relative to the HMI and converting the gesture to the corresponding command are performed utilizing one or more convolutional neural networks.
7. A method according to claim 1, wherein the HMI is arranged in front of a seat, wherein detecting the face of the first person attempting to interact with the HMI further comprises verifying an identity of the first person as corresponding to the seat.
9. A human machine interface according to claim 8 wherein the personal protection equipment comprises a face mask.
10. A human machine interface according to claim 8 wherein the processing circuitry is configured to analyze information regarding the face of the first person by estimating point cloud data representative of the face of the first person to determine whether the first person is wearing the personal protection equipment.
12. A human machine interface according to claim 11 wherein the first sensor comprises a near infrared (NIR) sensor and the second sensor comprises an electro-optic (EO) sensor.
13. A human machine interface according to claim 8 wherein the processing circuitry comprises one or more convolutional neural networks configured to detect the face of the first person attempting to interact with the HMI, analyze information regarding the face of the first person that has been detected to determine whether the first person is wearing personal protection equipment, analyze the gesture made by the first person relative to the HMI and convert the gesture to the corresponding command.
14. A human machine interface according to claim 8, wherein the HMI is arranged in front of a seat, wherein the processing circuitry configured to detect the face of the first person attempting to interact with the HMI further comprises processing circuitry configured to verify an identity of the first person as corresponding to the seat.
16. A computer program product according to claim 15 wherein the personal protection equipment comprises a face mask.
17. A computer program product according to claim 15 wherein the program code instructions to analyze information regarding the face of the first person comprise program code instructions to estimate point cloud data representative of the face of the first person to determine whether the first person is wearing the personal protection equipment.
18. A computer program product according to claim 15 wherein the program code instructions to detect the face of the first person attempting to interact with the HMI and detect the gesture made by the first person relative to the HMI comprise program code instructions to detect the face of the first person based upon signals received from a first sensor and detect the gesture made by the first person relative to the HMI based upon signals received from a second sensor, different than the first sensor.
19. A computer program product according to claim 15 wherein the program code instructions comprise one or more convolutional neural networks configured to detect the face of the first person attempting to interact with the HMI analyze information regarding the face of the first person that has been detected to determine whether the first person is wearing personal protection equipment, analyze the gesture made by the first person relative to the HMI and convert the gesture to the corresponding command.
20. A computer program product according to claim 15, wherein the HMI is arranged in front of a seat, wherein the program code instructions to detect the face of the first person attempting to interact with the HMI further comprise program code instructions to verify an identity of the first person as corresponding to the seat.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
December 14, 2021
January 30, 2024
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.