Described herein is a pilot monitoring system (100) including a camera (106) configured to capture a plurality of digital images of an aircraft (104) cockpit including one or both of a pilot (102) and a co-pilot (103) and a processor (114) configured to process the captured images. The images are processed to determine an alertness state of the pilot (102) and/or the co-pilot (103) during operation of the aircraft (104) based on detected facial features of the pilot (102) and/or co-pilot (103) in the images. Processor (114) receives information indicative of a current phase of flight of the aircraft and displays a visualisation of the alertness level and/or determines a suitability of a controlled rest period for the pilot (102) or co-pilot (103) based on the alertness state of the pilot and (102)/or the co-pilot (103) and the current phase of flight. An alert is issued alert to the pilot (102) and/or co-pilot (103) including a determination of a suitability of a controlled rest period for either the pilot (102) or co-pilot (103).
Legal claims defining the scope of protection, as filed with the USPTO.
2. The method according to claim 1 wherein the processor is further configured to receive input indicative of sleep history and/or a recent duty history of the pilot and/or co-pilot and wherein the determination of a controlled rest period for the pilot or co-pilot is based on the sleep/duty history.
3. The method according to claim 1 including the step of receiving input from the pilot or co-pilot to specify a controlled rest event.
4. The method according to claim 1 including the step of performing facial recognition on at least a subset of the digital images to detect a presence of known cabin crew members in the cockpit.
5. The method according to claim 1 including receiving biometric information of the pilot and/or co-pilot from a biometric reader device and wherein the determination of a controlled rest period for either the pilot or co-pilot is based on the received biometric information.
6. The method according to claim 1 wherein the alertness state is calculated based at least in part on a detected body position of the pilot and/or co-pilot in the received images.
7. The method according to claim 1 wherein the alertness state is calculated based at least in part on detected head and/or body motion of the pilot and/or co-pilot across a plurality of the received images.
8. The method according to claim 1 wherein the alertness state is calculated based at least in part on detected mouth movement of the pilot and/or co-pilot across a plurality of the received images.
9. The method according to claim 1 wherein the alertness state is calculated based at least in part on detected speech of the pilot and/or co-pilot.
10. The method according to claim 1 wherein the alertness state is characterized at least in part by a drowsiness measure based on the Karolinska Sleepiness Scale or Samn-Perelli scale.
11. The method according to claim 1 including the step of issuing a wake up alert to a resting pilot once a period of controlled rest ends.
12. The method according to claim 1 wherein the step of displaying the alertness state to the pilot and co-pilot includes visualising the alertness state on a graphical user interface in the cockpit.
13. The method according to claim 1 including the step of issuing a pilot briefing to the pilot or co-pilot based on a current or past alertness state.
14. The method according to claim 13 wherein the pilot briefing is issued during the flight.
15. The method according to claim 13 wherein the pilot briefing is issued post-flight.
16. The method according to claim 15 wherein the post-flight briefing includes a current or future predicted fatigue or alertness level of pilot or co-pilot.
17. The method according to claim 15 wherein the post-flight briefing includes a future flight or rest schedule for the pilot or co-pilot.
18. The method according to claim 1 including the step of issuing a first alert to the pilot and/or co-pilot including a determination of a controlled rest period for either the pilot or co-pilot.
19. The method according to claim 18 including the step of detecting a controlled rest event for the pilot or co-pilot based on detection of a sleep state or an elevated drowsiness state and a current phase of flight.
20. The method according to claim 19 including the step of monitoring the visual alertness state of the pilot or co-pilot who is awake during a detected controlled rest period of either the pilot or co-pilot.
21. The method according to claim 20 including the step of issuing a second alert if the pilot or co-pilot who is awake enters a distracted, low vigilance drowsy, asleep or incapacitated attention state.
22. The method according to claim 21 wherein the second alert includes an alert that a pilot is drowsy and which is issued to an in-aircraft crew rest facility within the aircraft.
23. The method according to claim 21 wherein the second alert includes a notification that a controlled rest period has been entered and which is issued to a cabin crew notification device to alert a cabin crew member.
24. The method according to claim 21 wherein the second alert is issued to a ground terminal to alert a dispatch, safety or operations member of a controlled rest period.
25. The method according to claim 1 wherein the post-flight briefing provides risk analysis and pilot pairing analysis.
26. The method according to claim 1 wherein the post-flight briefing is based at least in part on historical objective pilot fatigue data.
28. The system according to claim 27 including a single wide angle camera positioned to simultaneously image the pilot and co-pilot.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
January 10, 2022
September 17, 2024
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.