An action recommendation system uses reinforcement learning that provides a next action recommendation to a traffic controller to give to a vehicle pilot such as an aircraft pilot. The action recommendation system uses data of past human actions to create a reinforcement learning model and then uses the reinforcement learning model with current ABS-B data to provide the next action recommendation to the traffic controller. The action recommendation system may use an anisotropic reward function and may also include an expanding state space module that uses a non-uniform granularity of the state space.
Legal claims defining the scope of protection, as filed with the USPTO.
2. The apparatus of claim 1 wherein the action recommendation system further comprises an anisotropic reward function to exploit that the destination is known.
3. The apparatus of claim 2 wherein the anisotropic reward function uses flight criteria comprising: vehicle type, length of the trip, lateness of the arrival, weather conditions, and remaining fuel.
4. The apparatus of claim 1 wherein the past human actions are sourced from Automatic Dependent Surveillance Broadcast data.
5. The apparatus of claim 1 wherein the current vehicle data is derived from Automatic Dependent Surveillance Broadcast data for the current vehicle.
6. The apparatus of claim 1 wherein the reinforcement learning model comprises a neural network with deep Q reinforcement learning.
8. The method of claim 7 wherein the reward function is an anisotropic reward function that exploits that the destination is known.
9. The method of claim 8 wherein the reward function uses flight criteria comprising: the aircraft type, length of the trip, lateness of the arrival, weather conditions, FAA regulations, and remaining fuel.
10. The method of claim 7 wherein the past human actions are derived from Automatic Dependent Surveillance Broadcast data.
11. The method of claim 7 wherein the current aircraft data is derived from Automatic Dependent Surveillance Broadcast data of the aircraft.
12. The method of claim 7 wherein the reinforcement learning model is a neural network used with deep Q reinforcement learning.
14. The method of claim 13 wherein the past human actions are derived from Automatic Dependent Surveillance Broadcast data.
15. The method of claim 13 wherein the reinforcement learning model comprises a neural network with deep Q reinforcement learning.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
May 21, 2019
August 9, 2022
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.