A method to control a device may include forming an integrated simulation model of an actual environment in which the device is or will be operating. The integrated simulation model may be formed using pre-existing data and real-time data related to the actual environment. The method may also include presenting a simulation including a representation of the device operable in the integrated simulation model of the actual environment and allowing control of operation of the simulation of the device in the integrated simulation model of the actual environment. The method may further include controlling operation of the device in the actual environment using the simulation of the representation of the device in the integrated simulation model of the actual environment.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A method to control a device, comprising: forming an integrated simulation model of an actual environment in which the device is or will be operating, wherein the integrated simulation model is formed using pre-existing data and real-time data related to the actual environment, the pre-existing data comprising at least one of imagery of the actual environment, terrain contours, weather predictions, descriptions and locations of any points of interest, and locations and descriptions of any threats or hazards to the device in the actual environment, and wherein the pre-existing data is obtained from at least one of a satellite, aerial reconnaissance, ground reconnaissance and a sensor; presenting a simulation including a representation of the device operable in the integrated simulation model of the actual environment; allowing control of the simulation of the representation of the device in the integrated simulation model of the actual environment; and controlling operation of the device in the actual environment using the simulation of the device in the integrated simulation model of the actual environment.
2. The method of claim 1 , further comprising allowing control of operation of the representation of the device to perform a simulated function in the integrated simulation model of the actual environment a predetermined time period before the device in the actual environment performs an actual function corresponding to the simulated function previously performed in the integrated simulation model.
3. The method of claim 2 , further comprising allowing adjustment of the predetermined time period by permitting speeding up and slowing down of the simulation of the device in the integrated simulation model of the actual environment.
4. The method of claim 2 , further comprising allowing a selected portion of the simulation to be re-done in response to the predetermined time period being sufficiently long to permit the selected portion to be re-done and transferred to the device in the actual environment before performance of any functions corresponding to the selected portion of the simulation to be re-done.
5. The method of claim 2 , further comprising: allowing the simulation to be slowed-down to provide additional time for adjustments and control of the representation of the device within the simulation and decision making; and allowing the simulation to be speed-up to increase the predetermined time period.
6. The method of claim 1 , further comprising: permitting operation of the simulation of the device in the integrated simulation model of the actual environment with altered performance characteristics compared to the device in the actual environment; and compensating for any operating limitations of the device in the actual environment.
7. The method of claim 1 , allowing selection of a viewpoint from which the simulation of the device in the integrated simulation model of the actual environment may be viewed and controlled.
8. The method of claim 1 , further comprising: creating a control script from the simulation of the device in the integrated simulation model of the actual environment; and transferring the control script to the device in the actual environment to control operation of the device.
9. The method of claim 1 , wherein the device comprises one of an aerospace vehicle, a terrestrial vehicle, a watercraft and a machine.
10. A method to control a device, comprising: obtaining pre-existing data related to an actual environment in which the device is or will be operating to create a simulation of the actual environment; generating a first virtual model of the actual environment based on the pre-existing data; obtaining real-time data related to the actual environment in which the device is or will be operating; generating a second virtual model of the actual environment based on the real-time data; merging the first virtual model and the second virtual model to form an integrated simulation model of the actual environment in which the device is or will be operating; presenting a simulation including a representation of the device operable in the integrated simulation model of the actual environment to permit control of operation of the representation of the device in the integrated simulation model of the actual environment; allowing control of operation of the representation of the device in the integrated simulation model of the actual environment; creating a control script from operation of the simulation of the device in the integrated simulation model of the actual environment; and transferring the control script to the device operating in the actual environment to control operation of the device in the actual environment based on the control script.
11. The method of claim 10 , further comprising creating the control script a predetermined time period before the device is required to perform a function in the actual environment using the control script.
12. The method of claim 11 , further comprising allowing adjustment of the predetermined time period by permitting speeding up and slowing down of the simulation of the device in the integrated simulation model of the actual environment.
13. The method of claim 10 , further comprising: continuously creating the control script while controlling operation of the simulation of the device in the integrated simulation model of the actual environment; and streaming the control script to the device in the actual environment.
14. The method of claim 10 , further comprising allowing control of operation of the simulation of the device in the integrated simulation model of the actual environment a predetermined time period before operation of the device in the actual environment corresponding to the operation of the device in the simulation.
15. The method of claim 10 , further comprising providing a re-do feature to re-do a portion of the simulation of the device operating in the integrated simulation model of the actual environment.
16. The method of claim 10 , further comprising providing a feature to control a speed of operation of the simulation of the device in the integrated simulation model of the actual environment.
17. The method of claim 10 , further comprising compensating for any operating limitations of the device in the actual environment.
18. The method of claim 10 , further comprising: permitting operation of the simulation of the device in the integrated simulation model of the actual environment with altered performance characteristics compared to the device in the actual environment; and compensating for any operating limitations of the device in the actual environment in the control script.
19. The method of claim 10 , wherein obtaining pre-existing data related to the actual environment comprises at least one of: obtaining terrain contours of the actual environment; obtaining images of the actual environment from a satellite; obtaining weather predictions; obtaining threat information; and obtaining target information.
20. The method of claim 10 , wherein obtaining real-time data related to the actual environment comprises at least one of: obtaining real-time images and information from a satellite; obtaining real-time images and information from the device; obtaining real-time images and information from other devices; and obtaining real-time images and information from fixed locations within the environment.
21. A method to remotely control operation of a vehicle, comprising: forming an integrated simulation model of an actual environment in which the vehicle is or will be operating, wherein the integrated simulation model is formed using pre-existing data and real-time data related to the actual environment, the pre-existing data comprising at least one of imagery of the actual environment, terrain contours, weather predictions, descriptions and locations of any points of interest, and locations and descriptions of any threats or hazards to the device in the actual environment, and wherein the pre-existing data is obtained from at least one of a satellite, aerial reconnaissance, ground reconnaissance and a sensor; presenting a simulation including a representation of the vehicle operable in the integrated simulation model of the actual environment; allowing control of operation of the representation of the vehicle in the integrated simulation model of the actual environment; controlling operation of the vehicle in the actual environment using the simulation of the vehicle in the integrated simulation model of the actual environment; and allowing the simulation of the vehicle in the integrated simulation model of the actual environment to be run a predetermined time period before the vehicle in the actual environment to substantially avoid latency and to permit correction of any mistakes.
22. The method of claim 21 , further comprising: allowing the simulation to be slowed-down to provide additional time for maneuvering and more precise operation of the representation of the vehicle within the simulation; and allowing the simulation to be sped-up to increase the predetermined time period.
23. The method of claim 21 , further comprising: permitting operation of the representation of the vehicle in the integrated simulation model of the actual environment with altered performance characteristics compared to the vehicle in the actual environment; and compensating for any operating limitations of the vehicle in the actual environment.
24. The method of claim 21 , further comprising allowing selection of a viewpoint from which the simulation of the vehicle in the integrated simulation model of the actual environment may be viewed and controlled.
25. The method of claim 24 , wherein allowing selection of the viewpoint comprises at least one of: providing a viewpoint from within the representation of the vehicle in the simulation; providing a viewpoint outside of the representation of the vehicle in the simulation; and providing a viewpoint including instrumentation associated with the vehicle.
26. The method of claim 21 , wherein controlling operation of the vehicle comprises controlling operation of one of an aerospace vehicle, a terrestrial vehicle, a planetary rover, and a watercraft.
27. A system to control a device, comprising: a processor to form an integrated simulation model of an actual environment in which the device is or will be operating, wherein the integrated simulation model is formed using pre-existing data and real-time data related to the actual environment, the pre-existing data comprising at least one of imagery of the actual environment, terrain contours, weather predictions, descriptions and locations of any points of interest, and locations and descriptions of any threats or hazards to the device in the actual environment, and wherein the pre-existing data is obtained from at least one of a satellite, aerial reconnaissance, ground reconnaissance and a sensor; a display to present a simulation including a representation of the device operable in the integrated simulation model of the actual environment; a control interface to allow control of operation of the simulation of the device in the integrated simulation model of the actual environment; and a transceiver to transmit a control script to the device in the actual environment to control operation of the device in the actual environment using the simulation of the representation of the device in the integrated simulation model.
28. The system of claim 27 , further comprising an image processor to generate an image of the actual environment using the pre-existing data and real-time data related to the actual environment.
29. The system of claim 28 , wherein the image processor comprises: a geometric transformation module to transform data related to the actual environment received from at least one of the device being controlled, the satellite, a fixed location within the actual environment and other devices for use in forming the integrated simulation model of the actual environment; an overlay control module to properly align different virtual models of the actual environment generated from the pre-existing data and real-time data related to the actual environment to form the integrated simulation model of the actual environment; a symbology module to assign symbols to represent different landmarks or features in the actual environment in the integrated simulation model; and a tracking control module to track any movement of the device in the actual environment.
30. The system of claim 28 , further comprising an environmental simulation module associated with the image processor to form the integrated simulation model of the actual environment for presentation on the display.
31. The system of claim 30 , wherein the environmental simulation module comprises: a geography database to store any pre-existing geographical data related to the actual environment; a threat and target database to store any pre-existing data related to any potential threats to the device within the actual environment and to store any pre-existing data related to any target within the actual environment; and a imagery database to store any pre-existing data of any images related to the actual environment.
32. The system of claim 27 , wherein the control interface comprises: a control to control movement of the device; a control to control a speed of the simulation; and a control to control a predetermined time delay between the simulation and operation of the device in the actual environment.
33. The system of claim 27 , further comprising a device simulation module to control operational characteristics of the representation of the device in the simulation and to compensate for any operating limitations of the device in the actual environment.
34. The system of claim 27 , further comprising a device control and systems processor to generate the control script based on the simulation.
35. The system of claim 27 , further comprising a buffer to buffer the control script before being transmitted to the device in the actual environment.
36. A computer program product to control a device, the computer program product comprising: a computer usable medium having computer usable program code embodied therewith, the computer usable medium comprising: computer usable program code configured to form an integrated simulation model of an actual environment in which the device is or will be operating, wherein the integrated simulation model is formed using pre-existing data and real-time data related to the actual environment, the pre-existing data comprising at least one of imagery of the actual environment, terrain contours, weather predictions, descriptions and locations of any points of interest, and locations and descriptions of any threats or hazards to the device in the actual environment, and wherein the pre-existing data is obtained from at least one of a satellite, aerial reconnaissance, ground reconnaissance and a sensor; computer usable program code configured to present a simulation including a representation of the device operable in the integrated simulation model of the actual environment; computer usable program code configured to allow control of operation of the simulation of the representation of the device in the integrated simulation model of the actual environment; and computer usable program code configured to control operation of the device in the actual environment using the simulation of the representation of the device in the integrated simulation model of the actual environment.
37. The computer program product of claim 36 , further comprising computer usable program code configured to allow control of operation of the simulation of the device to perform a simulated function in the integrated simulation model of the actual environment a predetermined time period before the device in the actual environment performs an actual function corresponding to the simulated function previously performed in the integrated simulation model.
38. The computer program product of claim 37 , further comprising computer usable program code configured to allow a selected portion of the simulation to be re-done in response to the predetermined time period being sufficiently long to permit the selected portion to be re-done and transferred to the device in the actual environment before performance of any functions corresponding to the selected portion of the simulation to be re-done.
39. The computer program product of claim 37 , further comprising: computer usable program code configured to allow the simulation to be slowed-down to provide additional time for adjustments and control of the representation of the device within the simulation and additional time for decision making; and computer usable program code configured to allow the simulation to be speed-up to increase the predetermined time period.
40. The computer program product of claim 37 , further comprising: computer usable program code configured to permit operation of the representation of the device in the integrated simulation model of the actual environment with altered performance characteristics compared to the device in the actual environment; and computer usable program code configured to compensate for any operating limitations of the device in the actual environment.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
September 14, 2007
September 13, 2011
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.