A system determines an otherwise unknown position and orientation of a camera in a working environment, relative to an associated coordinate system, based on visually identifiable unique objects in images taken by the camera. The system utilizes a database that includes or may be updated to include position coordinates for unique objects of interest. The system identifies a plurality of objects within images taken by the camera at a given location, and enters the database either to determine position coordinates for the respective identified objects or to add position coordinates to the data base for the respective identified objects, or both. The system may also update the database to include newly identified unique objects and objects that are altered between images, to determine position and orientation of the camera in a changing environment. Sensors may be included to add additional information relative to the position and orientation of the camera.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A positioning system including a database that associates, with visually identifiable unique objects in a working environment, position coordinates within an associated coordinate system; one or more cameras of known internal characteristics and unknown positions, movements, or both within the environment, the one or more cameras taking one or more images of the environment from respective positions; a processing subsystem for identifying unique objects within one or more of the images taken by a given camera; determining if at least three unique objects are identified within the one or more images taken by the given camera; based on determining that the at least the three unique objects are identified within the one or more images taken by the given camera; (1) entering the database to extract position coordinates associated with the at least three unique objects that match entries in the database, and (2) determining position and orientation of the given camera in the environment based on the retrieved position coordinates for the at least three unique objects; based on determining that the at least three unique objects are not identified within the one or more images taken by the given camera, processing one or more additional images to identify the at least three unique objects; and updating the database to include one or both of information relating to newly identified unique objects in the one or more images taken by the respective cameras, the information including the position coordinates for the newly identified objects as determined from the images taken by the cameras and more precise position coordinates for previously identified objects.
2. The positioning system of claim 1 wherein the database further includes orientation coordinates for the identifiable unique objects that are composites of other unique objects, and the orientation coordinates are updated when the associated position coordinates are updated.
3. The positioning system of claim 2 wherein the database is empty when the one or more cameras take a first image and the processing subsystem populates the database to include the position coordinates, the orientation coordinates, or both, relating to identified unique objects in the first image and next images taken by the one or more cameras.
4. The positioning system of claim 3 wherein the database further associates with the identified unique objects attributes of the identified objects that are determined from the images.
5. The positioning system of claim 1 wherein the position coordinates of the identified unique objects are calculated by 3-dimensional triangulation based on the images taken by the one or more cameras.
6. The positioning system of claim 1 wherein the processing subsystem processes altered unique objects as the newly identified unique objects.
7. The positioning system of claim 1 wherein the system utilizes information from maps, floor plans and combinations thereof and associates with the database entry of a unique object a description of where the unique object is visible from within the map, floor plans and combinations thereof.
8. The positioning system of claim 7 , wherein the description of where the unique object is on a wall consists of a mathematical description of a plane and a side of the plane from which the object is visible.
9. The positioning system of claim 8 , wherein the camera moves in directions of traffic flow in the environment to take the images.
10. The positioning system of claim 6 wherein the system includes in the database as observations known reference information about the environment.
11. The positioning system of claim 10 wherein the reference information is one or more of elevations of respective floors, elevations of defined areas within the environment, pointing directions of views from windows, pointing directions based on visibility of features located on walls, relative locations of walls, windows and other features of a floor plan.
12. The positioning system of claim 1 wherein a position and orientation of the camera for a first image are set to arbitrary values and positions and orientations of the unique objects are determined relative to the arbitrary values.
13. The positioning system of claim 12 further including an uncalibrated inertial measurement unit (IMU), and the one or more processors determine a gravity vector based on readings of the IMU and provide an orientation reference to the working environment.
14. The positioning system of claim 1 wherein the one or more processors identify within the images unique strategically placed objects that are associated in the database with known observations, the one or more processors use the known observations in calculations to determine position and orientation of the camera.
15. The positioning system of claim 14 wherein the unique strategically placed objects correspond to one or more of different elevations and different pointing directions.
16. The positioning system of claim 1 wherein the one or more processors determine the unknown movement of the camera to a next position and orientation based on the images taken before and after the movement.
17. The positioning system of claim 16 further including an inertial measurement subsystem associated with a given camera, the one or more processors calibrate and update the inertial measurement subsystem based on the determination of position and orientation of the given camera using the images, and the one or more processors use information from the inertial measurement system to determine camera position and orientation in between times the one or more processors process the images to determine camera position and orientation.
18. The positioning system of claim 2 wherein a given camera is associated with a Global Navigation Satellite System (GNSS) receiver and GNSS position and orientation information is used by the processing subsystem to determine position and orientation information for identified unique objects, update position and orientation information for the identified unique objects, or both.
19. The positioning system of claim 18 wherein the given camera is further associated with an inertial measurement system, and the processing subsystem initializes and updates the inertial measurement subsystem using the position and orientation of the given camera as determined by the system and uses information from the inertial measurement subsystem to determine position and orientation information for the given camera based on sensed movement.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
November 1, 2012
August 16, 2016
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.