Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A computer-implemented method to provide an extended vision within an environment having a plurality of items, the extended vision based on a field of view of a person having a first computing device coupled with a first visual sensor within the environment, the environment associated with at least a second computing device coupled with a plurality of second visual sensors disposed within the environment, at least one of the first computing device and the second computing device coupled with a display device and including a user interface (UI) for traversing the extended vision, the method comprising: analyzing first image information acquired using the first visual sensor to determine the field of view of the person, the first image information including one or more first items of the plurality of items; analyzing second image information acquired using the plurality of second visual sensors, the second image information including one or more second items of the plurality of items; determining that an arrangement of the plurality of second visual sensors results in at least a first overlap area representing a first portion of the environment acquired in the image information of two or more second visual sensors of the plurality of second visual sensors; analyzing, for each of the two or more second visual sensors, corresponding image information to identify a respective count of the plurality of items included therein; responsive to determining that none of the identified counts of the plurality of items meet a predetermined threshold value: storing, when the field of view of the person overlaps at least part of the first overlap area, a portion of the first image information corresponding to the overlapping part in the memory of the second computing device; associating, based on the determined field of view, the first image information with the second image information to produce a combined image information; and displaying, via the display device, at least a portion of the combined image information to thereby provide the extended vision, wherein the portion of the combined image information is determined based on input received through the UI.
A computer system creates an "extended vision" view of an environment by combining images from multiple cameras. A person wears or carries a device (like a phone or smart glasses) with a camera that captures their view. The environment also has fixed cameras. The system analyzes the person's camera feed to determine their field of view and identifies items within that view. It also analyzes the fixed cameras' feeds. If multiple fixed cameras have overlapping views, the system checks how many items are visible in each camera's overlapping area. If none of the camera views in that overlap contain enough "items", the system will store the user's overlapping view in memory. The person's view is then combined with the fixed camera views to generate the extended vision, which is displayed to the user. The user can navigate the combined view through a user interface.
2. The method of claim 1 , wherein associating the first image information with the second image information includes stitching the first image information and the second image information.
The method to provide an extended vision within an environment having a plurality of items, where the extended vision is based on a field of view of a person having a first computing device coupled with a first visual sensor within the environment, the environment associated with at least a second computing device coupled with a plurality of second visual sensors disposed within the environment, at least one of the first computing device and the second computing device coupled with a display device and including a user interface (UI) for traversing the extended vision, as described in the previous extended vision description, combines the person's camera feed with the fixed camera feeds by stitching them together to create a seamless, wider view.
3. The method of claim 1 , wherein the combined image information is navigable via the display device.
The method to provide an extended vision within an environment having a plurality of items, where the extended vision is based on a field of view of a person having a first computing device coupled with a first visual sensor within the environment, the environment associated with at least a second computing device coupled with a plurality of second visual sensors disposed within the environment, at least one of the first computing device and the second computing device coupled with a display device and including a user interface (UI) for traversing the extended vision, as described in the previous extended vision description, allows the user to navigate the combined image on the display device, enabling them to explore different parts of the environment.
4. The method of claim 1 , wherein the first computing device is a body-worn computing device and the first visual sensor is a forward-looking camera integrated with the first computing device.
The method to provide an extended vision within an environment having a plurality of items, where the extended vision is based on a field of view of a person having a first computing device coupled with a first visual sensor within the environment, the environment associated with at least a second computing device coupled with a plurality of second visual sensors disposed within the environment, at least one of the first computing device and the second computing device coupled with a display device and including a user interface (UI) for traversing the extended vision, as described in the previous extended vision description, uses a wearable device like smart glasses for the person's camera, and the camera faces forward to capture their view.
5. The method of claim 1 , wherein the first computing device is a handheld computing device and the person manipulates an orientation of the handheld computing device to acquire the first image information.
The method to provide an extended vision within an environment having a plurality of items, where the extended vision is based on a field of view of a person having a first computing device coupled with a first visual sensor within the environment, the environment associated with at least a second computing device coupled with a plurality of second visual sensors disposed within the environment, at least one of the first computing device and the second computing device coupled with a display device and including a user interface (UI) for traversing the extended vision, as described in the previous extended vision description, uses a handheld device like a phone or tablet as the person's camera. The person moves the device to capture the desired view, which is then integrated into the extended vision.
6. The method of claim 1 , further comprising: assigning, based on a largest number of the identified numbers of the plurality of items, the corresponding visual sensor of the two or more second visual sensors as a primary view for the first overlap area; and assigning one other visual sensor of the two or more second visual sensors as an auxiliary view for the first overlap area.
The method to provide an extended vision within an environment having a plurality of items, where the extended vision is based on a field of view of a person having a first computing device coupled with a first visual sensor within the environment, the environment associated with at least a second computing device coupled with a plurality of second visual sensors disposed within the environment, at least one of the first computing device and the second computing device coupled with a display device and including a user interface (UI) for traversing the extended vision, as described in the previous extended vision description, designates one of the fixed cameras in an overlapping area as the "primary view" based on which camera sees the most "items". Another camera in the overlapping area is assigned as an "auxiliary view," likely to provide redundancy or alternative perspectives.
7. The method of claim 1 , further comprising: performing, in response to the input provided through the UI, at least one of reorienting and displacing a second visual sensor of the plurality of second visual sensors.
The method to provide an extended vision within an environment having a plurality of items, where the extended vision is based on a field of view of a person having a first computing device coupled with a first visual sensor within the environment, the environment associated with at least a second computing device coupled with a plurality of second visual sensors disposed within the environment, at least one of the first computing device and the second computing device coupled with a display device and including a user interface (UI) for traversing the extended vision, as described in the previous extended vision description, allows the user, through the user interface, to remotely adjust the position or angle of the fixed cameras, changing their coverage of the environment.
8. The method of claim 1 , further comprising: determining that an arrangement of the plurality of second visual sensors results in at least a first deadspace area representing a first portion of the environment that is not acquired in the image information of any of the plurality of second visual sensors; and upon determining that the field of view of the person overlaps at least part of the first deadspace area, storing a portion of the first image information corresponding to the overlapping part in a memory of the second computing device.
The method to provide an extended vision within an environment having a plurality of items, where the extended vision is based on a field of view of a person having a first computing device coupled with a first visual sensor within the environment, the environment associated with at least a second computing device coupled with a plurality of second visual sensors disposed within the environment, at least one of the first computing device and the second computing device coupled with a display device and including a user interface (UI) for traversing the extended vision, as described in the previous extended vision description, identifies "deadspace" areas not covered by any of the fixed cameras. If the person's view includes any of this deadspace, that portion of their camera feed is stored for inclusion in the extended vision.
9. The method of claim 8 , further comprising: transmitting, to the first computing device, a prompt suggesting that the person should acquire image information corresponding to the first deadspace area, wherein determining that the field of view of the person overlaps at least part of the first deadspace area is performed after transmitting the prompt.
The method to provide an extended vision within an environment having a plurality of items, where the extended vision is based on a field of view of a person having a first computing device coupled with a first visual sensor within the environment, the environment associated with at least a second computing device coupled with a plurality of second visual sensors disposed within the environment, at least one of the first computing device and the second computing device coupled with a display device and including a user interface (UI) for traversing the extended vision, determining that an arrangement of the plurality of second visual sensors results in at least a first deadspace area representing a first portion of the environment that is not acquired in the image information of any of the plurality of second visual sensors, and upon determining that the field of view of the person overlaps at least part of the first deadspace area, storing a portion of the first image information corresponding to the overlapping part in a memory of the second computing device, as described in the previous deadspace description, also sends a message to the person's device, prompting them to capture an image of the deadspace area. The system checks for the person's view in the deadspace *after* sending this prompt.
10. The method of claim 9 , wherein the prompt includes one of: a request that the person move to the first deadspace area to acquire image information corresponding to the first deadspace area, and a suggestion for one or more items of the plurality of items that are determined to be located within or adjacent to the first deadspace area.
The method to provide an extended vision within an environment having a plurality of items, where the extended vision is based on a field of view of a person having a first computing device coupled with a first visual sensor within the environment, the environment associated with at least a second computing device coupled with a plurality of second visual sensors disposed within the environment, at least one of the first computing device and the second computing device coupled with a display device and including a user interface (UI) for traversing the extended vision, determining that an arrangement of the plurality of second visual sensors results in at least a first deadspace area representing a first portion of the environment that is not acquired in the image information of any of the plurality of second visual sensors, upon determining that the field of view of the person overlaps at least part of the first deadspace area, storing a portion of the first image information corresponding to the overlapping part in a memory of the second computing device, and transmitting, to the first computing device, a prompt suggesting that the person should acquire image information corresponding to the first deadspace area, wherein determining that the field of view of the person overlaps at least part of the first deadspace area is performed after transmitting the prompt, as described in the previous deadspace prompting description, provides specific instructions in the prompt, such as asking the person to move into the deadspace or suggesting specific items in or near the deadspace that they should focus on.
11. A computer program product to provide an extended vision within an environment having a plurality of items, the extended vision based on a field of view of a person having a first computing device coupled with a first visual sensor within the environment, the environment associated with at least a second computing device coupled with a plurality of second visual sensors disposed within the environment, at least one of the first computing device and the second computing device coupled with a display device and including a user interface (UI) for traversing the extended vision, the computer program product comprising: a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code executable by one or more computer processors to perform an operation that includes: analyzing first image information acquired using the first visual sensor to determine the field of view of the person, the first image information including one or more first items of the plurality of items; analyzing second image information acquired using the plurality of second visual sensors, the second image information including one or more second items of the plurality of items; determining that the arrangement of the plurality of second visual sensors results in at least a first overlap area representing a second portion of the environment acquired in the image information of two or more second visual sensors of the plurality of second visual sensors; analyzing, for each of the two or more second visual sensors, corresponding image information to identify a respective number of the plurality of items included therein; responsive to determining that none of the identified counts of the plurality of items meet a predetermined threshold value: storing, when the field of view of the person overlaps at least part of the first overlap area, a portion of the first image information corresponding to the overlapping part in the memory of the second computing device; associating, based on the determined field of view, the first image information with the second image information to produce a combined image information; and displaying, via the display device, at least a portion of the combined image information to thereby provide the extended vision, wherein the portion of the combined image information is determined based on input received through the UI.
A computer program stored on a computer-readable medium creates an "extended vision" view of an environment by combining images from multiple cameras. A person wears or carries a device (like a phone or smart glasses) with a camera that captures their view. The environment also has fixed cameras. The program analyzes the person's camera feed to determine their field of view and identifies items within that view. It also analyzes the fixed cameras' feeds. If multiple fixed cameras have overlapping views, the program checks how many items are visible in each camera's overlapping area. If none of the camera views in that overlap contain enough "items", the program will store the user's overlapping view in memory. The person's view is then combined with the fixed camera views to generate the extended vision, which is displayed to the user. The user can navigate the combined view through a user interface.
12. The computer program product of claim 11 , wherein associating the first image information with the second image information includes stitching the first image information and the second image information.
This invention relates to image processing, specifically techniques for combining multiple images to create a cohesive visual representation. The problem addressed is the difficulty in accurately merging images from different sources or perspectives, particularly when there are misalignments, overlaps, or gaps between them. The solution involves a computer program product that processes image data to align and integrate multiple images seamlessly. The system captures or receives a first set of image information and a second set of image information, which may originate from different cameras, sensors, or time points. The key innovation is the method of associating these images by stitching them together. Stitching involves analyzing overlapping regions, correcting distortions, and blending edges to produce a unified image. This process may include feature matching, geometric transformation, and color correction to ensure visual continuity. The invention also includes preprocessing steps to enhance image quality before stitching, such as noise reduction or contrast adjustment. Post-processing may further refine the stitched image, such as by smoothing transitions or removing artifacts. The result is a high-quality composite image that retains the details of the original inputs while eliminating seams or inconsistencies. This technology is applicable in fields like panoramic photography, medical imaging, surveillance, and virtual reality, where accurate image fusion is critical. The automated stitching process reduces manual effort and improves consistency compared to traditional methods.
13. The computer program product of claim 11 , wherein the combined image information is navigable via the display device.
A system and method for processing and displaying combined image information from multiple sources, such as medical imaging devices, to enhance visualization and analysis. The invention addresses the challenge of integrating disparate image data into a cohesive, interactive display that allows users to navigate through the combined information. The system captures image data from different imaging modalities, such as X-ray, MRI, or ultrasound, and processes the data to align and overlay the images. The processed images are then displayed on a display device, where the combined image information is navigable, enabling users to explore different views, zoom in or out, and adjust the display parameters. The navigation functionality allows for real-time interaction, improving diagnostic accuracy and efficiency. The system may also include features for annotating, measuring, and comparing the combined images, providing a comprehensive tool for medical professionals. The invention ensures seamless integration of multi-source image data into a single, interactive interface, enhancing decision-making in medical imaging applications.
14. The computer program product of claim 11 , the operation further comprising: assigning, based on a largest number of the identified numbers of the plurality of items, the corresponding visual sensor of the two or more second visual sensors as a primary view for the first overlap area; and assigning one other visual sensor of the two or more second visual sensors as an auxiliary view for the first overlap area.
This invention relates to a computer program product for managing visual sensor coverage in an overlapping surveillance area. The problem addressed is the need to efficiently assign primary and auxiliary views in regions where multiple visual sensors overlap, ensuring optimal coverage and minimizing redundancy. The system identifies a first overlap area where two or more visual sensors provide coverage. For each item within this overlap area, the system determines the number of visual sensors that can detect the item. The visual sensor that detects the largest number of items is assigned as the primary view for the overlap area, ensuring the most comprehensive coverage. A secondary visual sensor is then assigned as an auxiliary view to provide additional perspective or redundancy. The invention also includes a method for processing visual data from the primary and auxiliary views, such as stitching or comparing the data to enhance surveillance accuracy. The system may also adjust assignments dynamically if sensor performance or environmental conditions change. This approach improves monitoring efficiency by optimizing sensor usage in overlapping regions.
15. The computer program product of claim 11 , the operation further comprising: performing, in response to the input provided through the UI, at least one of reorienting and displacing a second visual sensor of the plurality of second visual sensors.
This invention relates to a computer program product for controlling visual sensors in a system with multiple visual sensors. The system includes a first visual sensor and a plurality of second visual sensors, where the first visual sensor captures a first image and the second visual sensors capture second images. The program product generates a user interface (UI) that displays the first image and the second images, allowing a user to interact with the UI to provide input. In response to this input, the program product performs at least one of reorienting or displacing one or more of the second visual sensors. This adjustment ensures that the second visual sensors maintain proper alignment or positioning relative to the first visual sensor, improving the accuracy and consistency of the captured images. The system may be used in applications requiring precise visual monitoring, such as surveillance, medical imaging, or industrial inspection, where maintaining optimal sensor alignment is critical. The program product dynamically adjusts the second visual sensors based on user input, enhancing the system's adaptability and usability.
16. The computer program product of claim 11 , the operation further comprising: determining that an arrangement of the plurality of second visual sensors results in at least a first deadspace area representing a first portion of the environment that is not acquired in the image information of any of the plurality of second visual sensors; upon determining that the field of view of the person overlaps at least part of the first deadspace area, storing a portion of the first image information corresponding to the overlapping part in a memory of the second computing device.
This invention relates to a system for enhancing environmental monitoring using multiple visual sensors, particularly addressing gaps in coverage (deadspace areas) where certain portions of an environment are not captured by any sensor. The system includes a plurality of second visual sensors configured to capture image information of the environment and a second computing device that processes this data. The system detects when the arrangement of these sensors creates deadspace areas, meaning parts of the environment remain unmonitored. If a person's field of view overlaps with any deadspace area, the system stores relevant image information from the person's perspective in the memory of the second computing device. This ensures that critical areas not covered by the fixed sensors are still documented, leveraging human observation to fill coverage gaps. The system may also include a first computing device that processes image information from a plurality of first visual sensors, which may be used for initial environmental monitoring or calibration. The invention improves situational awareness by dynamically integrating human observation into automated surveillance systems, particularly useful in security, industrial, or safety applications where continuous monitoring is essential.
17. A system to provide an extended vision within an environment having a plurality of items, the extended vision based on a determined field of view of a person and traversable using a user interface (UI), the system comprising: a plurality of computer processors, at least one processor included in a first computing device associated with the person, and at least one other processor included in at least one second computing device associated with the environment and communicatively coupled with the first computing device; a first visual sensor communicatively coupled with the first computing device, the first visual sensor disposed within the environment and configured to acquire first image information; a plurality of second visual sensors that are communicatively coupled with the at least one second computing device, the plurality of second visual sensors disposed within the environment and configured to acquire second image information; and a display device communicatively coupled with at least one of the first computing device and the second computing device; wherein the plurality of computer processors are configured to perform an operation that includes: analyzing the first image information to determine the field of view of the person, the first image information including one or more identified first items of the plurality of items; analyzing the second image information, the second image information including one or more identified second items of the plurality of items; determining that the arrangement of the plurality of second visual sensors results in at least a first overlap area representing a second portion of the environment acquired in the image information of two or more second visual sensors of the plurality of second visual sensors; analyzing, for each of the two or more second visual sensors, corresponding image information to identify a respective number of the plurality of items included therein; responsive to determining that none of the identified counts of the plurality of items meet a predetermined threshold value: storing, when the field of view of the person overlaps at least part of the first overlap area, a portion of the first image information corresponding to the overlapping part in the memory of the second computing device; associating, based on the determined field of view, the first image information with the second image information to produce a combined image information; and displaying, via the display device, at least a portion of the combined image information to thereby provide the extended vision, wherein the portion of the combined image information is determined based on input received through the UI.
A system uses multiple processors and cameras to create an "extended vision" view. One processor and camera are associated with a person, such as on their phone or glasses. Other processors and fixed cameras are placed within the environment. The system combines images from the person's camera and the fixed cameras. It analyzes the person's camera feed to determine their field of view and identifies items. It also analyzes the fixed cameras' feeds. If multiple fixed cameras have overlapping views, the system checks how many items are visible in each camera's overlapping area. If none of the camera views in that overlap contain enough "items", the system will store the user's overlapping view in memory. The person's view is then combined with the fixed camera views to generate the extended vision, which is displayed to the user. The user can navigate the combined view through a user interface.
18. The system of claim 17 , wherein the first computing device is a body-worn computing device and the first visual sensor is a forward-looking camera integrated with the first computing device.
This invention relates to wearable computing systems and the problem of providing contextually relevant information to a user based on their environment. The system includes a first computing device, which is a body-worn device. This device is equipped with a first visual sensor, specifically a forward-looking camera integrated directly into the body-worn computing device. This camera captures visual data of the user's surroundings. The system also comprises a second computing device, which is a mobile computing device. A communication link is established between the first and second computing devices, enabling them to exchange data. The first computing device transmits the visual data captured by its forward-looking camera to the second computing device. The second computing device then processes this visual data. Based on the processed visual data, the second computing device generates and presents information to the user. This information is designed to be relevant to the user's current environment as perceived by the forward-looking camera.
19. The system of claim 17 , wherein the first computing device is a handheld computing device and the person manipulates an orientation of the handheld computing device to acquire the first image information.
A system for capturing and processing image data using a handheld computing device involves a user manipulating the device's orientation to acquire image information. The handheld device includes a camera and sensors to detect its position and orientation. The system processes the acquired image data to determine spatial relationships or environmental features, such as object locations or scene geometry. The handheld device may communicate with a second computing device to share or analyze the captured data. The system may also include a display to provide real-time feedback or augmented visualizations based on the processed image information. The technology addresses challenges in mobile imaging, such as ensuring accurate spatial data capture despite device movement or user handling variations. Applications include augmented reality, navigation, and environmental mapping, where precise image acquisition and orientation tracking are critical for reliable results. The system enhances usability by allowing intuitive, hands-on interaction with the device to capture and process spatial data efficiently.
20. The system of claim 17 , wherein the combined image information is navigable via the display device.
A system for processing and displaying combined image information addresses the challenge of integrating and navigating multiple image sources in a unified interface. The system captures image data from at least two different sources, such as cameras or sensors, and merges this data into a single composite image. The combined image information is then displayed on a display device, allowing users to interact with and navigate through the integrated visual data. This navigation may include zooming, panning, or selecting specific regions of the combined image for detailed viewing. The system ensures that the combined image retains sufficient resolution and clarity to enable effective analysis or decision-making based on the merged data. By providing a seamless and interactive display of multi-source imagery, the system enhances situational awareness and simplifies the interpretation of complex visual information. The technology is particularly useful in applications such as surveillance, medical imaging, or remote sensing, where integrating multiple image sources into a single view improves efficiency and accuracy.
21. The system of claim 17 , the operation further comprising: determining that an arrangement of the plurality of second visual sensors results in at least a first deadspace area representing a first portion of the environment that is not acquired in the image information of any of the plurality of second visual sensors; upon determining that the field of view of the person overlaps at least part of the first deadspace area, storing a portion of the first image information corresponding to the overlapping part in a memory of the second computing device.
This invention relates to a surveillance system using multiple visual sensors to monitor an environment, addressing the challenge of blind spots or deadspace areas where certain portions of the environment are not captured by any sensor. The system includes a plurality of second visual sensors arranged to capture image information of the environment, where their arrangement may create deadspace areas that are not covered by any sensor. To mitigate this, the system determines when a person's field of view overlaps with a deadspace area. When this occurs, the system stores a portion of the image information corresponding to the overlapping part in a memory of a second computing device. This ensures that critical areas not covered by the sensors are still monitored when a person is present, enhancing overall surveillance coverage. The system may also include a first visual sensor for capturing a first image of the environment, and the second visual sensors may be arranged to capture additional image information of the environment. The system dynamically adjusts to ensure no critical areas are left unmonitored, improving situational awareness in surveillance applications.
Unknown
December 12, 2017
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.