Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A method for performing feature based high resolution motion estimation from a plurality of low resolution images, comprising: performing feature detection with respect to a first sequence of low resolution images captured by a first imager in an imager array using a processor configured by software to identify initial locations for a plurality of detected features in the first sequence of low resolution images, where the first sequence of low resolution images is part of a set of sequences of low resolution images captured from different perspectives by different imagers in the imager array, a first plurality of images includes one image from each sequence of low resolution images taken by different imagers from different perspectives at a first point in time, and a second plurality of images includes one images from each sequence of low resolution images taken by different imagers from different perspectives at a second point in time; synthesizing a first set of high resolution image portions from the set of sequences of low resolution images captured from different perspectives using the processor configured by software to perform a super-resolution process using the first plurality of images and parallax information, where the synthesized high resolution image portions contain the identified plurality of detected features from the sequence of low resolution images; synthesizing a second set of high resolution image portions from the set of sequences of low resolution images captured from different perspectives using the processor configured by software to perform a super-resolution process using the second plurality of images and parallax information, where the synthesized high resolution image portions contain the identified plurality of detected features from the sequence of low resolution images; performing feature detection within the first and second sets of high resolution image portions to identify locations for said plurality of detected features to a higher precision than the initial locations identified in the low resolution images using the processor configured by software; and estimating camera motion using the high precision locations for said plurality of detected features using the processor configured by software.
A method for estimating camera motion from low-resolution images using feature detection and super-resolution. The method uses an array camera with multiple imagers capturing a set of low-resolution image sequences from slightly different viewpoints. First, features are detected in a sequence of low-resolution images. Next, the method synthesizes high-resolution portions of images using super-resolution techniques and parallax information from multiple low-resolution images captured at the same time instant by different cameras. This is done for two different time instants. Feature detection is then performed on these high-resolution portions to find the precise location of previously detected features. Finally, the camera motion is estimated based on the movement of these precisely located features between the two time instances.
2. The method of claim 1 , wherein the detected features are selected from the group consisting of: edges, corners, and blobs.
The feature-based motion estimation method described above uses edges, corners, or blobs as the detected features used to estimate motion. These features are detected in low-resolution images, enhanced using super-resolution, and tracked to estimate camera movement.
3. The method of claim 1 , wherein performing feature detection with respect to a sequence of low resolution images further comprises: detecting the location of features in a first frame from the low resolution sequence of images; and detecting the location of features in a second frame from the low resolution sequence of images.
In the feature-based motion estimation method described above, the feature detection process involves identifying features in at least two frames from a low-resolution image sequence. First, locations of features are detected in the first frame. Second, locations of features are detected in a subsequent frame.
4. The method of claim 3 , wherein detecting the location of features in a second frame from the sequence of low resolution images further comprises searching the second frame from the sequence of low resolution images to locate features detected in the first frame from the sequence of low resolution images.
The feature-based motion estimation method described above includes a feature detection process, where after detecting features in a first frame, the process searches for those same features in a second frame from the low-resolution image sequence to track feature displacement.
5. The method of claim 4 , wherein searching the second frame from the sequence of low resolution images to locate features detected in the first frame from the sequence of low resolution images further comprises: identifying an image patch surrounding the location of the given feature in the first frame in the sequence of low resolution images; and searching the second frame in the sequence of low resolution images for a corresponding image patch using a matching criterion.
The method for locating features in subsequent frames involves defining an image patch around a feature detected in the first frame, and then searching for a corresponding image patch in the second frame using a matching criterion. This allows the algorithm to track the movement of the feature between frames.
6. The method of claim 5 , wherein the matching criterion involves minimizing an error distance metric.
The method for locating features in subsequent frames uses an error distance metric as the matching criterion to find corresponding image patches. The algorithm searches for the image patch in the second frame that minimizes the difference between it and the image patch from the first frame.
7. The method of claim 3 , wherein performing feature detection within the high resolution image portions to identify high precision locations for said plurality of detected features further comprises searching the high resolution image regions containing the features from the second frame in the sequence of low resolution images for features from the first frame in the sequence of low resolution images using the high resolution image regions containing the features from the first frame in the low resolution sequence of images.
The method uses feature detection within high-resolution image portions, and includes searching high-resolution regions of a second frame for features that were initially detected in a first frame using corresponding high-resolution regions from the first frame. This allows for precise feature tracking in the enhanced images.
8. The method of claim 7 , wherein searching the high resolution image regions containing the features from the second frame in the sequence of low resolution images for features from the first frame in the sequence of low resolution images further comprises comparing high resolution image regions containing features from the second frame in the sequence of low resolution images to the high resolution image portions containing the features from the first frame in the sequence of low resolution images using a matching criterion.
The method for feature detection within high-resolution images compares high-resolution image regions containing features from the second frame to the high-resolution image portions containing the features from the first frame using a matching criterion. This comparison helps refine the location of the features.
9. The method of claim 8 , wherein the matching criterion involves minimizing an error distance metric.
The matching criterion used to compare high-resolution image regions containing features from the first and second frame involves minimizing an error distance metric. This ensures accurate feature matching for motion estimation.
10. The method of claim 1 , wherein the processor is part of an array camera that further comprises an imager array, the method further comprising capturing at least a plurality of the sequences of low resolution images in the set of sequences of low resolution images from different perspectives using the imager array.
The method for motion estimation is performed by a processor that is part of an array camera. The array camera itself captures multiple sequences of low-resolution images from different viewpoints using an imager array, enabling parallax-based super-resolution.
11. The method of claim 1 , wherein the high precision locations for said plurality of detected features estimate feature location at a subpixel precision relative to the size of the pixels of the frames in the sequence of low resolution images.
The method achieves subpixel precision in feature location relative to the original low-resolution images by using the high-resolution image portions. The high-precision feature locations estimated from super-resolution allows for more accurate motion estimation.
12. An array camera configured to perform feature based high resolution motion estimation from low resolution images captured using the array camera, comprising: an imager array comprising a plurality of imagers; a processor configured by software to control various operating parameters of the imager array; wherein the software further configures the processor to: capture a set of sequences of low resolution images captured from different perspectives using the imager array, where a first imager captures a first sequence of low resolution images from a first perspective, a second imager captures a second sequence of low resolution images from a second perspective, a first plurality of images includes one image from each sequence of low resolution images taken by different imagers from different perspectives at a first point in time, and a second plurality of images includes one images from each sequence of low resolution images taken by different imagers from different perspectives at a second point in time; perform feature detection with respect to one of the set of sequences of low resolution images to identify initial locations for a plurality of detected features in the sequence of low resolution images; synthesize a first set of high resolution image portions from the set of sequences of low resolution images captured from different perspectives using a super-resolution process using the first plurality of images and parallax information, where the high resolution image portions contain the identified plurality of detected features from the sequence of low resolution images; synthesize a second set of high resolution image portions from the set of sequences of low resolution images captured from different perspectives using a super-resolution process using the second plurality of images and parallax information, where the synthesized high resolution image portions contain the identified plurality of detected features from the sequence of low resolution images; perform feature detection within the first and second sets of high resolution image portions to identify locations for said plurality of detected features to a higher precision than the initial locations identified in the low resolution images; and estimate camera motion using the high precision locations for said plurality of detected features.
An array camera system performs feature-based high-resolution motion estimation from low-resolution images. It has an imager array and a processor. The processor captures sets of low-resolution image sequences from different perspectives using the array's imagers. The processor detects initial feature locations in one image sequence, synthesizes high-resolution image portions using super-resolution techniques and parallax data from multiple camera views at two different time instants, refines feature locations within the high-resolution portions to improve precision, and estimates camera motion based on the movement of these refined feature locations.
13. The array camera of claim 12 , where the detected features are selected from the group consisting of: edges, corners, and blobs.
The array camera uses edges, corners, or blobs as the detected features. The processor is configured to detect these features in low-resolution images and then track them in the synthesized high-resolution portions to estimate motion.
14. The array camera of claim 12 , wherein the processor is further configured to perform feature detection with respect to a sequence of low resolution images by: detecting the location of features in a first frame from the sequence of low resolution images; and detecting the location of features in a second frame from the sequence of low resolution images.
The array camera's processor detects feature locations in a first frame of the low-resolution image sequence and then detects the location of the same features in a subsequent frame from the low-resolution sequence. These detections, performed on multiple frames, allow the processor to analyze feature displacement.
15. The array camera of claim 14 , wherein the processor is further configured by software to detect the location of features in a second frame from the sequence of low resolution images by searching the second frame from the sequence of low resolution images to locate features detected in the first frame from the sequence of low resolution images.
The array camera's processor locates features in the second frame by searching the second frame for the features previously detected in the first frame. This establishes correspondence between features across frames to track motion.
16. The array camera of claim 15 , wherein the processor is further configured by software to search a second frame from the sequence of low resolution images to locate a given feature detected in the first frame from the sequence of low resolution images by: identifying an image patch surrounding the location of the given feature in the first frame in the sequence of low resolution images; and searching the second frame in the sequence of low resolution images for a corresponding image patch using a matching criterion.
To locate features in a second frame, the processor identifies an image patch around the feature in the first frame and then searches for a corresponding image patch in the second frame using a matching criterion.
17. The array camera of claim 16 , wherein the matching criterion involves minimizing an error distance metric.
The array camera's processor uses an error distance metric as the matching criterion when searching for corresponding image patches to track features between frames. The algorithm minimizes this metric to ensure accurate matching.
18. The array camera of claim 14 , wherein the processor is further configured by software to perform feature detection within the high resolution image portions to identify high precision locations for said plurality of detected features by searching the high resolution image regions containing the features from the second frame in the sequence of low resolution images for features from the first frame in the sequence of low resolution images using the high resolution image regions containing the features from the first frame in the low resolution sequence of images.
When finding precise feature locations in the high-resolution portions, the array camera searches the high-resolution image regions from a second frame for features identified in the first frame, using the high-resolution regions from the first frame as a reference. This refines the feature tracking process.
19. The array camera of claim 18 , wherein the processor is further configured by software to search the high resolution image regions containing the features from the second frame in the sequence of low resolution images for features from the first frame in the sequence of low resolution images by comparing high resolution image regions containing features from the second frame in the sequence of low resolution images to the high resolution image portions containing the features from the first frame in the sequence of low resolution images using a matching criterion.
The array camera processor compares the high-resolution regions containing features from the second frame to the high-resolution portions containing features from the first frame using a matching criterion. This comparison helps find corresponding features with higher precision.
20. The array camera of claim 19 , wherein the matching criterion involves minimizing an error distance metric.
The array camera's matching criterion in high-resolution feature detection minimizes an error distance metric when comparing high-resolution regions between frames. This enhances the accuracy of feature matching and motion estimation.
Unknown
November 7, 2017
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.