Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A computer-implemented method for image processing, the method comprising: obtaining first light information from a set of light-sensitive pixels for a scene, the set of light-sensitive pixels including a plurality of phase detection (PD) pixels and a plurality of non-PD pixels; obtaining location information associated with each of the plurality of PD pixels; generating a first PD pixel image from the first light information based on location information obtained from the plurality of PD pixels, wherein the first PD pixel image has a first resolution; generating a higher resolution image from the first light information obtained from the plurality of non-PD pixels, wherein the higher resolution image has a second resolution that is greater than the first resolution of the first PD pixel image; matching a first pixel of the first PD pixel image to the higher resolution image, wherein the matching is based on a set of correlations between the first pixel and non-PD pixels within a predetermined distance of the first pixel; and determining a disparity map for an image associated with the first light information, based on the matching.
This invention relates to image processing techniques for enhancing depth perception in captured images using phase detection (PD) pixels and non-PD pixels. The problem addressed is the limited resolution of traditional PD pixel-based depth sensing, which restricts accurate disparity mapping in high-resolution imaging systems. The solution involves a computer-implemented method that processes light information from a mixed pixel array containing both PD and non-PD pixels. The method first obtains light data from the scene, then generates a lower-resolution PD pixel image using the PD pixels' location information. Simultaneously, a higher-resolution image is created from the non-PD pixels. The method then matches pixels from the lower-resolution PD image to the higher-resolution image by analyzing correlations between PD pixels and nearby non-PD pixels within a predefined distance. This matching process enables accurate alignment of depth information with high-resolution image data. Finally, a disparity map is generated based on the matched pixels, improving depth accuracy without sacrificing image quality. The technique leverages the strengths of both pixel types to enhance depth sensing in imaging systems.
2. The computer-implemented method of claim 1 , wherein higher resolution image comprises an interpolated image generated by interpolating light information obtained from the plurality of non-PD pixels.
This invention relates to image processing techniques for enhancing image resolution in imaging systems, particularly those using non-photodiode (non-PD) pixels. The problem addressed is the limited resolution of images captured by sensors with non-PD pixels, which lack the traditional photodiode structure found in conventional image sensors. The solution involves generating a higher resolution image by interpolating light information obtained from the plurality of non-PD pixels. The interpolation process reconstructs missing pixel data to produce an image with improved resolution compared to the original sensor output. This method leverages the light information captured by the non-PD pixels to estimate values for intermediate pixels, effectively filling in gaps in the image data. The technique may involve spatial interpolation, where neighboring pixel values are used to predict missing data, or other interpolation algorithms tailored to the specific characteristics of non-PD pixel arrays. The resulting interpolated image provides a higher resolution output while maintaining the benefits of using non-PD pixel technology, such as reduced power consumption or improved sensitivity. This approach is particularly useful in applications where traditional photodiode-based sensors are impractical or where enhanced resolution is desired without increasing sensor complexity.
3. The computer-implemented method of claim 1 , wherein the matching comprises matching the first pixel to the higher resolution image based on a location information associated with the first pixel and an observation window size, in order to obtain a first set of cost values indicating how well the first pixel matches pixels of the higher resolution image within an observation window.
This invention relates to image processing techniques for enhancing the resolution of digital images. The problem addressed is the challenge of accurately matching pixels from a lower-resolution image to corresponding pixels in a higher-resolution image, which is essential for tasks like super-resolution imaging, image reconstruction, and computational photography. The method involves determining how well a pixel from the lower-resolution image aligns with pixels in the higher-resolution image by analyzing their spatial relationships and local context. The process begins by selecting a pixel from the lower-resolution image and using its location information to identify a corresponding region in the higher-resolution image. An observation window size is defined to limit the search area around the pixel's location, ensuring computational efficiency. Within this window, the method calculates a set of cost values that quantify the similarity between the selected pixel and nearby pixels in the higher-resolution image. These cost values help determine the best possible match, improving the accuracy of pixel alignment and enhancing the overall image resolution. The technique leverages spatial correlations and local features to refine the matching process, ensuring that the resulting higher-resolution image retains structural and visual fidelity. This approach is particularly useful in applications requiring precise image reconstruction, such as medical imaging, satellite imagery, and digital forensics.
4. The computer-implemented method of claim 3 , wherein the plurality of PD pixels includes a plurality of PD pixel types, and wherein each PD pixel type corresponds with a direction the PD pixel is sensitive to.
This invention relates to a computer-implemented method for processing signals from photodiode (PD) pixels in an imaging system, particularly where the PD pixels are of multiple types, each sensitive to light from a specific direction. The method addresses the challenge of accurately determining the direction of incident light in imaging applications, such as depth sensing or light field imaging, where conventional single-type PD pixels lack directional sensitivity. The method involves analyzing signals from a plurality of PD pixels, where the pixels are categorized into different types based on their directional sensitivity. Each PD pixel type is designed to respond to light arriving from a distinct direction, allowing the system to distinguish between light sources or reflections from different angles. By processing the signals from these directionally sensitive PD pixels, the method reconstructs directional information about the incident light, improving accuracy in applications like 3D imaging, object tracking, or scene reconstruction. The method may include steps such as capturing light signals from the PD pixels, classifying the signals based on their corresponding pixel types, and combining the directional data to generate a directional light profile. This approach enhances the imaging system's ability to resolve fine details in depth or angular information, overcoming limitations of traditional imaging sensors that lack directional sensitivity. The invention is particularly useful in advanced imaging systems requiring high-precision spatial or angular resolution.
5. The computer-implemented method of claim 4 , wherein the first PD pixel image is generated based on a first type of PD pixels and further comprising: generating a second PD pixel image from the first light information based on light information obtained from a second type of PD pixels; matching a second pixel of the second PD pixel image to the higher resolution image based on a location of a second PD pixel associated with the second pixel within the observation window size to obtain a second set of cost values; determining that a point on a uniform grid is associated with values for both the first set of cost values and the second set of cost values; and determining a disparity value for the point based on a weighted average of the associated values from the first set of cost values and the second set of cost values.
This invention relates to computer vision techniques for generating depth maps using photodiode (PD) pixel data from an image sensor. The problem addressed is improving depth estimation accuracy by leveraging multiple types of PD pixels to enhance spatial resolution and reduce noise in disparity calculations. The method involves capturing light information from a scene using an image sensor with at least two distinct types of PD pixels, each type having different light-sensing characteristics. A first PD pixel image is generated from light information obtained from a first type of PD pixels, and a second PD pixel image is generated from a second type of PD pixels. Each pixel in these images is matched to a higher-resolution reference image based on the physical location of its corresponding PD pixel within a defined observation window. This matching process produces two sets of cost values representing potential disparities for each point in the scene. For points on a uniform grid where both sets of cost values are available, a disparity value is computed as a weighted average of the corresponding values from both sets. This fusion of data from multiple PD pixel types improves depth accuracy by combining complementary information, reducing ambiguity and noise in the final depth map. The technique is particularly useful in applications requiring high-resolution depth sensing, such as augmented reality, robotics, and advanced driver-assistance systems.
6. The computer-implemented method of claim 5 , wherein the weighted average is based on a weight assigned to cost values of a set of cost values based on how similar the cost values of the set of cost values are to each other.
This invention relates to a computer-implemented method for optimizing cost calculations in a system where multiple cost values are involved. The problem addressed is the need to accurately assess and compare costs when dealing with a set of cost values that may vary in similarity, ensuring that the most relevant or representative cost values are given appropriate weight in the final calculation. The method involves computing a weighted average of cost values, where the weight assigned to each cost value is determined by its similarity to other cost values in the set. Cost values that are more similar to others in the set are given higher weights, while those that are less similar receive lower weights. This approach ensures that the weighted average reflects the most representative or consistent cost values, improving the accuracy of cost assessments in scenarios where cost data may be noisy or inconsistent. The method may be applied in various domains, such as financial analysis, resource allocation, or supply chain management, where cost comparisons and optimizations are critical. By dynamically adjusting weights based on similarity, the system avoids skewing results due to outliers or less representative cost values, leading to more reliable decision-making. The technique can be integrated into larger cost analysis frameworks to enhance their precision and robustness.
7. The computer-implemented method of claim 4 , wherein the higher resolution image comprises a full resolution image, wherein first PD pixel image is generated based on a first type of PD pixels, and wherein the first resolution is based on a number of the first type of PD pixels.
This invention relates to image processing in digital imaging systems, particularly for enhancing image resolution using photodiode (PD) pixel arrays. The problem addressed is the limited resolution of images captured by conventional imaging sensors due to the physical constraints of pixel density. The solution involves generating a higher resolution image, such as a full-resolution image, by leveraging multiple types of PD pixels within the sensor array. The method generates a first lower-resolution image from a subset of PD pixels of a first type, where the resolution of this image is determined by the number of these first-type PD pixels. The higher-resolution image is then constructed by combining data from these lower-resolution images, potentially using additional PD pixel types or processing techniques to improve detail and clarity. This approach allows for more efficient use of sensor hardware while achieving superior image quality compared to traditional single-pixel-type sensors. The technique is particularly useful in applications requiring high-resolution imaging, such as medical imaging, surveillance, and advanced photography.
8. The computer-implemented method of claim 3 , further comprising adjusting the observation window size based on one or more properties associated with the first light information.
This invention relates to computer-implemented methods for processing light information, particularly in systems that analyze or interpret light-based data, such as imaging, sensing, or optical communication applications. The core problem addressed is optimizing the observation window size used to capture or analyze light information, ensuring accurate and efficient data processing while adapting to varying conditions. The method involves dynamically adjusting the observation window size based on properties associated with the first light information. These properties may include intensity, wavelength, phase, polarization, or other characteristics that influence how the light is detected or processed. By modifying the window size in response to these properties, the system can improve signal quality, reduce noise, or enhance resolution. For example, if the light intensity is low, a larger window may be used to gather more photons, while a smaller window may be preferred for high-intensity light to avoid saturation. The method builds on a broader system that captures light information from a scene or source, processes it to extract relevant data, and may involve comparing it to reference light information. The dynamic adjustment of the window size ensures that the system remains adaptable to different lighting conditions, environmental factors, or operational requirements, leading to more reliable and precise results. This approach is particularly useful in applications where light conditions are variable, such as in medical imaging, remote sensing, or autonomous vehicle navigation.
9. The computer-implemented method of claim 3 , further comprising generating a grid of disparity values based on the obtained first set of cost values.
A computer-implemented method addresses the challenge of accurately determining depth information from stereo images by generating a grid of disparity values. The method involves obtaining a first set of cost values representing the similarity between corresponding pixels in a pair of stereo images. These cost values are derived from a cost volume, which is constructed by comparing pixel intensities or features across the images at multiple disparity levels. The method then generates a grid of disparity values by analyzing the cost volume, where each disparity value corresponds to the most likely depth displacement between the stereo images at a given pixel location. This grid can be refined further using optimization techniques, such as semi-global matching or stereo matching algorithms, to improve accuracy. The resulting disparity map is used to reconstruct a 3D scene or depth map, enabling applications in autonomous driving, robotics, and computer vision. The method ensures robust depth estimation by leveraging cost volume analysis and grid-based disparity computation.
10. The computer-implemented method of claim 1 , wherein the first PD pixel has a direction dependent light sensitivity based on a portion of a field of view of the PD pixel that is blocked.
A computer-implemented method improves the performance of photodiode (PD) pixels in imaging systems by enhancing their directional light sensitivity. The method addresses the challenge of achieving accurate light detection in environments where light sources are not uniformly distributed or where certain angles of incidence are more critical. The solution involves modifying the light sensitivity of a PD pixel based on a portion of its field of view that is intentionally blocked. By selectively blocking part of the field of view, the PD pixel becomes more sensitive to light coming from specific directions, improving signal-to-noise ratio and reducing interference from unwanted light sources. This directional sensitivity is particularly useful in applications such as depth sensing, where precise light detection from specific angles is essential. The method may also include additional steps such as adjusting the blocking mechanism dynamically to adapt to changing lighting conditions or scene requirements. The overall approach enhances the accuracy and reliability of light detection in imaging systems by optimizing the PD pixel's response to directional light inputs.
11. A non-transitory program storage device comprising instructions stored thereon to cause one or more processors to: obtain first light information from a set of light-sensitive pixels for a scene, the set of light-sensitive pixels including a plurality of phase detection (PD) pixels and a plurality of non-PD pixels; obtain location information for the plurality of PD pixels; generate a first PD pixel image from the first light information based on location information obtained from the plurality of PD pixels, wherein the first PD pixel image has a first resolution; generate a higher resolution image from the first light information obtained from the plurality of non-PD pixels, wherein the higher resolution image has a second resolution that is greater than the first resolution of the first PD pixel image; match a first pixel of the first PD pixel image to the higher resolution image, wherein the match is based on a set of correlations between the first pixel and non-PD pixels within a predetermined distance of the first pixel; and determine a disparity map for an image associated with the first light information, based on the match.
This invention relates to image processing in digital imaging systems, specifically for generating disparity maps using phase detection (PD) pixels and non-PD pixels in a sensor array. The problem addressed is the limited resolution of PD pixel-based depth sensing, which restricts accurate disparity map generation. The solution involves a method to enhance depth estimation by combining low-resolution PD pixel data with higher-resolution non-PD pixel data. The system obtains light information from a sensor containing both PD and non-PD pixels. PD pixels are used to capture phase-based depth information, while non-PD pixels provide higher-resolution color or intensity data. The system generates a low-resolution PD pixel image from the PD pixel data and a higher-resolution image from the non-PD pixel data. To align these images, the system matches PD pixels to corresponding non-PD pixels within a predefined search area, using correlation-based techniques. This alignment enables the generation of a disparity map, which represents depth information at a resolution higher than what PD pixels alone could provide. The method improves depth sensing accuracy in imaging devices, particularly for applications requiring high-resolution depth maps, such as 3D imaging and autofocus systems.
12. The non-transitory program storage device of claim 11 , wherein higher resolution image comprises an interpolated image generated by interpolating light information obtained from the plurality of non-PD pixels.
This invention relates to image processing, specifically improving image resolution in imaging systems. The problem addressed is the limited resolution of images captured by sensors with non-photosensitive (non-PD) pixels, which cannot directly detect light. These non-PD pixels are typically used for other functions, such as phase detection autofocus, but their inability to capture light reduces the overall image resolution. The invention provides a method to generate a higher resolution image by interpolating light information obtained from the plurality of non-PD pixels. The process involves using data from these non-PD pixels, which may not directly capture light, to estimate or reconstruct missing light information. This interpolated data is then combined with light information from photosensitive (PD) pixels to produce an image with improved resolution compared to what would be achievable using only the PD pixels. The interpolation may involve algorithms that predict light values at non-PD pixel locations based on surrounding PD pixel data or other available information. The invention is implemented in a non-transitory program storage device, such as a memory or storage medium, containing executable instructions for performing the interpolation and image reconstruction. This approach enhances image quality in systems where non-PD pixels are present, such as advanced camera modules with phase detection autofocus capabilities. The solution is particularly useful in applications where maximizing resolution is critical, such as high-end photography or imaging systems with specialized pixel arrangements.
13. The non-transitory program storage device of claim 11 , wherein the instructions for matching the first pixel comprise the instructions to cause the one or more processors to match the first pixel of the first PD pixel image to the higher resolution image based on a location of the first pixel and an observation window size in order to obtain a first set of cost values indicating how well the first pixel matches pixels of the higher resolution image within an observation window.
This invention relates to image processing, specifically techniques for matching pixels from a lower-resolution photodiode (PD) pixel image to a higher-resolution image. The problem addressed is accurately aligning and matching pixels between images of different resolutions, which is critical for applications like image enhancement, super-resolution reconstruction, or sensor fusion. The solution involves a method for matching a first pixel from a PD pixel image to a higher-resolution image by analyzing its location and an observation window size. The observation window defines a region in the higher-resolution image where potential matches for the first pixel are evaluated. The system computes a set of cost values representing how well the first pixel matches pixels within this window, enabling precise alignment. The observation window size can be dynamically adjusted based on factors like image content or noise levels to improve matching accuracy. This approach ensures robust pixel correspondence even in challenging conditions, such as low-light environments or when dealing with misaligned sensors. The method is implemented via a non-transitory program storage device containing executable instructions for a processor to perform the matching process. The invention also includes techniques for refining the matching process by considering additional pixels or applying optimization algorithms to minimize cost values, ensuring high-quality image alignment.
14. The non-transitory program storage device of claim 13 , wherein the plurality of PD pixels includes a plurality of PD pixel types, and wherein each PD pixel type corresponds with a direction the PD pixel is sensitive to.
A non-transitory program storage device contains instructions for processing image data from a pixel array that includes multiple photodiode (PD) pixels. The PD pixels are of different types, each type being sensitive to light from a specific direction. This directional sensitivity allows the device to capture and analyze light incident from different angles, enabling applications such as light field imaging, depth sensing, or motion detection. The program storage device processes the directional data from the PD pixels to reconstruct or enhance the image, improving accuracy in determining light source directionality or object positioning. The system may also include additional processing steps, such as filtering or combining data from multiple PD pixel types to refine the output. This approach enhances imaging capabilities by leveraging directional light sensitivity, which is useful in advanced imaging systems requiring spatial or angular resolution beyond traditional pixel arrays. The technology addresses limitations in conventional imaging by providing more detailed directional information, improving applications like 3D imaging, augmented reality, or computational photography.
15. The non-transitory program storage device of claim 14 , wherein the first PD pixel image is generated based on a first type of PD pixels and wherein the instructions further cause the one or more processors to: generate a second PD pixel image from the first light information based on light information obtained from a second type of PD pixels; match a second pixel of the second PD pixel image to the higher resolution image based on a location of a second PD pixel associated with the second pixel within the observation window size to obtain a second set of cost values; determine that a point on a uniform grid is associated with values for both the first set of cost values and the second set of cost values; and determine a disparity value for the point based on a weighted average of the associated values from the first set of cost values and the second set of cost values.
This invention relates to image processing for depth sensing using phase detection (PD) pixels in imaging systems. The problem addressed is improving depth accuracy by leveraging multiple types of PD pixels to enhance disparity estimation. The system captures light information from a scene using an image sensor with at least two types of PD pixels, each type providing different phase detection data. A first PD pixel image is generated from light information obtained from a first type of PD pixels, and a second PD pixel image is generated from a second type of PD pixels. The system then matches pixels from both PD pixel images to a higher-resolution reference image based on the physical locations of the PD pixels within a defined observation window. This matching process produces two sets of cost values representing potential disparities. For points on a uniform grid where both sets of cost values are available, the system calculates a disparity value by computing a weighted average of the corresponding values from both sets. This approach combines information from multiple PD pixel types to improve depth estimation accuracy and robustness. The method is implemented via instructions stored on a non-transitory program storage device, executed by one or more processors.
16. The non-transitory program storage device of claim 15 , wherein the weighted average is based on a weight assigned to cost values of a set of cost values based on how similar the cost values of the set of cost values are to each other.
This invention relates to a non-transitory program storage device containing instructions for optimizing cost calculations in a system. The system addresses the problem of accurately determining costs in scenarios where multiple cost values are involved, particularly when these values exhibit varying degrees of similarity. The program storage device includes instructions for computing a weighted average of cost values, where the weighting is determined by the similarity of the cost values within a set. Higher similarity between cost values results in greater influence on the weighted average, improving the accuracy of cost assessments. The system may also include instructions for generating a cost model, which is used to predict future costs based on historical data. The cost model may incorporate machine learning techniques to refine predictions over time. Additionally, the system may include instructions for adjusting the cost model based on feedback, ensuring continuous improvement in cost estimation accuracy. The weighted average calculation helps mitigate the impact of outliers or dissimilar values, leading to more reliable cost assessments in applications such as financial forecasting, resource allocation, or supply chain management. The invention enhances decision-making by providing a more precise and adaptable cost analysis framework.
17. The non-transitory program storage device of claim 14 , wherein the higher resolution image comprises a full resolution image, wherein first PD pixel image is generated based on a first type of PD pixels, and wherein the first resolution is based on a number of the first type of PD pixels.
This invention relates to image processing in digital imaging systems, particularly for enhancing image resolution using photodiode (PD) pixel data. The problem addressed is the limited resolution of images captured by conventional imaging sensors, which often rely on a single type of PD pixel, restricting the achievable resolution. The invention involves a non-transitory program storage device containing instructions for processing image data. The system generates a higher resolution image, such as a full-resolution image, by utilizing multiple types of PD pixels. Specifically, a first PD pixel image is created based on a first type of PD pixels, where the resolution of this image is determined by the number of these first-type PD pixels. The system may also incorporate additional PD pixel types to further refine the image, improving detail and clarity. The method involves capturing raw image data from the sensor, processing the data to extract information from different PD pixel types, and combining this data to produce a higher-resolution output. This approach leverages the distinct characteristics of different PD pixel types to enhance image quality beyond what is possible with a single pixel type. The invention is particularly useful in applications requiring high-resolution imaging, such as medical imaging, surveillance, and high-end photography.
18. The non-transitory program storage device of claim 13 , wherein the instructions further comprise instructions to cause the one or more processors to generate a grid of disparity values based on the obtained first set of cost values.
A system and method for generating a grid of disparity values in computer vision applications, particularly for depth estimation or 3D reconstruction. The invention addresses the challenge of efficiently computing disparity maps from stereo image pairs or multi-view imagery, which is computationally intensive and often requires optimization techniques to balance accuracy and performance. The system obtains a first set of cost values representing matching costs between corresponding pixels in multiple images, typically derived from stereo matching or multi-view stereo algorithms. These cost values are used to generate a grid of disparity values, where each disparity value represents the horizontal (or vertical) shift between corresponding pixels in the images. The grid may be refined using additional constraints, such as smoothness or occlusion handling, to improve accuracy. The invention optimizes the disparity computation process by leveraging precomputed cost values, reducing redundant calculations and improving efficiency. This approach is useful in applications like autonomous driving, robotics, and 3D imaging, where real-time depth estimation is critical. The system may be implemented in hardware or software, with the grid generation step being a key component of the overall disparity estimation pipeline.
19. The non-transitory program storage device of claim 11 , wherein the first PD pixel has a direction dependent light sensitivity based on a portion of a field of view of the PD pixel that is blocked.
A system for enhancing image capture in electronic devices addresses the challenge of improving light sensitivity in photodiode (PD) pixels, particularly in low-light conditions. The invention involves a non-transitory program storage device containing instructions for controlling a pixel array, where at least one PD pixel exhibits direction-dependent light sensitivity. This sensitivity is achieved by selectively blocking a portion of the pixel's field of view, which modifies its response to incoming light based on the angle of incidence. The blocked portion can be implemented using a physical barrier or a light-absorbing material positioned within or near the pixel structure. The system may also include additional PD pixels with different blocking configurations to capture directional light information, enabling advanced imaging techniques such as depth sensing or high dynamic range imaging. The instructions further process the captured light data to reconstruct an image with improved clarity and detail, particularly in scenarios where light sources are directional or partially obscured. This approach enhances the performance of imaging devices in low-light environments while maintaining compatibility with existing pixel array architectures.
20. An electronic device, comprising: one or more image capture devices; a display; a user interface; and one or more processors operatively coupled to a memory, wherein the one or more processors are configured to execute instructions stored in the memory causing the one or more processors to: obtain first light information from a set of light-sensitive pixels for a scene, the set of light-sensitive pixels including a plurality of phase detection (PD) pixels and a plurality of non-PD pixels; obtain location information associated with each of the plurality of PD pixels; generate a first PD pixel image from the first light information based on location information obtained from the plurality of PD pixels, wherein the first PD pixel image has a first resolution; obtain a higher resolution image having a second resolution greater than the first resolution of the first PD pixel image; match a first pixel of the first PD pixel image to the higher resolution image, wherein the match is based on a set of correlations between the first pixel and non-PD pixels within a predetermined distance of the first pixel; and determine a disparity map for an image associated with the first light information, based on the match.
This invention relates to electronic devices with enhanced image processing capabilities, particularly for improving depth sensing and autofocus using phase detection (PD) pixels. The problem addressed is the limited resolution of PD pixel-based imaging, which restricts accurate depth and focus calculations. The solution involves an electronic device with image capture devices, a display, a user interface, and processors that process light information from both PD and non-PD pixels. The device captures light data from a scene using a sensor array containing PD pixels and non-PD pixels, then generates a low-resolution PD pixel image based on the PD pixel locations. A higher-resolution image is obtained separately, and the system matches pixels from the low-resolution PD image to the higher-resolution image by correlating each PD pixel with nearby non-PD pixels within a predefined distance. This matching process enables the generation of a disparity map, which is used for depth estimation or autofocus adjustments. The technique improves depth sensing accuracy by leveraging higher-resolution data while utilizing the phase detection capabilities of PD pixels.
21. The electronic device of claim 20 , wherein the higher resolution image comprises an interpolated image generated by interpolating light information obtained from the plurality of non-PD pixels.
The invention relates to electronic devices with image sensors that capture images using non-photodiode (non-PD) pixels. A common challenge in such devices is achieving high-resolution images while maintaining image quality, as non-PD pixels may not directly capture light in the same way as traditional photodiodes. The invention addresses this by generating a higher resolution image through interpolation of light information obtained from multiple non-PD pixels. The device includes an image sensor with an array of non-PD pixels that detect light and convert it into electrical signals. These signals are processed to produce an interpolated image, where missing pixel data is estimated based on surrounding pixel values. This interpolation technique enhances resolution without requiring additional physical pixels, improving image quality in devices where non-PD pixels are used. The method involves capturing raw light information from the non-PD pixels, applying interpolation algorithms to fill in gaps, and outputting a refined, higher-resolution image. This approach is particularly useful in applications where traditional photodiode-based sensors are impractical or where space constraints limit sensor size. The invention ensures that images captured by non-PD pixels are sharp and detailed, making it suitable for compact imaging systems, mobile devices, and other applications requiring high-resolution output from non-standard pixel architectures.
22. The electronic device of claim 20 , wherein the one or more processors are configured to match the first pixel by executing instructions to cause the one or more processors to match the first pixel of the first PD pixel image to the higher resolution image based on a location of the first pixel and an observation window size in order to obtain a first set of cost values indicating how well the first pixel matches pixels of the higher resolution image within an observation window.
This invention relates to image processing in electronic devices, specifically improving the accuracy of pixel matching between a lower-resolution phase detection (PD) pixel image and a higher-resolution image. The problem addressed is the challenge of precisely aligning pixels from a lower-resolution sensor with corresponding pixels in a higher-resolution image, which is critical for applications like autofocus and image stabilization. The electronic device includes one or more processors configured to match a first pixel from the PD pixel image to the higher-resolution image. The matching process involves analyzing the pixel's location and an observation window size to determine how well the first pixel aligns with pixels in the higher-resolution image within that window. The result is a set of cost values that quantify the match quality, enabling more accurate pixel correspondence. This technique helps improve the reliability of pixel-level alignment, which is essential for tasks like phase detection autofocus and image reconstruction. The observation window size defines the search area in the higher-resolution image where potential matches for the first pixel are evaluated. By comparing the first pixel against multiple candidate pixels within this window, the system generates cost values that reflect the similarity between the PD pixel and the higher-resolution image pixels. This approach enhances the precision of pixel matching, reducing errors in applications that depend on accurate alignment between different image resolutions.
23. The electronic device of claim 22 , wherein the one or more processors are configured to execute instructions to cause the one or more processors to generate a grid of disparity values based on the obtained first set of cost values.
This invention relates to electronic devices configured for depth estimation in computer vision, addressing the challenge of accurately determining depth information from stereo images. The device includes one or more processors that obtain a first set of cost values representing matching costs between corresponding pixels in a pair of stereo images. These cost values are used to generate a grid of disparity values, where each disparity value indicates the horizontal shift between corresponding pixels in the stereo images. The grid of disparity values is then used to construct a depth map, which provides a three-dimensional representation of the scene captured in the images. The device may further refine the disparity values by applying aggregation techniques to reduce noise and improve accuracy. Additionally, the processors may generate a second set of cost values based on the first set, incorporating additional constraints such as smoothness or uniqueness to enhance depth estimation. The final disparity values are then used to generate a depth map, which can be applied in various applications such as autonomous navigation, 3D reconstruction, and augmented reality. The invention improves upon existing methods by leveraging cost aggregation and refinement techniques to produce more accurate and reliable depth information from stereo images.
24. The electronic device of claim 20 , wherein the first PD pixel has a direction dependent light sensitivity based on a portion of a field of view of the PD pixel that is blocked.
The invention relates to electronic devices with phase detection (PD) pixels in imaging systems, addressing the challenge of improving autofocus accuracy by enhancing light sensitivity control in PD pixels. The device includes an image sensor with PD pixels, where at least one PD pixel has a direction-dependent light sensitivity. This sensitivity is adjusted by blocking a portion of the pixel's field of view, which modifies how light is received from different angles. The blocked portion creates an asymmetric light sensitivity profile, allowing the pixel to distinguish between light rays arriving from different directions. This directional sensitivity improves phase detection accuracy, enabling more precise autofocus performance in imaging applications. The device may also include additional PD pixels with similar or different blocking configurations to further enhance autofocus capabilities. The overall system integrates these PD pixels with processing circuitry to analyze phase differences and adjust focus accordingly. The invention is particularly useful in digital cameras, smartphones, and other imaging devices requiring high-precision autofocus.
Unknown
September 1, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.