To simplify a configuration for increasing an amount of information on an observation space, taken image acquisition means of an image processing system acquires taken images that have been taken by image taking means, which is movable in a real space. Observation space information acquisition means acquires, based on changes in position of a feature point cloud in the taken images, observation space information including three-dimensional coordinates of the feature point cloud in an observation space. Machine learning means acquires, based on machine learning data on a feature of an object, additional information on a feature of a photographed object shown in the taken images. Integration means integrates the observation space information and the additional information.
Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
2. The image processing system according to claim 1, wherein the at least one processor is configured to partially change the mesh after changing the scale of the mesh based on the result of the comparison between the two-dimensional observation information and the two-dimensional feature amount information.
3. The image processing system according to claim 1, wherein the additional information includes information on a three-dimensional shape of the photographed object, which is estimated based on the machine learning data.
The invention relates to an image processing system that enhances captured images by incorporating additional information derived from machine learning. The system addresses the challenge of limited visual data in photographs by estimating and integrating supplementary details, such as the three-dimensional (3D) shape of the photographed object. This is achieved by analyzing the image using pre-trained machine learning models to infer depth, contours, or other structural features that are not explicitly visible in the original two-dimensional image. The system then overlays or merges this estimated 3D shape information with the original image to produce an enriched output. This approach improves the realism and utility of the image, particularly for applications requiring spatial awareness, such as augmented reality, object recognition, or 3D modeling. The machine learning data used for estimation may include trained neural networks or other algorithms that have been exposed to large datasets of 3D shapes and corresponding 2D images, enabling accurate predictions. The system dynamically processes the input image to extract and apply the relevant 3D shape information, ensuring seamless integration without manual intervention. This enhances the image's informational content while maintaining visual coherence.
4. The image processing system according to claim 3, wherein the additional information includes information on the mesh of the photographed object.
This invention relates to an image processing system designed to enhance the accuracy of three-dimensional (3D) reconstruction from captured images. The system addresses the challenge of generating precise 3D models by incorporating additional information about the photographed object, specifically details about its mesh structure. The mesh information helps refine the 3D reconstruction process by providing a structured framework that improves the alignment and accuracy of the generated model. The system captures images of the object from multiple angles and processes these images to extract depth and surface data. By integrating mesh data, the system ensures that the reconstructed 3D model maintains geometric consistency and fidelity to the original object. This approach is particularly useful in applications requiring high-precision 3D modeling, such as industrial design, medical imaging, and virtual reality. The mesh information may include vertex coordinates, edge connections, and surface normals, which are used to optimize the reconstruction algorithm. The system dynamically adjusts the reconstruction parameters based on the mesh data to minimize errors and distortions in the final 3D model. This method ensures that the reconstructed object retains its original shape and dimensions with high accuracy.
5. The image processing system according to claim 4, wherein the at least one processor is configured to set the mesh in the observation space based on the additional information, and change the mesh based on the observation space information.
6. The image processing system according to claim 5, wherein the at least one processor is configured to change a first mesh portion of the mesh that corresponds to the three-dimensional coordinates of the feature point cloud indicated by the observation space information, and then change a second mesh portion around the first mesh portion.
8. The image processing system according to claim 5, wherein the at least one processor is configured to change scale of the mesh based on the observation space information.
The image processing system operates in the domain of computer vision and 3D modeling, addressing the challenge of accurately representing and processing complex scenes with varying levels of detail. The system includes a mesh-based representation of a 3D scene, where the mesh is dynamically adjusted to optimize computational efficiency and accuracy. A key feature is the ability to change the scale of the mesh based on observation space information, which refers to data about the viewing perspective, distance, or other contextual factors that influence how the scene is perceived. By dynamically scaling the mesh, the system can reduce computational load when high detail is unnecessary, such as when objects are far from the observer, while maintaining fine detail in regions of interest. This adaptive scaling improves rendering performance and resource utilization without sacrificing visual fidelity. The system also includes a processor that generates and updates the mesh based on input data, such as sensor or camera feeds, ensuring real-time or near-real-time adjustments to the mesh structure. The dynamic scaling mechanism is particularly useful in applications like virtual reality, augmented reality, and autonomous navigation, where efficient 3D scene processing is critical. The invention enhances existing mesh-based rendering techniques by introducing adaptive resolution control, making it more suitable for resource-constrained environments.
9. The image processing system according to claim 3, wherein the additional information includes information on a normal of the photographed object.
10. The image processing system according to claim 3, wherein the additional information includes information on a classification of the photographed object.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
August 14, 2017
November 8, 2022
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.