Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A method for three-dimensional modeling, comprising: receiving a first two-dimensional image and a depth map corresponding to the first two-dimensional image, wherein the first two-dimensional image and the depth map respectively comprises a face; and fitting a three-dimensional face model by a first three-dimensional face database according to a position of a face feature point in the first two-dimensional image and the depth map corresponding to the first two-dimensional image.
This invention relates to three-dimensional (3D) face modeling using two-dimensional (2D) images and depth maps. The problem addressed is the accurate reconstruction of 3D facial models from 2D images, which often lack depth information, leading to incomplete or distorted models. The solution involves combining a 2D image of a face with a corresponding depth map to generate a precise 3D face model. The method begins by receiving a 2D image of a face and its corresponding depth map, which provides depth information for each pixel in the 2D image. The system then identifies key facial feature points (e.g., eyes, nose, mouth) in both the 2D image and the depth map. Using these feature points, the method fits a 3D face model from a pre-existing 3D face database to the input data. The database contains multiple 3D face models, allowing the system to select the best match based on the detected feature points and depth information. This approach ensures that the generated 3D model accurately represents the facial structure, improving realism and usability in applications such as facial recognition, animation, or virtual reality. The use of depth maps enhances accuracy compared to traditional 2D-to-3D conversion methods.
2. The method according to claim 1 , wherein the first three-dimensional face database comprises an average face model and at least one of the following: a shape feature vector and an expression feature vector.
This invention relates to three-dimensional face modeling and analysis, specifically improving the accuracy and efficiency of face recognition or reconstruction systems. The core challenge addressed is the variability in facial shapes and expressions, which can hinder reliable face processing in applications like biometrics, animation, or medical imaging. The method involves using a first three-dimensional face database that includes an average face model, which serves as a reference template for facial structure. This database is enhanced with additional data to improve modeling precision. Specifically, it incorporates a shape feature vector, which captures key geometric variations in facial contours, and an expression feature vector, which encodes dynamic changes in facial expressions. These vectors allow the system to adjust the average face model to match specific individuals or expressions more accurately. By combining the average face model with these feature vectors, the system can generate more detailed and adaptable 3D face representations. This approach enhances the ability to reconstruct or recognize faces under varying conditions, such as different lighting, angles, or expressions. The inclusion of both shape and expression data ensures that the model can handle both static and dynamic facial characteristics, improving overall robustness in face processing applications.
3. The method according to claim 2 , wherein the step of fitting the three-dimensional face model by the first three-dimensional face database according to the position of the face feature point in the first two-dimensional image and the depth map corresponding to the first two-dimensional image comprises: estimating an initial transformation matrix of a three-dimensional point cloud model corresponding to the depth map from the average face model, according to the initial transformation matrix and at least one of the following: the shape feature vector and the expression feature vector, and aiming at a first condition, fitting the three-dimensional face model from the average face model.
This invention relates to three-dimensional face modeling using depth maps and two-dimensional images. The problem addressed is accurately fitting a three-dimensional face model to a subject's face by leveraging both two-dimensional image data and depth information. The method involves estimating an initial transformation matrix for a three-dimensional point cloud model derived from a depth map, using an average face model as a reference. The transformation matrix is refined by incorporating shape and expression feature vectors to optimize the fit. The process aims to meet a first condition, which likely involves minimizing errors between the model and the actual face data. The technique improves upon prior methods by combining multiple feature vectors and depth information to enhance the accuracy of the three-dimensional face reconstruction. This approach is particularly useful in applications requiring precise facial modeling, such as biometric identification, augmented reality, and facial recognition systems. The method ensures that the resulting three-dimensional model accurately represents the subject's facial structure and expressions.
4. The method according to claim 3 , wherein the first condition comprises at least one of the following: a distance between a projection position of a feature point of the three-dimensional face model in an image coordinate system and the position of the feature point of the first two-dimensional image corresponding to the feature point of the three-dimensional face model is smallest; and a distance between the three-dimensional face model and a point pair corresponding to the three-dimensional point cloud mapped by the depth map is smallest.
This invention relates to three-dimensional face modeling and alignment techniques, specifically addressing the challenge of accurately aligning a three-dimensional face model with two-dimensional images and depth maps. The method involves optimizing the alignment process by evaluating specific conditions to ensure precise matching between the 3D model and the input data. The technique focuses on refining the alignment by minimizing distances between key points. One condition involves determining the projection position of a feature point from the 3D face model in an image coordinate system and comparing it to the corresponding feature point in a first two-dimensional image. The alignment is optimized when this distance is minimized. Another condition involves comparing the 3D face model to a point pair derived from a three-dimensional point cloud mapped by a depth map, again optimizing alignment when the distance between these points is minimized. By evaluating these conditions, the method ensures that the 3D face model accurately aligns with both the 2D image and depth map data, improving the overall accuracy of facial reconstruction and tracking applications. This approach enhances the reliability of facial recognition, animation, and other applications requiring precise 3D facial modeling.
5. The method according to claim 3 , wherein the step of estimating the initial transformation matrix of the three-dimensional point cloud model corresponding to the depth map by the average face model comprises: calculating a three-dimensional position of the face feature point according to the position of the face feature point in the first two-dimensional image and the depth map corresponding to the first two-dimensional image; and estimating the initial transformation matrix of the three-dimensional point cloud model corresponding to the depth map from the average face model according to the three-dimensional position of the face feature point and the three-dimensional position of the feature point of the average face model.
This invention relates to three-dimensional face modeling, specifically improving the accuracy of aligning a depth map-derived 3D point cloud model with an average face model. The problem addressed is the challenge of precisely transforming a 3D point cloud model, generated from a depth map and a 2D image, to match a standardized average face model for applications like facial recognition or animation. The method involves calculating the 3D positions of facial feature points by combining the 2D coordinates of these points in an image with corresponding depth values from the depth map. These 3D feature points are then used to estimate an initial transformation matrix that aligns the 3D point cloud model with the average face model. The transformation accounts for differences between the observed 3D feature points and the known 3D positions of corresponding feature points in the average face model, ensuring accurate alignment. This approach enhances the accuracy of 3D face reconstruction by leveraging both depth information and a pre-existing average face model, reducing errors in pose and scale estimation. The technique is particularly useful in applications requiring precise 3D face modeling, such as biometric identification, facial animation, or medical imaging.
6. A device for three-dimensional modeling, comprising: a receiver, configured to receive a first two-dimensional image and a depth map corresponding to the first two-dimensional image, wherein the first two-dimensional image and the depth map respectively comprise a face; and a processor, configured to fit a three-dimensional face model by a first three-dimensional face database according to a position of a face feature point in the first two-dimensional image and the depth map corresponding to the first two-dimensional image.
This invention relates to three-dimensional (3D) face modeling using 2D images and depth maps. The problem addressed is the accurate reconstruction of 3D facial models from 2D images, which often lack depth information, leading to incomplete or distorted models. The solution involves a device that processes a 2D image of a face along with a corresponding depth map to generate a precise 3D face model. The device includes a receiver that obtains a 2D image and its associated depth map, both depicting a face. A processor then uses these inputs to fit a 3D face model by referencing a pre-existing 3D face database. The fitting process relies on the positions of facial feature points identified in the 2D image and the depth map. By aligning these points with the database, the system constructs an accurate 3D representation of the face. The depth map provides essential depth information that complements the 2D image, enabling the system to resolve ambiguities in facial geometry. The 3D face database serves as a reference, ensuring the generated model adheres to realistic facial structures. This approach improves the accuracy and realism of 3D face modeling compared to methods relying solely on 2D images. Applications include facial recognition, animation, and medical imaging.
7. The device according to claim 6 , wherein the first three-dimensional face database comprises an average face model and at least one of the following: a shape feature vector and an expression feature vector.
This invention relates to a device for facial recognition or analysis, specifically addressing the challenge of accurately modeling and comparing three-dimensional facial features. The device includes a first three-dimensional face database that stores an average face model, which serves as a reference for standard facial geometry. Additionally, the database includes at least one of two key feature vectors: a shape feature vector and an expression feature vector. The shape feature vector captures the static geometric structure of a face, such as the relative positions of facial landmarks, while the expression feature vector represents dynamic facial expressions, such as smiles or frowns. These vectors allow the device to analyze and compare facial features in a structured manner, improving accuracy in applications like biometric authentication, emotion recognition, or facial reconstruction. The inclusion of both static and dynamic features enables the device to handle variations in facial expressions and lighting conditions, enhancing robustness in real-world scenarios. The device may also include a second three-dimensional face database for additional reference models or user-specific data, further improving personalization and accuracy. The overall system leverages these databases to process and interpret facial data efficiently, addressing limitations in traditional two-dimensional facial recognition methods.
8. The device according to claim 7 , wherein the processor is specifically configured to: estimate an initial transformation matrix of a three-dimensional point cloud model corresponding to the depth map from the average face model, according to the initial transformation matrix and at least one of the following: the shape feature vector and the expression feature vector, and aiming at a first condition, fitting the three-dimensional face model from the average face model.
This invention relates to three-dimensional (3D) face modeling and transformation, addressing the challenge of accurately fitting a 3D face model to a depth map derived from a captured image. The system uses an average face model as a reference to generate an initial transformation matrix for aligning a 3D point cloud model corresponding to the depth map. The processor then refines this alignment by incorporating shape and expression feature vectors, optimizing the fit to meet a predefined condition. The shape feature vector captures structural variations in facial geometry, while the expression feature vector accounts for dynamic facial expressions. By combining these features with the initial transformation matrix, the system achieves a more precise and adaptable 3D face model that accurately represents the subject's facial structure and expressions. This approach enhances applications in facial recognition, animation, and augmented reality by improving the accuracy and realism of 3D face reconstructions. The method ensures robustness by dynamically adjusting the model based on both static and dynamic facial characteristics, reducing errors in alignment and improving overall model fidelity.
9. The device according to claim 8 , wherein the first condition comprises at least one of the following: a distance between a projection position of a feature point of the three-dimensional face model in the image coordinate system and the position of the feature point of the first two-dimensional image corresponding to the feature point of the three-dimensional face model is smallest; and a distance between the three-dimensional face model and a point pair corresponding to the three-dimensional point cloud mapped by the depth map is smallest.
This invention relates to three-dimensional face modeling and alignment techniques, specifically addressing the challenge of accurately aligning a three-dimensional face model with two-dimensional images and depth maps. The device includes a processor configured to generate a three-dimensional face model from a three-dimensional point cloud obtained via a depth map. The processor aligns this model with a first two-dimensional image by optimizing a first condition, which ensures the model's feature points correspond accurately to the image. The alignment process involves minimizing the distance between the projection of the model's feature points in the image coordinate system and their corresponding positions in the first two-dimensional image. Additionally, the alignment may minimize the distance between the three-dimensional face model and the point pairs in the three-dimensional point cloud mapped by the depth map. This ensures the model accurately represents the subject's facial structure. The device may further refine the alignment using a second two-dimensional image, optimizing a second condition that minimizes the distance between the model's feature points and their corresponding positions in the second image. The system may also adjust the model's pose and scale to improve alignment accuracy. The invention enhances the precision of three-dimensional face modeling by leveraging multiple two-dimensional images and depth maps, ensuring robust and accurate facial feature mapping.
10. The device according to claim 8 , wherein the processor is specifically configured to: calculate a three-dimensional position of the face feature point according to the position of the face feature point in the first two-dimensional image and the depth map corresponding to the first two-dimensional image; and estimate the initial transformation matrix of the three-dimensional point cloud model corresponding to the depth map from the average face model according to the three-dimensional position of the face feature point and the three-dimensional position of the feature point of the average face model.
This invention relates to facial recognition and three-dimensional modeling using depth sensing technology. The problem addressed is accurately aligning a three-dimensional point cloud model of a face with an average face model to improve facial recognition or modeling accuracy. The device includes a processor configured to calculate the three-dimensional position of a face feature point. This is done by combining the position of the feature point in a two-dimensional image with a corresponding depth map. The depth map provides depth information, allowing the conversion of the two-dimensional feature point into a three-dimensional coordinate. The processor then estimates an initial transformation matrix that aligns the three-dimensional point cloud model (derived from the depth map) with an average face model. This alignment is based on the three-dimensional position of the detected face feature point and the corresponding feature point in the average face model. The transformation matrix helps adjust the orientation and position of the point cloud model to match the average face model, improving accuracy in facial recognition or modeling applications. This approach enhances the precision of facial feature mapping by leveraging depth information and a reference average face model, reducing errors in three-dimensional face reconstruction.
11. A computer readable storage medium, storing a computer program, wherein when the computer program is executed by a first processor, the following steps are performed: receiving a first two-dimensional image and a depth map corresponding to the first two-dimensional image, wherein the first two-dimensional image and the depth map respectively comprises a face; and fitting a three-dimensional face model by a first three-dimensional face database according to a position of a face feature point in the first two-dimensional image and the depth map corresponding to the first two-dimensional image.
This invention relates to computer vision and three-dimensional face modeling. The problem addressed is the accurate reconstruction of a three-dimensional face model from a two-dimensional image and its corresponding depth map. Existing methods may struggle with precise alignment and feature extraction, leading to inaccuracies in the generated 3D model. The invention provides a computer-readable storage medium containing a program that, when executed by a processor, performs the following steps. First, it receives a two-dimensional image and a depth map, both containing a face. The depth map provides spatial information to complement the 2D image. Next, the program fits a three-dimensional face model using a pre-existing 3D face database. This fitting process relies on the positions of facial feature points identified in the 2D image and the corresponding depth data. The depth map enhances the accuracy of feature localization, ensuring the 3D model closely matches the actual facial structure. The system leverages a database of 3D face models to refine the reconstruction, improving realism and precision. This approach is useful in applications like facial recognition, augmented reality, and biometric authentication, where accurate 3D face modeling is critical. The depth map integration ensures better depth perception, reducing errors in feature alignment and model fitting.
12. A device for three-dimensional modeling, comprising a memory, a second processor, and a computer program stored in the memory and operated by the second processor, wherein when the computer program is executed by the second processor, the following steps are performed: receiving a first two-dimensional image and a depth map corresponding to the first two-dimensional image, wherein the first two-dimensional image and the depth map respectively comprises a face; and fitting a three-dimensional face model by a first three-dimensional face database according to a position of a face feature point in the first two-dimensional image and the depth map corresponding to the first two-dimensional image.
This invention relates to three-dimensional modeling, specifically for generating a 3D face model from a 2D image and a corresponding depth map. The problem addressed is the accurate reconstruction of a 3D face model from limited input data, such as a single 2D image and its depth map, which often lacks sufficient detail for precise modeling. The device includes a memory, a processor, and a computer program stored in the memory. When executed, the program receives a 2D image and its corresponding depth map, both containing a face. The system then fits a 3D face model by referencing a pre-existing 3D face database. This fitting process uses the position of facial feature points in the 2D image and the depth map to align and refine the 3D model, ensuring accurate representation. The 3D face database provides a reference structure, allowing the system to adjust the model based on the input data's feature points and depth information. This approach enhances the accuracy of 3D face reconstruction from 2D inputs, improving applications in facial recognition, animation, and virtual reality.
13. The method according to claim 4 , wherein the step of estimating the initial transformation matrix of the three-dimensional point cloud model corresponding to the depth map by the average face model comprises: calculating a three-dimensional position of the face feature point according to the position of the face feature point in the first two-dimensional image and the depth map corresponding to the first two-dimensional image; and estimating the initial transformation matrix of the three-dimensional point cloud model corresponding to the depth map from the average face model according to the three-dimensional position of the face feature point and the three-dimensional position of the feature point of the average face model.
This invention relates to three-dimensional face modeling, specifically improving the accuracy of aligning a depth map-derived 3D point cloud model with an average face model. The problem addressed is the challenge of accurately estimating the initial transformation matrix that aligns a 3D point cloud model, generated from a depth map and a 2D image, with a pre-existing average face model. The solution involves calculating the 3D positions of face feature points by combining the 2D coordinates from the image with corresponding depth values from the depth map. These 3D positions are then compared to the known 3D positions of corresponding feature points in the average face model. The initial transformation matrix is estimated by determining the spatial relationship between these two sets of 3D points, enabling more accurate alignment of the depth map-derived model with the average face model. This process enhances the precision of 3D face reconstruction by leveraging both depth data and a reference face model.
14. The device according to claim 9 , wherein the processor is specifically configured to: calculate a three-dimensional position of the face feature point according to the position of the face feature point in the first two-dimensional image and the depth map corresponding to the first two-dimensional image; and estimate the initial transformation matrix of the three-dimensional point cloud model corresponding to the depth map from the average face model according to the three-dimensional position of the face feature point and the three-dimensional position of the feature point of the average face model.
This invention relates to facial recognition and three-dimensional modeling using depth-sensing technology. The problem addressed is accurately aligning a three-dimensional point cloud model of a face with an average face model to improve facial recognition or reconstruction accuracy. The device includes a processor configured to calculate the three-dimensional position of a face feature point. This is done by combining the position of the feature point in a two-dimensional image with a corresponding depth map. The depth map provides depth information, allowing the processor to determine the spatial coordinates of the feature point in three dimensions. Next, the processor estimates an initial transformation matrix that aligns a three-dimensional point cloud model (derived from the depth map) with an average face model. This is achieved by comparing the calculated three-dimensional position of the face feature point with the corresponding feature point in the average face model. The transformation matrix defines the rotation, translation, and scaling needed to align the point cloud model with the average face model, improving the accuracy of facial recognition or reconstruction. This approach enhances the precision of facial modeling by leveraging depth information and an average face model as a reference, reducing errors in alignment and improving the reliability of subsequent facial analysis tasks.
15. The computer readable storage medium according to claim 11 , wherein the first three-dimensional face database comprises an average face model and at least one of the following: a shape feature vector and an expression feature vector.
This invention relates to computer vision and facial recognition systems, specifically improving the accuracy of three-dimensional (3D) face modeling and recognition. The problem addressed is the variability in facial features due to shape and expression changes, which can degrade recognition performance. The solution involves a computer-readable storage medium storing a first 3D face database that includes an average face model, which serves as a reference template, along with additional data such as a shape feature vector and an expression feature vector. The shape feature vector captures geometric variations in facial structure, while the expression feature vector encodes dynamic changes in facial expressions. These components enable the system to normalize and compare facial data more effectively, improving recognition accuracy under varying conditions. The database may also include a second 3D face database for additional reference or comparison purposes. The system processes input facial data by aligning it with the average face model and extracting relevant features for matching against the stored vectors, enhancing robustness in real-world applications. This approach reduces errors caused by facial deformations and lighting variations, making it suitable for security, biometrics, and human-computer interaction systems.
16. The computer readable storage medium according to claim 11 , wherein the step of fitting the three-dimensional face model by the first three-dimensional face database according to the position of the face feature point in the first two-dimensional image and the depth map corresponding to the first two-dimensional image comprises: estimating an initial transformation matrix of a three-dimensional point cloud model corresponding to the depth map from the average face model; according to the initial transformation matrix and at least one of the following: the shape feature vector and the expression feature vector, and aiming at a first condition, fitting the three-dimensional face model from the average face model.
This invention relates to three-dimensional face modeling using depth maps and two-dimensional images. The problem addressed is accurately fitting a three-dimensional face model to a subject's face by leveraging both depth information and two-dimensional facial feature points. The solution involves a multi-step process that begins with estimating an initial transformation matrix for a three-dimensional point cloud model derived from a depth map. This initial matrix is based on an average face model. The system then refines the three-dimensional face model by applying adjustments based on shape and expression feature vectors, optimizing the fit to meet a predefined condition. The process ensures that the final three-dimensional model accurately represents the subject's facial geometry and expressions, improving applications in facial recognition, animation, and augmented reality. The method combines depth data with two-dimensional feature points to enhance modeling precision, addressing challenges in capturing fine facial details and dynamic expressions.
17. The computer readable storage medium according to claim 16 , wherein the first condition comprises at least one of the following: a distance between a projection position of a feature point of the three-dimensional face model in an image coordinate system and the position of the feature point of the first two-dimensional image corresponding to the feature point of the three-dimensional face model is smallest; and a distance between the three-dimensional face model and a point pair corresponding to the three-dimensional point cloud mapped by the depth map is smallest.
This invention relates to three-dimensional face modeling and alignment techniques, addressing challenges in accurately mapping and aligning a three-dimensional face model with two-dimensional images and depth maps. The technology focuses on optimizing the alignment process by evaluating specific conditions to determine the best match between a three-dimensional face model and input data. The system involves comparing a three-dimensional face model with a first two-dimensional image and a depth map to generate a three-dimensional point cloud. The alignment process evaluates two key conditions to determine the optimal match. The first condition checks whether the projection of a feature point from the three-dimensional model onto an image coordinate system is closest to the corresponding feature point in the first two-dimensional image. The second condition assesses whether the distance between the three-dimensional face model and a point pair derived from the depth map is minimized. By analyzing these conditions, the system ensures precise alignment of the three-dimensional model with the input data, improving accuracy in applications such as facial recognition, animation, or biometric analysis. The method enhances the reliability of three-dimensional face modeling by leveraging both two-dimensional and depth-based information for optimal alignment.
18. The computer readable storage medium according to claim 16 , wherein the step of estimating the initial transformation matrix of the three-dimensional point cloud model corresponding to the depth map by the average face model comprises: calculating a three-dimensional position of the face feature point according to the position of the face feature point in the first two-dimensional image and the depth map corresponding to the first two-dimensional image; and estimating the initial transformation matrix of the three-dimensional point cloud model corresponding to the depth map from the average face model according to the three-dimensional position of the face feature point and the three-dimensional position of the feature point of the average face model.
This invention relates to three-dimensional face modeling using depth maps and two-dimensional images. The problem addressed is accurately aligning a three-dimensional point cloud model of a face with an average face model to improve facial recognition or reconstruction. The solution involves estimating an initial transformation matrix that aligns the point cloud model with the average face model by leveraging depth information and feature point correspondence. The method calculates the three-dimensional positions of face feature points from a two-dimensional image and its corresponding depth map. These positions are then compared to the known three-dimensional positions of feature points in an average face model. By matching these points, the system estimates a transformation matrix that aligns the point cloud model with the average face model. This alignment improves the accuracy of subsequent facial modeling or recognition tasks by providing a more reliable initial pose estimation. The technique is particularly useful in applications requiring precise facial reconstruction, such as biometric authentication, augmented reality, or medical imaging, where accurate alignment between captured data and reference models is critical. The use of depth maps enhances the robustness of the alignment process by providing depth information that complements the two-dimensional image data.
19. The computer readable storage medium according to claim 17 , wherein the step of estimating the initial transformation matrix of the three-dimensional point cloud model corresponding to the depth map by the average face model comprises: calculating a three-dimensional position of the face feature point according to the position of the face feature point in the first two-dimensional image and the depth map corresponding to the first two-dimensional image; and estimating the initial transformation matrix of the three-dimensional point cloud model corresponding to the depth map from the average face model according to the three-dimensional position of the face feature point and the three-dimensional position of the feature point of the average face model.
This invention relates to three-dimensional face modeling using depth maps and two-dimensional images. The problem addressed is accurately aligning a three-dimensional point cloud model of a face with an average face model to improve facial recognition or reconstruction. The solution involves estimating an initial transformation matrix that maps the point cloud model to the average face model, enabling precise alignment. The process begins by calculating the three-dimensional positions of face feature points. These positions are derived from the coordinates of the feature points in a two-dimensional image and the corresponding depth map, which provides depth information. The depth map is used to convert the two-dimensional feature point positions into three-dimensional space. Next, the three-dimensional positions of the feature points in the point cloud model are compared to the three-dimensional positions of corresponding feature points in an average face model. Using this comparison, an initial transformation matrix is estimated. This matrix defines the spatial relationship between the point cloud model and the average face model, allowing the point cloud model to be accurately aligned with the average face model. The transformation matrix can then be refined for applications such as facial recognition, animation, or reconstruction.
20. The device for three-dimensional modeling according to claim 12 , wherein the first three-dimensional face database comprises an average face model and at least one of the following: a shape feature vector and an expression feature vector.
This invention relates to three-dimensional modeling, specifically for creating personalized 3D facial models. The technology addresses the challenge of accurately capturing and reconstructing facial features and expressions from limited input data, such as images or partial scans. The device includes a first three-dimensional face database containing an average face model, which serves as a reference template. The database also includes additional data such as a shape feature vector, which represents deviations in facial structure from the average model, and an expression feature vector, which captures dynamic facial expressions. These components allow the system to generate detailed 3D facial reconstructions by combining the average model with the feature vectors, enabling realistic and customizable 3D face modeling. The invention improves upon existing methods by providing a structured approach to incorporating both static and dynamic facial characteristics, enhancing the accuracy and flexibility of 3D face modeling applications. This technology is useful in fields such as animation, virtual reality, medical imaging, and biometric identification.
21. The device for three-dimensional modeling according to claim 20 , wherein the step of fitting the three-dimensional face model by the first three-dimensional face database according to the position of the face feature point in the first two-dimensional image and the depth map corresponding to the first two-dimensional image comprises: estimating an initial transformation matrix of a three-dimensional point cloud model corresponding to the depth map from the average face model; according to the initial transformation matrix and at least one of the following: the shape feature vector and the expression feature vector, and aiming at a first condition, fitting the three-dimensional face model from the average face model.
This invention relates to three-dimensional face modeling, specifically improving the accuracy of fitting a 3D face model to a 2D image and its corresponding depth map. The problem addressed is the challenge of precisely aligning a 3D face model with real-world facial data, which is crucial for applications like facial recognition, animation, and augmented reality. The device estimates an initial transformation matrix for a 3D point cloud model derived from the depth map, using an average face model as a reference. This initial alignment is refined by incorporating either a shape feature vector or an expression feature vector, or both, to optimize the fit. The fitting process targets a first condition, which likely refers to minimizing the difference between the projected 3D model and the 2D image features. The shape feature vector may encode facial structure variations, while the expression feature vector accounts for dynamic facial expressions. By leveraging these vectors, the system achieves a more accurate and personalized 3D face reconstruction. The method ensures that the final 3D model closely matches the input 2D image and depth data, improving realism and usability in various applications.
22. The device for three-dimensional modeling according to claim 21 , wherein the first condition comprises at least one of the following: a distance between a projection position of a feature point of the three-dimensional face model in an image coordinate system and the position of the feature point of the first two-dimensional image corresponding to the feature point of the three-dimensional face model is smallest; and a distance between the three-dimensional face model and a point pair corresponding to the three-dimensional point cloud mapped by the depth map is smallest. 18 . The computer readable storage medium according to claim 11 , storing a computer program, wherein the steps of the claim 5 is performed when the computer program is executed by a first processor.
This invention relates to three-dimensional (3D) face modeling, addressing challenges in accurately aligning 3D face models with two-dimensional (2D) images and depth maps. The technology improves the precision of 3D face reconstruction by optimizing the alignment process using feature points and depth data. The device for 3D modeling includes a mechanism to align a 3D face model with a 2D image and a depth map. The alignment is refined by evaluating two conditions: first, ensuring the projection of a feature point from the 3D model onto the 2D image is as close as possible to the corresponding feature point in the 2D image; second, minimizing the distance between the 3D model and point pairs derived from the depth map. This dual-condition approach enhances the accuracy of the 3D model by leveraging both geometric and depth-based constraints. Additionally, the invention includes a computer-readable storage medium containing a program that, when executed by a processor, performs the alignment steps. The program ensures that the 3D model is accurately mapped to the 2D image and depth data, improving the overall fidelity of the reconstructed 3D face. This method is particularly useful in applications requiring high-precision 3D face modeling, such as facial recognition, animation, and medical imaging.
23. The device for three-dimensional modeling according to claim 21 , wherein the step of estimating the initial transformation matrix of the three-dimensional point cloud model corresponding to the depth map by the average face model comprises: calculating a three-dimensional position of the face feature point according to the position of the face feature point in the first two-dimensional image and the depth map corresponding to the first two-dimensional image; and estimating the initial transformation matrix of the three-dimensional point cloud model corresponding to the depth map from the average face model according to the three-dimensional position of the face feature point and the three-dimensional position of the feature point of the average face model.
This invention relates to three-dimensional modeling, specifically improving the alignment of a three-dimensional point cloud model derived from a depth map with an average face model. The problem addressed is accurately estimating the initial transformation matrix that aligns the point cloud model with the average face model, which is crucial for applications like facial recognition, augmented reality, and medical imaging. The method involves calculating the three-dimensional positions of face feature points from a first two-dimensional image and its corresponding depth map. These feature points are then matched to the three-dimensional positions of corresponding feature points in an average face model. By comparing these positions, the system estimates an initial transformation matrix that optimally aligns the point cloud model with the average face model. This alignment ensures accurate representation and further processing of the three-dimensional facial structure. The technique leverages depth information to enhance the precision of feature point mapping, reducing errors in the transformation matrix estimation. This approach is particularly useful in scenarios where initial alignment is critical for subsequent modeling or analysis tasks. The method improves upon traditional techniques by incorporating depth data to refine the alignment process, leading to more accurate three-dimensional reconstructions.
24. The device for three-dimensional modeling according to claim 22 , wherein the step of estimating the initial transformation matrix of the three-dimensional point cloud model corresponding to the depth map by the average face model comprises: calculating a three-dimensional position of the face feature point according to the position of the face feature point in the first two-dimensional image and the depth map corresponding to the first two-dimensional image; and estimating the initial transformation matrix of the three-dimensional point cloud model corresponding to the depth map from the average face model according to the three-dimensional position of the face feature point and the three-dimensional position of the feature point of the average face model.
This invention relates to three-dimensional modeling, specifically improving the alignment of a three-dimensional point cloud model derived from a depth map with an average face model. The problem addressed is accurately estimating an initial transformation matrix to align the point cloud model with the average face model, which is essential for applications like facial recognition, animation, or medical imaging. The method involves calculating the three-dimensional positions of face feature points from a two-dimensional image and its corresponding depth map. These 3D positions are then compared with the known 3D positions of feature points in an average face model. By matching these points, the system estimates an initial transformation matrix that aligns the point cloud model with the average face model. This transformation accounts for differences in scale, rotation, and translation, ensuring accurate alignment for further processing or analysis. The technique leverages depth information to enhance the precision of 3D face modeling, reducing errors in alignment and improving the accuracy of subsequent applications. The method is particularly useful in scenarios where initial alignment is critical, such as in real-time facial tracking or medical imaging where precise 3D reconstructions are required.
Unknown
November 17, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.