Described herein is a technique for creating a 3D face model using images obtained from an inexpensive camera associated with a general-purpose computer. Two still images of the user are captured, and two video sequences. The user is asked to identify five facial features, which are used to calculate a mask and to perform fitting operations. Based on a comparison of the still images, deformation vectors are applied to a neutral face model to create the 3D model. The video sequences are used to create a texture map. The process of creating the texture map references the previously obtained 3D model to determine poses of the sequential video images.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A method to create a three-dimensional model of a face, comprising: constructing a three-dimensional face model from two or more two-dimensional face images, wherein the constructed face model comprises a plurality of points specified in three dimensions; applying a combination of deformation vectors to a pre-specified face model to deform the pre-specified face model approximately to the constructed face model, wherein the pre-specified face model comprises a plurality points specified in three dimensions and each deformation vector specifies a plurality of displacements corresponding respectively to the plurality of points of the pre-specified face model; wherein the applying comprises calculating a linear combination of the deformation vectors and the pre-specified face model; multiplying each deformation vector by a respective multiplicative constant before when calculating the linear combination; and selecting the multiplicative constants to minimize a summation of distances between the deformed face model and the points of the constructed face model.
2. A method as recited in claim 1 , further comprising: identifying one or more points of the constructed face model and corresponding points of the pre-specified face model; and further selecting the multiplicative constants to minimize a summation of the distances between the identified points of the constructed face model and the corresponding points of the pre-specified face model.
3. One or more computer-readable media containing a program that is executable by a computer to create a three-dimensional model of a face, the program causing the computer to perform the following action, comprising: applying a combination of deformation vectors to a pre-specified face model to deform the pre-specified face model approximately to a reconstructed face model, wherein the pre-specified face model comprises a plurality of points specified in three dimensions and each deformation vector specifies a plurality of displacements corresponding respectively to the plurality of three dimensional points of the pre-specified face model; wherein the applying comprises calculating a linear combination of the deformation vectors and the pre-specified face model; multiplying each deformation vector by a respective multiplicative constant before when calculating the linear combination; and selecting the multiplicative constants to minimize a summation of distances between the deformed face model and the points of the constructed face model.
4. One or more computer-readable media as recited in claim 3 , the actions further comprising: identifying one or more points of the constructed face model and corresponding points of the pre-specified face model; and further selecting the multiplicative constants to minimize a summation of distances between the identified points of the constructed face model and the corresponding points of the pre-specified face model.
5. One or more computer-readable media containing a program that is executable by a computer to create a three-dimensional model of a face, the program causing the computer to perform the following action, comprising: specifying a neutral face model in terms of three dimensional points; specifying a plurality of deformation vectors, wherein each deformation vector indicates a plurality of displacements corresponding respectively to the points of the neutral face model, and wherein each deformation vector deforms the neutral face model in a particular way when applied to the neutral face model; specifying degree of deformation for each deformation vector in terms of a multiplicative constant; deforming the neutral face model by calculating a linear combination of the neutral face model and the deformation vectors, each deformation vector being multiplied by its multiplicative constant; and selecting the multiplicative constants to minimize a summation of distances between the deformed face model and the points of the constructed face model.
6. One or more computer-readable media as recited in claim 5 , the actions further comprising: identifying one or more points of the constructed face model and corresponding points of the pre-specified face model; and further selecting the multiplicative constants to minimize a summation of distances between the identified points of the constructed face model and the corresponding points of the pre-specified face model.
7. One or more computer-readable media as recited in claim 5 , the actions further comprising: accepting input from a human user identifying one or more points of the constructed face model that correspond to particular points of the pre-specified face model; and further selecting the multiplicative constants to minimize a summation of distances between the identified points of the constructed face model and the corresponding points of the pre-specified face model.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
February 1, 2005
July 25, 2006
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.