Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A neural network architecture for manipulating a facial image, the architecture comprising: a disentanglement portion that includes one or more first neural network layers, the disentanglement portion trained to disentangle at least one physical property captured in the facial image, the disentanglement portion receiving the facial image and outputting a disentangled representation of the facial image based on the at least one physical property; and a rendering portion that includes one or more second neural network layers, the rendering portion trained to perform a facial manipulation of the facial image based upon an image formation equation and the at least one physical property, thereby generating a manipulated facial image.
Image manipulation using neural networks. This invention addresses the problem of modifying facial images by disentangling and then re-rendering them based on physical properties. The system utilizes a neural network architecture comprising two main portions. The first portion, a disentanglement portion, consists of one or more neural network layers. This portion is trained to separate at least one physical characteristic, such as lighting or pose, from the original facial image. It takes the input facial image and produces a disentangled representation that isolates this specific physical property. The second portion is a rendering portion, also composed of one or more neural network layers. This portion is trained to perform facial manipulation. It uses an image formation equation and the disentangled physical property obtained from the first portion to generate a new, manipulated facial image. This allows for controlled modifications to the facial image based on specific physical attributes.
2. The neural network architecture of claim 1 , wherein the rendering portion operates on the disentangled representation of the facial image to generate the manipulated facial image.
This invention relates to neural network architectures for facial image manipulation, addressing the challenge of generating realistic and controllable modifications to facial images while preserving key facial features. The architecture includes a disentanglement module that processes an input facial image to separate it into a disentangled representation, isolating different facial attributes such as identity, expression, and lighting. A rendering portion then operates on this disentangled representation to generate a manipulated facial image, allowing for precise control over specific facial attributes while maintaining natural appearance. The disentanglement module ensures that modifications to one attribute (e.g., expression) do not unintentionally alter others (e.g., identity). The rendering portion applies learned transformations to the disentangled representation, producing a high-quality output image that reflects the desired manipulations. This approach enables applications in facial reenactment, expression transfer, and identity-preserving facial editing, where maintaining realism and attribute independence is critical. The architecture improves upon prior methods by explicitly separating facial attributes before manipulation, reducing artifacts and enhancing control over the final output.
3. The neural network architecture of claim 1 , wherein the rendering portion generates the manipulated facial image in response to receiving the disentangled representation of the facial image from the disentanglement portion.
This invention relates to a neural network architecture for manipulating facial images, specifically addressing the challenge of generating realistic and controllable facial modifications. The architecture consists of two primary components: a disentanglement portion and a rendering portion. The disentanglement portion processes an input facial image to extract a disentangled representation, separating facial attributes such as identity, expression, and lighting into independent components. This separation allows for precise control over individual attributes during manipulation. The rendering portion then uses this disentangled representation to generate a manipulated facial image, applying modifications to the desired attributes while preserving the remaining features. The system ensures that changes to one attribute do not unintentionally affect others, resulting in natural-looking modifications. The architecture is designed to handle various facial attributes, enabling applications in facial reenactment, expression transfer, and identity-preserving edits. The invention improves upon existing methods by providing a more robust and flexible approach to facial image manipulation, ensuring high-quality outputs with minimal artifacts.
4. The neural network architecture of claim 1 , wherein the disentanglement portion includes a neural network layer that encodes a map that transforms the facial image to an intermediate result associated with the at least one physical property.
This invention relates to neural network architectures for facial image processing, specifically addressing the challenge of disentangling and extracting specific physical properties from facial images. The architecture includes a disentanglement portion that processes facial images to isolate and encode physical properties such as facial structure, texture, or lighting conditions. A neural network layer within this portion generates a transformation map that converts the input facial image into an intermediate result, which represents the desired physical property in a structured format. This intermediate result can then be used for further analysis, modification, or synthesis tasks. The architecture ensures that the disentangled properties remain independent and can be manipulated separately, enabling applications in facial recognition, image editing, or synthetic image generation. The transformation map dynamically adjusts based on the input, allowing the system to adapt to variations in facial features or imaging conditions. This approach improves the accuracy and flexibility of facial property extraction compared to traditional methods that rely on fixed feature detectors or manual annotations. The invention is particularly useful in scenarios requiring precise control over facial attributes for tasks like identity verification, facial reenactment, or style transfer.
5. The neural network architecture of claim 1 , wherein the rendering portion includes a neural network layer arranged according to the image formation equation.
This invention relates to neural network architectures for image rendering, specifically addressing the challenge of efficiently generating high-quality images from input data. The architecture includes a rendering portion that incorporates a neural network layer structured according to the image formation equation, which mathematically models how light interacts with objects in a scene to produce an image. This layer processes input data, such as scene geometry and lighting parameters, to simulate the physical principles of image formation, improving realism and computational efficiency. The architecture may also include additional neural network components, such as layers for feature extraction or optimization, to enhance performance. By integrating the image formation equation into the neural network, the system achieves more accurate and physically plausible rendering compared to traditional methods, reducing artifacts and improving visual fidelity. The approach is particularly useful in applications like real-time rendering, virtual reality, and computer graphics, where both speed and quality are critical. The invention provides a novel way to combine neural networks with established optical principles to advance image synthesis technology.
6. The neural network architecture of claim 1 , wherein: each of a plurality of intermediate disentanglement results generated by the disentanglement portion is associated with a respective intermediate disentanglement loss function; each of the intermediate disentanglement results corresponds to one of the physical properties; each of a plurality of intermediate rendering results generated by the rendering portion is associated with a respective intermediate rendering loss function; the disentanglement portion is trained by (a) imposing each of the intermediate disentanglement loss functions upon its respective intermediate disentanglement result to generate a plurality of disentanglement weights, and (b) assigning the plurality of disentanglement weights in the disentanglement portion; and the rendering portion is trained by (a) imposing each of the intermediate rendering loss functions upon its respective intermediate rendering result to generate a plurality of rendering weights, and (b) assigning the plurality of rendering weights in the rendering portion.
A neural network architecture is designed for disentangling and rendering physical properties of objects in images. The system addresses challenges in separating and reconstructing distinct physical attributes, such as shape, texture, and lighting, from complex visual data. The architecture consists of a disentanglement portion and a rendering portion. The disentanglement portion processes input data to generate multiple intermediate disentanglement results, each corresponding to a specific physical property. Each of these results is associated with a dedicated intermediate disentanglement loss function, which guides the training process by optimizing the disentanglement weights assigned to the portion. Similarly, the rendering portion produces intermediate rendering results, each linked to an intermediate rendering loss function that refines the rendering weights. The training process involves applying these loss functions to their respective results, adjusting the weights in both portions to improve accuracy in disentangling and rendering. This structured approach ensures that the network effectively isolates and reconstructs physical properties while maintaining high-fidelity output. The system enhances applications in computer vision, image synthesis, and augmented reality by providing precise control over individual object attributes.
7. A computer program product including one or more non- transitory computer readable mediums encoded with instructions that when executed by one or more processors cause operations of a neural network architecture to be carried out, the operations comprising: receiving, by a disentanglement portion of the neural network architecture, a facial image; disentangling, by the disentanglement portion, at least one physical property captured in the facial image; outputting, by the disentanglement portion, a disentangled representation of the facial image based on the at least one physical property; receiving the disentangled representation of the facial image by a rendering portion of the neural network architecture, wherein the rendering portion is trained to perform a facial manipulation of the facial image based upon an image formation equation and the at least one physical property; and generating, by the rendering portion, a manipulated facial image.
This invention relates to a neural network architecture for facial image manipulation, addressing the challenge of accurately modifying facial images while preserving natural appearance. The system uses a disentanglement portion to analyze a facial image, extracting physical properties such as lighting, texture, or facial structure. This portion outputs a disentangled representation that isolates these properties. The rendering portion then processes this representation, applying an image formation equation to manipulate the facial image based on the extracted properties. The rendering portion is trained to perform specific facial manipulations, such as altering expressions or lighting conditions, while maintaining realism. The result is a manipulated facial image that retains natural characteristics. The architecture ensures that modifications are physically plausible by leveraging the disentangled properties, improving over traditional methods that may produce unnatural artifacts. This approach is useful in applications like virtual try-ons, facial recognition, and digital content creation.
8. The computer program product of claim 7 , wherein the rendering portion generates the manipulated facial image in response to receiving the disentangled representation of the facial image from the disentanglement portion.
This invention relates to computer vision and image processing, specifically to systems that manipulate facial images while preserving key facial attributes. The problem addressed is the difficulty in modifying facial images without unintentionally altering important features like identity, expression, or lighting. The solution involves a computer program product that processes facial images through a disentanglement portion and a rendering portion. The disentanglement portion analyzes an input facial image to separate its features into distinct, independent representations, such as identity, expression, and lighting. The rendering portion then uses these disentangled representations to generate a manipulated facial image. The manipulation can include changes to expression, lighting, or other attributes while maintaining the original identity or other desired features. The system ensures that modifications are applied precisely to the intended features without unintended side effects. This approach improves the accuracy and control of facial image editing, making it useful in applications like virtual reality, animation, and identity verification.
9. The computer program product of claim 7 , wherein the rendering portion operates on the disentangled representation of the facial image to generated the manipulated facial image.
This invention relates to computer vision and facial image manipulation, specifically addressing the challenge of generating realistic and controllable facial images from input data. The system involves a disentangled representation of facial images, which separates facial attributes (such as identity, expression, and lighting) into independent components. This allows for precise manipulation of specific attributes while preserving others. The invention includes a rendering portion that processes the disentangled representation to produce a manipulated facial image, enabling controlled modifications like changing expressions or lighting conditions without altering the underlying identity. The disentangled representation is derived from an input facial image, which is decomposed into multiple latent factors representing different facial attributes. These factors are then selectively adjusted to achieve the desired manipulation. The rendering portion synthesizes the modified latent factors into a coherent and realistic facial image. This approach improves upon traditional methods by providing finer control over facial attributes and reducing artifacts in the generated output. The invention is particularly useful in applications like virtual avatars, facial recognition, and digital content creation, where accurate and flexible facial image manipulation is required.
10. The computer program product of claim 7 , wherein the disentanglement portion includes a layer that encodes a map that transforms the facial image to an intermediate result associated with the at least one physical property.
This invention relates to computer vision and machine learning techniques for analyzing facial images. The technology addresses the challenge of disentangling and extracting specific physical properties from facial images, such as facial expressions, lighting conditions, or identity features, to enable more accurate and interpretable facial analysis. The system uses a disentanglement portion within a neural network to process facial images, where this portion includes a specialized layer that encodes a transformation map. This map converts the input facial image into an intermediate result that isolates and represents at least one physical property of the face, such as expression or lighting. The disentanglement process allows for controlled manipulation of these properties while preserving the remaining facial features. This approach improves the accuracy of facial recognition, expression analysis, and other applications by separating and analyzing individual physical properties independently. The system may be used in applications like emotion recognition, facial reconstruction, or identity verification, where isolating specific facial attributes is critical. The transformation map ensures that the disentanglement process is both efficient and effective, enabling real-time or near-real-time processing of facial images.
11. The computer program product of claim 7 , wherein: each of a plurality of intermediate disentanglement results generated by the disentanglement portion is associated with a respective intermediate disentanglement loss function; and each of the intermediate disentanglement results corresponds to one of the physical properties.
This invention relates to a computer program product for disentangling physical properties in data, particularly in machine learning applications. The problem addressed is the challenge of separating and identifying distinct physical properties within complex data representations, such as those generated by neural networks. Traditional approaches often struggle to isolate individual properties, leading to entangled or overlapping representations that hinder interpretability and control. The invention involves a disentanglement system that processes input data through a disentanglement portion, which generates multiple intermediate disentanglement results. Each result corresponds to a specific physical property, such as shape, texture, or color, and is associated with a dedicated intermediate disentanglement loss function. These loss functions guide the disentanglement process by penalizing deviations from desired property separations, ensuring that each property is independently represented. The system iteratively refines these results to achieve a fully disentangled output, where each physical property is isolated and can be manipulated independently. This approach improves interpretability, allows for targeted modifications, and enhances the usability of machine learning models in applications like image generation, data analysis, and simulation. The invention is particularly useful in domains requiring precise control over physical attributes, such as computer vision, robotics, and generative modeling.
12. The computer program product of claim 7 , wherein: each of a plurality of intermediate disentanglement results generated by the disentanglement portion is associated with a respective intermediate disentanglement loss function; each of the intermediate disentanglement results corresponds to one of the physical properties; and each of a plurality of intermediate rendering results generated by the rendering portion is associated with a respective intermediate rendering loss function.
This invention relates to a computer program product for processing data, particularly in the domain of disentanglement and rendering of physical properties from input data. The technology addresses the challenge of accurately separating and reconstructing physical properties from complex data inputs, such as images or sensor data, to enable more precise analysis or synthesis. The system includes a disentanglement portion that processes input data to generate intermediate disentanglement results, each corresponding to a distinct physical property. Each intermediate result is evaluated using a respective intermediate disentanglement loss function, which measures the accuracy or quality of the disentanglement process. Similarly, a rendering portion generates intermediate rendering results, each associated with a respective intermediate rendering loss function, to assess the fidelity of the rendered output. The disentanglement and rendering processes are designed to work in tandem, where the disentanglement portion isolates specific physical properties (e.g., texture, shape, or lighting) from the input data, and the rendering portion reconstructs these properties into a coherent output. The use of intermediate loss functions at each stage ensures that the system can iteratively refine its performance, improving the accuracy of both disentanglement and rendering. This approach is particularly useful in applications requiring high-fidelity data reconstruction, such as computer vision, generative modeling, or sensor-based analysis, where separating and accurately representing physical properties is critical. The system's modular design allows for flexibility in handling different types of input data and physical properties, making it adaptable to various real-world scenari
13. The computer program product of claim 7 , wherein: each of a plurality of intermediate disentanglement results generated by the disentanglement portion is associated with a respective intermediate disentanglement loss function; each of the intermediate disentanglement results corresponds to one of the physical properties; the operations further comprise imposing each of the intermediate disentanglement loss functions upon its respective intermediate disentanglement result to generate a plurality of disentanglement weights; and the operations further comprise assigning the plurality of disentanglement weights in the disentanglement portion of the neural network.
This invention relates to a neural network system for disentangling physical properties from input data, addressing the challenge of separating and identifying distinct physical characteristics in complex datasets. The system includes a disentanglement portion of a neural network that processes input data to generate multiple intermediate disentanglement results, each corresponding to a specific physical property. Each intermediate result is evaluated using a respective intermediate disentanglement loss function, which quantifies how well the disentanglement process isolates the target property. The system then generates disentanglement weights based on these loss functions and assigns them to the disentanglement portion of the neural network. These weights refine the network's ability to accurately separate and represent the physical properties in subsequent processing stages. The approach ensures that each physical property is independently and effectively disentangled, improving the network's performance in tasks requiring precise property identification or manipulation. The system is particularly useful in applications where input data contains overlapping or interdependent physical characteristics, such as in image processing, sensor data analysis, or scientific modeling.
14. The computer program product of claim 7 , wherein each of a plurality of intermediate rendering results generated by the rendering portion is associated with a respective intermediate rendering loss function.
This invention relates to computer graphics rendering, specifically improving the efficiency and accuracy of rendering processes by using intermediate loss functions. The problem addressed is the computational inefficiency and potential inaccuracies in traditional rendering pipelines, particularly when dealing with complex scenes or real-time rendering requirements. The solution involves generating multiple intermediate rendering results during the rendering process, where each intermediate result is associated with a distinct intermediate loss function. These loss functions quantify the deviation or error between the intermediate result and a target or reference rendering. By evaluating these intermediate loss functions, the system can optimize rendering parameters, adjust computational resources, or refine subsequent rendering steps to improve final output quality while reducing unnecessary computations. The approach allows for adaptive rendering, where the system dynamically balances quality and performance based on real-time feedback from the intermediate loss functions. This method is particularly useful in applications requiring high-quality visual output with constrained computational resources, such as real-time graphics rendering, virtual reality, or augmented reality systems. The invention enhances rendering efficiency by avoiding redundant calculations and ensuring that intermediate steps contribute meaningfully to the final result.
15. The computer program product of claim 7 , wherein: each of a plurality of intermediate rendering results generated by the rendering portion is associated with a respective intermediate rendering loss function; the operations further comprise imposing each of the intermediate rendering loss functions upon its respective intermediate rendering result to generate a plurality of rendering weights; and the operations further comprise assigning the plurality of rendering weights in the rendering portion of the neural network.
This invention relates to neural network-based rendering systems, specifically improving the training and performance of neural networks used for rendering tasks. The problem addressed is the challenge of optimizing intermediate rendering steps in neural networks to enhance the quality and efficiency of the final rendered output. Traditional neural rendering methods often struggle with balancing intermediate computations, leading to suboptimal results or inefficient training. The invention describes a computer program product that enhances a neural network's rendering portion by incorporating intermediate rendering loss functions. During rendering, the neural network generates multiple intermediate rendering results, each associated with a distinct intermediate rendering loss function. These loss functions evaluate the quality or accuracy of each intermediate result. The system then applies these loss functions to their respective intermediate results to compute a set of rendering weights. These weights are dynamically assigned within the rendering portion of the neural network, adjusting the influence of each intermediate result on the final output. This adaptive weighting mechanism improves the network's ability to refine intermediate steps, leading to higher-quality final renders and more efficient training. The approach is particularly useful in applications like real-time rendering, virtual reality, or any scenario requiring high-fidelity visual output from neural networks.
16. A method for manipulating a facial image, the method comprising: receiving, by a disentanglement portion of a neural network architecture, a facial image; associating a respective intermediate loss function with each of a plurality of intermediate results generated by the disentanglement portion, wherein each of the intermediate results corresponds to a property of the facial image; providing the intermediate results to a rendering portion of the neural network architecture, the rendering portion arranged according to an image formation equation; imposing each of the intermediate loss functions upon its respective intermediate result to generate a plurality of weights; assigning the generated weights in the disentanglement portion; disentangling, by the disentanglement portion, the facial image into one or more disentangled facial image properties; and generating, by the rendering portion, a manipulated facial image.
This invention relates to neural network-based facial image manipulation, addressing the challenge of accurately modifying specific facial properties while preserving others. The method uses a neural network architecture divided into a disentanglement portion and a rendering portion. The disentanglement portion processes an input facial image, generating intermediate results corresponding to distinct facial properties (e.g., expression, identity, lighting). Each intermediate result is associated with a unique intermediate loss function, which evaluates its accuracy or relevance. These intermediate results are then passed to the rendering portion, structured according to an image formation equation, which mathematically defines how facial properties contribute to the final image. The intermediate loss functions are applied to their respective results to generate weights, which are assigned back to the disentanglement portion. This weighted adjustment refines the disentanglement process, ensuring precise separation of facial properties. The disentangled properties are then used by the rendering portion to generate a manipulated facial image, where specific attributes (e.g., age, expression) can be altered while maintaining realism. The system enables controlled, high-quality facial image modifications by leveraging loss-driven optimization and structured neural network design.
17. The method of claim 16 , wherein the rendering portion receives the one or more disentangled facial image properties from the disentanglement portion.
A system and method for facial image processing involves disentangling and reconstructing facial images to modify specific attributes while preserving others. The technology addresses challenges in facial image editing, such as maintaining natural appearance and avoiding artifacts when altering features like expression, identity, or lighting. The method includes a disentanglement portion that separates a facial image into distinct, editable properties, such as identity, expression, and lighting. These properties are then passed to a rendering portion, which reconstructs the facial image by combining the modified properties. The rendering portion ensures that changes to one property do not unintentionally affect others, allowing for precise and independent adjustments. This approach enables applications in facial recognition, animation, and image editing by providing controlled modifications while maintaining visual coherence. The system improves upon traditional methods by reducing distortion and improving the realism of edited facial images.
18. The method of claim 16 , wherein the rendering portion generates the manipulated facial image in response to receiving the one or more disentangled facial image properties from the disentanglement portion.
This invention relates to facial image manipulation, specifically a system that processes facial images to extract and modify specific facial features independently. The method involves a disentanglement portion that analyzes an input facial image to separate and extract distinct facial properties, such as facial structure, expression, or texture. These properties are then passed to a rendering portion, which generates a manipulated facial image by applying modifications to the extracted properties. The rendering portion operates in response to receiving the disentangled properties, ensuring that changes to one facial feature do not unintentionally affect others. This approach allows for precise control over facial image editing, enabling applications in digital media, virtual avatars, and identity protection. The system avoids the limitations of traditional methods that struggle to isolate and modify individual facial attributes without altering unrelated features. By decoupling facial properties, the invention enables more natural and accurate facial image manipulation, improving the quality and realism of edited images.
19. The method of claim 16 , wherein the rendering portion operates on the one or more disentangled facial image properties to generate the manipulated facial image.
This invention relates to facial image manipulation using disentangled facial image properties. The problem addressed is the difficulty of precisely controlling specific facial features in an image while maintaining natural appearance. Traditional methods often struggle to isolate and modify individual attributes without unintended distortions. The method involves a system that processes an input facial image to extract disentangled properties, which are distinct, independent characteristics of the face such as identity, expression, or lighting. These properties are then modified or manipulated according to user-defined parameters. A rendering portion of the system operates on the modified disentangled properties to generate a manipulated facial image that reflects the desired changes while preserving realism. The system may include a disentanglement module that separates the input image into multiple disentangled properties, ensuring that each property can be independently adjusted. A manipulation module allows for selective modification of these properties, such as altering facial expressions, changing lighting conditions, or adjusting identity features. The rendering portion synthesizes the modified properties into a coherent output image, ensuring that the manipulated features appear natural and consistent with the original image. This approach enables precise control over facial attributes while maintaining the integrity of the remaining features, making it useful in applications like virtual avatars, facial recognition, and digital content creation.
20. The method of claim 16 , wherein the generated weights are assigned in the disentanglement and rendering portions.
A method for improving the generation and rendering of digital content, particularly in applications involving disentanglement and rendering of visual or audio data. The method addresses the challenge of efficiently separating and reconstructing different components of digital media, such as background, foreground, and other features, to enhance realism and flexibility in rendering. The method involves generating weights that are specifically assigned to different portions of the process, including disentanglement and rendering. Disentanglement refers to the separation of distinct elements within the data, such as isolating objects from their backgrounds or decomposing audio into different sound sources. Rendering involves reconstructing or synthesizing the data for output, ensuring that the disentangled components are accurately recombined. By assigning weights to these portions, the method optimizes the balance between disentanglement accuracy and rendering quality, improving the overall performance of digital content generation systems. The method is applicable in fields such as computer vision, augmented reality, virtual reality, and multimedia processing, where precise control over digital content is essential. The approach ensures that the generated weights enhance both the separation and reconstruction phases, leading to more realistic and adaptable digital outputs.
Unknown
June 23, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.