The disclosed artificial reality system can provide a user self representation in an artificial reality environment based on a self portion from an image of the user. The artificial reality system can generate the self representation by applying a machine learning model to classify the self portion of the image. The machine learning model can be trained to identify self portions in images based on a set of training images, with portions tagged as either depicting a user from a self-perspective or not. The artificial reality system can display the self portion as a self representation in the artificial reality environment by positioning them in the artificial reality environment relative to the user's perspective in the artificial reality environment. The artificial reality system can also identify movements of the user and can adjust the self representation to match the user's movement, providing more accurate self representations.
Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
5. The method of claim 1, wherein the machine learning model is trained using a set of images with portions of each image tagged to indicate whether that portion depicts a self portion of a user or not.
6. The method of claim 1, wherein classifying the self portion in each of the multiple images includes using the machine learning model to classify each of multiple individual pixels of each image as either depicting or not depicting a part of the user.
7. The method of claim 1, wherein classifying the self portion in each specific image of the multiple images includes classifying parts of the specific image as depicting particular body parts.
15. The computer-readable storage medium of claim 12, wherein the machine learning model is a deep neural network trained using a set of images with portions of each image tagged to indicate whether that portion depicts a part of a user or not.
17. The computer-readable storage medium of claim 12, wherein displaying the self portion of the at least one of the part of the multiple images includes overwriting a portion of a frame buffer, written to by an application controlling part of the artificial reality environment, with data for the self portion of the at least one of the part of the multiple images.
20. The computing system of claim 18, wherein displaying the self portion of the at least one of the part of the multiple images includes overwriting a portion of a frame buffer with data for the self portion of the at least one of the part of the multipleimages.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
January 3, 2020
October 18, 2022
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.