Binocular Pose Prediction

1. A method for computer animation, comprising: generating a plurality of candidate pose sequences of two or more candidate poses of a three-dimensional model of an animation character such that each of the two or more candidate poses has a segmentation map that matches a segmentation map of a corresponding character derived from a corresponding frame of a video sequence at each time step of a plurality of time steps, wherein each time step corresponds to a frame of the video sequence, wherein a distance between candidate poses at each time step is maximized; determining an optimum pose sequence of the plurality of candidate pose sequences; and using the optimum pose sequence to generate a corresponding sequence of frames of animation of the animation character.

2. The method of claim 1 , wherein generating the plurality of candidate pose sequences includes generating two or more initial poses having corresponding segmentation masks that match the segmentation mask of the corresponding character derived from a first frame of the video sequence and adjusting each of the two or more initial poses over one or more time steps to generate two or more corresponding candidate pose sequences.

3. The method of claim 2 , wherein determining the optimum pose sequence includes simulating movement of the animation character with a physics simulation.

4. The method of claim 2 , wherein determining the optimum pose sequence includes simulating movement of the animation character with a physics simulation to determine whether a candidate sequence would violate a physical constraint.

5. The method of claim 2 , wherein determining the optimum pose sequence includes simulating movement of the animation character with a physics simulation to determine whether a candidate sequence would violate cause the animation character to fall.

6. The method of claim 1 , wherein determining the optimum pose sequence includes inputting each pose sequence of the plurality of candidate pose sequences into one or more Neural Networks.

7. The method of claim 6 , wherein each candidate pose of each candidate pose sequence of the plurality of candidate pose sequences is presented to the one or more Neural Networks in axis-angle form.

8. The method of claim 6 , wherein each candidate pose of each candidate pose sequence of the plurality of candidate pose sequences is presented to the one or more Neural Networks in axis-angle form and encoded into a latent representation using one or more layers of different numbers of neurons with activation functions.

9. The method of claim 6 , wherein each candidate pose of each candidate pose sequence of the plurality of candidate pose sequences is presented to the one or more Neural Networks in axis-angle form and encoded into a latent representation using two layers of different numbers of neurons with leaky ReLU activation functions.

10. The method of claim 6 , wherein each candidate pose of each candidate pose sequence of the plurality of candidate pose sequences is presented to the one or more Neural Networks in axis-angle form and encoded into a latent representation using two layers of 128 and 64 neurons with leaky ReLU activation functions.

11. The method of claim 6 , further comprising controlling a robot with an output of the one or more Neural Networks.

12. The method of claim 6 , further comprising controlling a robot with an output of the one or more Neural Networks, wherein the one or more Neural Networks include a policy network, wherein an output of the policy network specifies a set of joint position derivatives for one or more joints of the robot.

13. The method of claim 6 , further comprising controlling a robot with an output of the one or more Neural Networks, wherein the one or more Neural Networks include a policy network and a critic network, wherein an output of the policy network specifies a set of joint position derivatives for one or more joints of the robot, wherein the critic network produces an output representing a quality of random attempts to improve the performance of the policy network during a training phase.

14. The method of claim 1 , wherein generating the plurality of candidate pose sequences includes generating two or more candidate poses at each time step corresponding to each frame in the video sequence.

15. The method of claim 14 , wherein determining the optimum pose sequence includes feeding pairs of candidate poses for consecutive frames of the corresponding animation sequence into a Neural Network that has been pre-trained to control the animation character in a physics simulation environment.

16. The method of claim 15 , wherein determining the optimum pose sequence includes simulating movement of the animation character with a physics simulation.

17. The method of claim 15 , wherein determining the optimum pose sequence includes simulating movement of the animation character with a physics simulation to determine whether a candidate sequence would violate a physical constraint.

18. The method of claim 15 , wherein determining the optimum pose sequence includes simulating movement of the animation character with a physics simulation to determine whether a candidate sequence would violate cause the animation character to fall.

19. An apparatus for computer animation, comprising: a processor; a memory; executable instructions embodied in the memory that, when executed by the processor cause the processor to implement a method for computer animation, the method comprising, generating a plurality of candidate pose sequences of two or more candidate poses of a three-dimensional model of an animation character such that each of the two or more candidate poses has a segmentation map that matches a segmentation map of a corresponding character derived from a corresponding frame of a video sequence at each time step of a plurality of time steps, wherein each time step corresponds to a frame of the video sequence, wherein a distance between candidate poses at each time step is maximized; determining an optimum pose sequence of the plurality of candidate pose sequences; and using the optimum pose sequence to generate a corresponding sequence of frames of animation of the animation character.

20. The apparatus of claim 19 , wherein the executable instructions include one or more Neural Networks, wherein determining the optimum pose sequence includes inputting each pose sequence of the plurality of poses sequences into the one or more Neural Networks.

21. The apparatus of claim 19 , wherein each candidate pose of each candidate pose sequence of the plurality of candidate pose sequences is presented to the one or more Neural Networks in axis-angle form.

22. The apparatus of claim 19 , wherein each candidate pose of each candidate pose sequence of the plurality of candidate pose sequences is presented to the one or more Neural Networks in axis-angle form and encoded into a latent representation using one or more layers of different numbers of neurons with activation functions.

23. The apparatus of claim 19 , wherein each candidate pose of each candidate pose sequence of the plurality of candidate pose sequences is presented to the one or more Neural Networks in axis-angle form and encoded into a latent representation using two layers of different numbers of neurons with leaky ReLU activation functions.

24. The apparatus of claim 19 , wherein each candidate pose of each candidate pose sequence of the plurality of candidate pose sequences is presented to the one or more Neural Networks in axis-angle form and encoded into a latent representation using two layers of 128 and 64 neurons with leaky ReLU activation functions.

25. The apparatus of claim 19 , wherein generating the plurality of candidate pose sequences includes generating two or more initial poses having corresponding segmentation masks that match the segmentation mask of the corresponding character derived from a first frame of the video sequence and adjusting each of the two or more initial poses over one or more time steps to generate two or more corresponding candidate pose sequences.

26. The apparatus of claim 19 , wherein generating the plurality of candidate pose sequences includes generating two or more candidate poses at each time step corresponding to each frame in the video sequence.

27. The apparatus of claim 26 , wherein determining the optimum pose sequence includes feeding pairs of candidate poses for consecutive frames of the corresponding animation sequence into a Neural Network that has been pre-trained to control the animation character in a physics simulation environment.

28. A non-transitory computer readable medium having executable instructions embodied therein that, when executed by a computer cause the computer to implement a method for computer animation, the method comprising, generating a plurality of candidate pose sequences of two or more candidate poses of a three-dimensional model of an animation character such that each of the two or more candidate poses has a segmentation map that matches a segmentation map of a corresponding character derived from a corresponding frame of a video sequence at each time step of a plurality of time steps, wherein each time step corresponds to a frame of the video sequence, wherein a distance between candidate poses at each time step is maximized; determining an optimum pose sequence of the plurality of candidate pose sequences; and using the optimum pose sequence to generate a corresponding sequence of frames of animation of the animation character.