Iterative Motion Segmentation

1. An image processing method of simultaneously extracting a background image, at least two object images, a shape of each object image and motion of each object image, which are defined as hidden parameters, from among plural images, said image processing method comprising: an image input step of accepting input of plural images arranged in time series; a hidden parameter estimation step of estimating a hidden parameter based on the plural images and a constraint enforcement parameter, wherein the constraint enforcement parameter indicates a condition of at least one of the hidden parameters, using an iterative learning method; a constraint enforcement parameter learning step of learning a constraint enforcement parameter related to the hidden parameter using an estimation result from said hidden parameter estimation step as a training signal; a complementary learning step of causing the estimation of the hidden parameter and the learning of the constraint enforcement parameter to be iterated, the estimation of the hidden parameters being performed in said hidden parameter estimation step, which uses a learning result given in said constraint enforcement parameter learning step, and the learning of the constraint enforcement parameter being performed in said constraint enforcement parameter learning step, which uses the estimation result of the hidden parameter given in said hidden parameter estimation step; an output step of outputting the hidden parameter estimated in said hidden parameter estimation step after the iterative learning is performed in said complementary learning step; and an intermediate time image synthesis step of: receiving the background image, the object images, the shapes of the object images, and the motion of the object images, which are hidden parameters outputted in said output step, synthesizing, for each of at least two objects, an object image within an intermediate time between input images using the background image, the object images, the shapes of the object images and the motion of the object images, and synthesizing, for the object image synthesized for each of the at least two objects, an intermediate time image by superimposing the object image at the intermediate time onto the background image at the corresponding time.

2. The image processing method according to claim 1 , wherein the constraint enforcement parameter is a parameter related to at least one of the following conditions: an area size for each object image, a color of each object image, and a pixel motion that is included in each object image.

3. The image processing method according to claim 1 , wherein, in said constraint enforcement parameter learning step, adjacent pixels are chosen spatially from the plural images to become a pair and the respective pixels are assessed to belong to either an image area of the background image or an image area of each object, and different respective constraints are applied to the hidden parameter depending on whether the respective pixels belong to different objects or the respective pixels belong to the same object.

4. The image processing method according to claim 1 , wherein said constraint enforcement parameter learning step includes learning the constraint enforcement parameter based on an energy minimization method using the estimation result from said hidden parameter estimation step as a training signal.

5. The image processing method according to claim 1 , wherein each of the hidden parameters is expressed by a probability distribution.

6. The image processing method according to claim 1 , wherein a background image motion, which is caused by camera motion, is included as one of the hidden parameters.

7. An image processing device adapted to simultaneously extract a background image, at least two object images, a shape of each object image and motion of each object image, which are defined as hidden parameters, from among plural images, said image processing device comprising: an image input unit operable to accept input of plural images arranged in time series; a hidden parameter estimation unit operable to estimate a hidden parameter based on the plural images and a constraint enforcement parameter, wherein the constraint enforcement parameter indicates a condition of at least one of the hidden parameters, using an iterative learning method; a constraint enforcement parameter learning unit operable to learn a constraint enforcement parameter related to the hidden parameter using an estimation result from said hidden parameter estimation unit as a training signal; a complementary learning unit operable to cause the estimation of the hidden parameter and the learning of the constraint enforcement parameter to be iterated, the estimation of the hidden parameters being performed by said hidden parameter estimation unit, which uses a learning result given by said constraint enforcement parameter learning unit, and the learning of the constraint enforcement parameter being performed by said constraint enforcement parameter learning unit, which uses the estimation result of the hidden parameter given by said hidden parameter estimation unit; an output unit arranged to output the hidden parameter estimated by said hidden parameter estimation unit after the iterative learning is performed by said complementary learning unit; and an intermediate time image synthesis unit operable to: receive the background image, the object images, the shapes of the object images, and the motion of the object images, which are hidden parameters outputted from said output unit, synthesize, for each of at least two objects, an object image within an intermediate time between input images using the background image, the object images, the shapes of the object images and the motion of the object images, and synthesize, for the object image synthesized for each of the at least two objects, an intermediate time image by superimposing the object image at the intermediate time onto the background image at the corresponding time, wherein at least one of (i) said image input unit, (ii) said hidden parameter estimation unit, (iii) said constraint enforcement parameter learning unit, (iv) said complementary learning unit, (v) said output unit, and (vi) said intermediate time image synthesis unit, is implemented by a processor.

8. A non-transitory computer-readable medium having stored thereon an image processing program for simultaneously extracting a background image, at least two object images, a shape of each object image and motion of each object image, which are defined as hidden parameters, from among plural images arranged in time series, said image processing program causing a computer to execute: an image input step of accepting input of plural images; a hidden parameter estimation step of estimating a hidden parameter based on the plural images and a constraint enforcement parameter, wherein the constraint enforcement parameter indicates a condition of at least one of the hidden parameters, using and iterative learning method; a constraint enforcement parameter learning step of learning a constraint enforcement parameter related to the hidden parameter using an estimation result from said hidden parameter estimation step as a training signal; a complementary learning step of causing the estimation of the hidden parameter and the learning of the constraint enforcement parameter to be iterated, the estimation of the hidden parameters being performed in said hidden parameter estimation step, which uses a learning result given in said constraint enforcement parameter learning step, and the learning of the constraint enforcement parameter being performed in said constraint enforcement parameter learning step, which uses the estimation result of the hidden parameter given in said hidden parameter estimation step; an output step of outputting the hidden parameter estimated in said hidden parameter estimation step after the iterative learning is performed in said complementary learning step; and an intermediate time image synthesis step of: receiving the background image, the object images, the shapes of the object images, and the motion of the object images, which are hidden parameters outputted in said output step, synthesizing, for each of at least two objects, an object image within an intermediate time between input images using the background image, the object images, the shapes of the object images and the motion of the object images, and synthesizing, for the object image synthesized for each of the at least two objects, an intermediate time image by superimposing the object image at the intermediate time onto the background image at the corresponding time.