Displacement-Oriented View Synthesis System and Method

1. A displacement-oriented view synthesis system, comprising: a plurality of three-dimensional (3D) warping devices each coupled to receive at least one input image captured from a corresponding reference view, and each performing 3D warping on the input image to generate at least one corresponding warped image in a target view; a view blending device coupled to receive the warped images, and performing view blending on the warped images to generate at least one blended image in the target view; and an inpainting device coupled to receive the blended image, and performing inpainting on the blended image to generate a synthesized image in the target view; wherein the inpainting is performed according to a difference displacement between frames of different views; wherein the difference displacement is obtained by the following steps: determining a foreground displacement associated with a foreground object, the foreground displacement being constructed by connecting a foreground-associated point in the target view with a corresponding point in the reference view; determining a background displacement associated with a background object, the background displacement being constructed by connecting a background-associated point in the target view with the corresponding point in the reference view; and obtaining the difference displacement by subtracting the background displacement from the foreground displacement.

2. The system of claim 1 , wherein the 3D warping device adopts backward warping that maps from the warped image to the input image.

3. The system of claim 1 , wherein the view blending is performed by a weighted average or winner-take-all technique.

4. The system of claim 1 , wherein holes in an area between a foreground edge and a background edge are filled by pixels or patches searched along the difference displacement.

5. The system of claim 1 , wherein the at least one input image comprises an input texture image and an associated input depth image.

6. The system of claim 1 , further comprising: a plurality of cameras each capturing the at least one input image from the corresponding reference view.

7. A displacement-oriented view synthesis method, comprising: receiving a plurality of input images captured from corresponding reference views respectively; performing three-dimensional (3D) warping on the input images to respectively generate corresponding warped images in a target view; performing view blending on the warped images to generate at least one blended image in the target view; and performing inpainting on the blended image to generate a synthesized image in the target view; wherein the inpainting is performed according to a difference displacement between frames of different views; wherein the difference displacement is obtained by the following steps: determining a foreground displacement associated with a foreground object, the foreground displacement being constructed by connecting a foreground-associated point in the target view with a corresponding point in the reference view; determining a background displacement associated with a background object, the background displacement being constructed by connecting a background-associated point in the target view with the corresponding point in the reference view; and obtaining the difference displacement by subtracting the background displacement from the foreground displacement.

8. The method of claim 7 , wherein the step of performing the 3D warping adopts backward warping that maps from the warped image to the input image.

9. The method of claim 7 , wherein the view blending is performed by a weighted average or winner-take-all technique.

10. The method of claim 7 , wherein holes in an area between a foreground edge and a background edge are filled by pixels or patches searched along the difference displacement.

11. The method of claim 7 , wherein the input image comprises an input texture image and an associated input depth image.

12. The method of claim 7 , further comprising: providing a plurality of cameras for capturing the plurality of input images from the corresponding reference views respectively.