Device and Method of Generating Multi-View Immersive Content

1. A method of generating multi-view immersive content, the method comprising: obtaining a multi-view background image from a plurality of cameras arranged in a curved shape; modeling the obtained multi-view background image to generate a codebook corresponding to the multi-view background image; obtaining a multi-view image including an object from the plurality of cameras and separating a foreground and a background from the obtained multi-view image by using the generated codebook; and synthesizing the object included in the separated foreground with a virtual background to generate multi-view immersive content, wherein the separating of the foreground and the background comprises: converting an RGB color value of the multi-view image into an HSV color value expressed in an HSV color space; calculating a hue difference between a first HSV color value of the multi-view background image and a second HSV color value of the multi-view image which are stored in the codebook; and comparing the calculated hue difference with a threshold value to separate the foreground and the background from the multi-view image, and wherein the calculating of the hue difference comprises calculating an area of a triangle formed between a first hue value of the first HSV color value and a second hue value of the second HSV color value on an RGB-HSV spectrum graph that includes an abscissa axis indicating a hue and an ordinate axis indicating intensity.

2. The method of claim 1 , wherein the obtaining of the multi-view background image comprises obtaining the multi-view background image from the plurality of cameras, arranged on a curved-shape rail that forms a certain radius with respect to a convergence point at which the object is located.

4. The method of claim 1 , wherein the separating of the foreground and the background comprises: converting an RGB vector of each of pixels of the multi-view image into an HSI vector in a cone-shaped HSI color space with intensity as a center axis; calculating an intensity boundary value by applying a weight variable to an intensity value of an arbitrary pixel in the HSI color space; and comparing the calculated intensity boundary value with a threshold value stored in the codebook to separate the foreground and the background from the multi-view image.

5. The method of claim 4 , wherein the calculating of the intensity boundary value comprises calculating the intensity boundary value according to Equation (6): Intensity ⁢ ⁢ boundary = [ max ⁢ { α ⁢ ⁢ I high , τ black } , max ⁢ { min ⁢ { β ⁢ ⁢ I high , I low α } , τ black } ] ( 6 ) where I high denotes a highest intensity value among intensity values of pixels which are collected in units of one frame, I low denotes a lowest intensity value among the intensity values of the pixels which are collected in units of one frame, α and β(α<β) each denote the weight variable determined based on a user input, and τ black denotes a minimum threshold value of the intensity boundary value.

6. A method of generating multi-view immersive content, the method comprising: obtaining a multi-view background image from a plurality of cameras arranged in a curved shape; modeling the obtained multi-view background image to generate a codebook corresponding to the multi-view background image; obtaining a multi-view image including an object from the plurality of cameras and separating a foreground and a background from the obtained multi-view image by using, the generated codebook; and synthesizing the object included in the separated foreground with a virtual background to generate multi-view immersive content, wherein the separating of the foreground and the background comprises: converting an RGB color value of each of pixels of the multi-view image into an HSI color value in a cone-shaped HSI color space with intensity as a center axis; calculating a corrected saturation value by dividing a saturation value of an arbitrary pixel in the HSI color space by a maximum saturation value expressible in the HSI color space; and comparing the corrected saturation value with the saturation value of the arbitrary pixel stored in the codebook to separate the foreground and the background from the multi-view image.

7. The method of claim 6 , wherein the calculating of the corrected saturation value comprises calculating the corrected saturation value according to Equations (3) to (5); l = || v → ⁢ || 2 ⁢ - ( v → · u → ) 2 || u → ⁢ || 2 ( 3 ) l max = || v max → ⁢ || 2 ⁢ - ( v max → · u → ) 2 || u → ⁢ || 2 ( 4 ) S = l / l max ⁡ ( 0 ≤ S ≤ 1 ) ( 5 ) where {right arrow over (u)} denotes a unit vector of a straight line which is expressed in an achromatic color in an RGB color space, {right arrow over (v)} denotes a color vector indicating the arbitrary pixel in the RGB color space or denotes a color vector indicating the arbitrary pixel in the HSI color space, l(0≤l≤l max ) denotes a vertical distance from coordinates of {right arrow over (v)} to the straight line in the RGB color space or, denotes a vertical distance from the coordinates of {right arrow over (v)} to the center axis in the HSI color space, and {right arrow over (v max )} denotes a vector having l max .

8. A multi-view immersive content generating device comprising: a background model generator configured to model a multi-view background image obtained from a plurality of cameras arranged in a curved shape to generate a codebook corresponding to the multi-view background image; a foreground/background separator configured to separate a foreground and a background from a multi-view image obtained from the plurality of cameras by using the codebook; and a synthesizer configured to synthesize the separated foreground with a virtual background to generate multi-view immersive content, wherein the foreground/background separator comprises: a color spacer converter configured to convert an RGB color value of the multi-view image into an HSV color value expressed in an HSV color space; a hue difference calculator configured to calculate a hue difference between a first HSV color value of the multi-view background image and a second HSV color value of the multi-view image which are stored in the codebook; and a comparison unit configured to compare the calculated hue difference with a threshold value to separate the foreground and the background from the multi-view image, and wherein the hue difference calculator calculates, as the hue difference, an area of a triangle formed between a first hue value of the first HSV color value and a second hue value of the second HSV color value on an RGB-HSV spectrum graph that includes an abscissa axis indicating a hue value and an ordinate axis indicating an intensity value.

9. The multi-view immersive content generating device of claim 8 , further comprising: a guide rail, the plurality of cameras being arranged on the guide rail, wherein the guide rail has a curved shape that forms a certain radius with respect to a convergence point.

11. The multi-view immersive content generating device of claim 8 , wherein the foreground/background separator comprises: a color space converter configured to convert an RGB color value of each of pixels of the multi-view image into an HSI color value in a cone-shaped HSI color space with intensity as a center axis; a saturation calculator configured to calculate a corrected saturation value by dividing a saturation value of an arbitrary pixel in the HSI color space by a maximum saturation value expressible in the HSI color space; and a comparison unit configured to compare the corrected saturation, value with the saturation value of the arbitrary pixel stored in the codebook to separate the foreground and the background from the multi-view image.

12. The multi-view immersive content generating device of claim 11 , wherein the hue difference calculator calculate the corrected saturation value according to Equations (3) to (5): l = || v → ⁢ || 2 ⁢ - ( v → · u → ) 2 || u → ⁢ || 2 ( 3 ) l max = || v max → ⁢ || 2 ⁢ - ( v max → · u → ) 2 || u → ⁢ || 2 ( 4 ) S = l / l max ⁡ ( 0 ≤ S ≤ 1 ) ( 5 ) where {right arrow over (u)} denotes a unit vector of a straight line which is expressed in an achromatic color in an RGB color space, {right arrow over (v)} denotes a color vector indicating the arbitrary pixel in the RUB color space or denotes a color vector indicating the arbitrary pixel in the HSI color space, l(0≤l≤l max ) denotes, a vertical distance from coordinates of {right arrow over (v)} to the straight line in the RGB color space or denotes a vertical distance from the coordinates of {right arrow over (v)} to the center axis in the HSI color space, and {right arrow over (v max )} denotes a vector having l max .

13. The multi-view immersive content generating device of claim 8 , wherein the foreground/background separator comprises: a color space converter configured to convert an RGB vector of each of pixels of the multi-view image into an HSI vector in a cone-shaped HSI color space with intensity as a center axis; an intensity calculator configured to calculate an intensity boundary value by applying a weight variable to an intensity value of an arbitrary pixel in the HSI color space; and a comparison unit configured to compare the calculated intensity boundary value with a threshold value stored in the codebook to separate the foreground and the background from the multi-view image.

14. The multi-view immersive content generating device of claim 13 , wherein the intensity calculator calculates the intensity boundary value according to Equation (6): Intensity ⁢ ⁢ boundary = [ max ⁢ { α ⁢ ⁢ I high , τ black } , max ⁢ { min ⁢ { β ⁢ ⁢ I high , I low α } , τ black } ] ( 6 ) where I high denotes a highest intensity value among intensity values of pixels which are collected in units of one frame, I low denotes a lowest intensity value among the intensity values of the pixels which are collected in units of one frame, α and β(α<β) each denote the weight variable determined based on a user input, and τ black denotes a threshold value that is set for the intensity boundary value not to be excessively reduced.