Method and Apparatus for Processing and Distributing Live Virtual Reality Content

1. A graphics processor comprising: a video interface to receive a first plurality of images from a corresponding first plurality of cameras; an image rectifier to perform a perspective re-projection of at least some of the first plurality of images to a common image plane to generate a rectified first plurality of images; a stitcher to analyze overlapping regions of adjacent images in the rectified first plurality and to identify corresponding pixels in the overlapping regions and to stitch the adjacent images in accordance with the corresponding pixels to generate a panoramic image comprising a stitched combination of the rectified first plurality of images, wherein the stitcher is to analyze the overlapping regions by executing a sequence of Belief Propagation operations that comprise: construction of an initial data cost volume using pixel data in the overlapping regions; construction of a data cost pyramid based on the initial data cost volume comprising a series of smaller volumes; iteration through the series of smaller volumes and the initial volume using Belief Propagation message passing to generate a final set of costs; and construction of a stitch map from the final set of costs; and a cylindrical projector to project the panoramic image onto a cylindrical surface to generate a final panoramic video image to be used to implement a virtual reality (VR) environment on a VR apparatus.

2. The graphics processor of claim 1 wherein the stitcher is to stitch the adjacent images in accordance with the final set of costs.

3. The graphics processor of claim 2 wherein for each pixel in the overlapping region the stitcher is to use the stitch map to determine which pixels to blend.

4. The graphics processor of claim 3 wherein the stitcher is to use a convex linear combination of pixels to blend from each image.

5. The graphics processor of claim 1 further comprising: a lookahead buffer to store stitching parameters determined from one or more prior images, the stitcher to use at least a portion of the stitching parameters stored in the lookahead buffer to stitch the adjacent images in the rectified first plurality of images.

6. The graphics processor of claim 1 wherein the perspective re-projection comprises a homography transformation.

7. The graphics processor of claim 1 wherein the first plurality of images comprise left images to be displayed, following processing, in a left display of the VR apparatus.

8. The graphics processor of claim 7 wherein the video interface is to receive a right plurality of images from a corresponding second plurality of cameras, wherein the image rectifier is to perform a perspective re-projection of at least some of the right plurality of images to a common image plane, wherein the stitcher is to analyze overlapping regions of adjacent images in the right plurality and to identify corresponding pixels in the overlapping regions and to stitch the adjacent images in accordance with the corresponding pixels to generate a panoramic right eye image comprising a stitched combination of the right plurality of images; and wherein the cylindrical projector is to project the panoramic right eye image onto a cylindrical surface to generate a final panoramic right eye image to be combined with a final panoramic left eye image to implement a virtual reality (VR) environment on a VR apparatus.

9. The graphics processor of claim 1 wherein the image rectifier, stitcher, and cylindrical projector comprise circuitry and/or executable software executed by an execution unit of the graphics processor.

10. A method comprising: receiving a first plurality of images from a corresponding first plurality of cameras; performing a perspective re-projection of at least some of the first plurality of images to a common image plane to generate a rectified first plurality of images; analyzing overlapping regions of adjacent images in the rectified first plurality by performing a sequence of Belief Propagation operations and responsively identifying corresponding pixels in the overlapping regions, wherein the sequence of Belief Propagation operations comprise: constructing an initial data cost volume using pixel data in the overlapping regions; constructing a data cost pyramid based on the initial data cost volume comprising a series of smaller volumes; iterating through the series of smaller volumes and the initial volume using Belief Propagation message passing to generate a final set of costs; and constructing a stitch map from the final set of costs; stitching the adjacent images in accordance with the corresponding pixels to generate a panoramic image comprising a stitched combination of the rectified first plurality of images; and projecting the panoramic image onto a cylindrical surface to generate a final panoramic video image to be used to implement a virtual reality (VR) environment on a VR apparatus.

11. The method of claim 10 wherein stitching comprises stitching the adjacent images in accordance with the final set of costs.

12. The method of claim 11 wherein for each pixel in the overlapping region the stitch map is used to determine which pixels to blend.

13. The method of claim 12 wherein a convex linear combination of pixels is selected to blend from each image.

14. The method of claim 10 further comprising: storing stitching parameters determined from one or more prior images, and stitching the adjacent images in the rectified first plurality of images using at least a portion of the stitching parameters.

15. The method of claim 10 wherein the perspective re-projection comprises a homography transformation.

16. The method of claim 10 wherein the first plurality of images comprise left images to be displayed, following processing, in a left display of the VR apparatus.

17. The method of claim 16 wherein the video interface is to receive a right plurality of images from a corresponding second plurality of cameras, the method further comprising: performing a perspective re-projection of at least some of the right plurality of images to a common image plane to generate rectified right plurality of images; analyzing overlapping regions of adjacent images in the rectified right plurality and identifying corresponding pixels in the overlapping regions; and stitching the adjacent images in accordance with the corresponding pixels to generate a panoramic right eye image comprising a stitched combination of the rectified right plurality of images; and projecting the panoramic right eye image onto a cylindrical surface to generate a final panoramic right eye image to be combined with a final panoramic left eye image to implement a virtual reality (VR) environment on a VR apparatus.

18. A non-transitory machine-readable medium having program code stored thereon which, when executed by a machine, causes the machine to perform operations of: receiving a first plurality of images from a corresponding first plurality of cameras; performing a perspective re-projection of at least some of the first plurality of images to a common image plane to generate a rectified first plurality of images; analyzing overlapping regions of adjacent images in the rectified first plurality by performing a sequence of Belief Propagation operations and responsively identifying corresponding pixels in the overlapping regions, wherein the sequence of Belief Propagation operations comprise: constructing an initial data cost volume using pixel data in the overlapping regions; constructing a data cost pyramid based on the initial data cost volume comprising a series of smaller volumes; iterating through the series of smaller volumes and the initial volume using Belief Propagation message passing to generate a final set of costs; and constructing a stitch map from the final set of costs; stitching the adjacent images in accordance with the corresponding pixels to generate a panoramic image comprising a stitched combination of the rectified first plurality of images; and projecting the panoramic image onto a cylindrical surface to generate a final panoramic video image to be used to implement a virtual reality (VR) environment on a VR apparatus.

19. The non-transitory machine-readable medium of claim 18 wherein stitching comprises stitching the adjacent images in accordance with the final set of costs.

20. The non-transitory machine-readable medium of claim 19 wherein for each pixel in the overlapping region the stitch map is used to determine which pixels to blend.

21. The non-transitory machine-readable medium of claim 20 wherein a convex linear combination of pixels is selected to blend from each image.

22. The non-transitory machine-readable medium of claim 18 wherein the operations further comprise: storing stitching parameters determined from one or more prior images, and stitching the adjacent images in the rectified first plurality of images using at least a portion of the stitching parameters.

23. The non-transitory machine-readable medium of claim 18 wherein the perspective re-projection comprises a homography transformation.

24. The non-transitory machine-readable medium of claim 18 wherein the first plurality of images comprise left images to be displayed, following processing, in a left display of the VR apparatus.

25. The non-transitory machine-readable medium of claim 24 wherein the video interface is to receive a right plurality of images from a corresponding second plurality of cameras, the operations further comprising: performing a perspective re-projection of at least some of the right plurality of images to a common image plane to generate rectified right plurality of images; analyzing overlapping regions of adjacent images in the rectified right plurality and identifying corresponding pixels in the overlapping regions; and stitching the adjacent images in accordance with the corresponding pixels to generate a panoramic right eye image comprising a stitched combination of the rectified right plurality of images; and projecting the panoramic right eye image onto a cylindrical surface to generate a final panoramic right eye image to be combined with a final panoramic left eye image to implement a virtual reality (VR) environment on a VR apparatus.