Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. An image rendering device for realizing stereoscopic viewing of joint image data, the joint image data being constructed of a plurality of pieces of background image data that are stitched together in a row direction, the image rendering device comprising: a pixel shifter that applies pixel shifting to each piece of background image data and conducts boundary processing relative to the pixel shifting, thereby to acquire two or more viewpoint textures for the joint image data; a mapper that places the two or more viewpoint textures onto inner surfaces of three-dimensional models in a three-dimensional modeling space; a viewport converter that extracts, for each of two or more viewpoints, a viewport image from the textured three-dimensional models, wherein the pixel shifting is to shift coordinates of respective pixels in each piece of background image data in the row direction, and the amount of shift applied to each pixel is determined based on a depth of a corresponding pixel in depth image data associated with the joint image data, and the boundary processing is to extract out-of-region pixels which are pixels to be located outside a display region of a given one of the pieces of background image data as a result of the pixel shifting, and write the extracted out-of-region pixels at an edge of another piece of background image data that is adjacent to the given piece of background image data in the row direction.
The image rendering device creates a stereoscopic 3D view from a joint panoramic image. This panoramic image is constructed by stitching multiple background images together horizontally. The device shifts pixels in each background image horizontally based on the depth of the corresponding pixel in a depth map. A boundary processing step handles pixels that shift outside the image boundaries, moving them to the opposite edge of the adjacent background image. The shifted images are mapped onto the inside of 3D models, and then rendered from two or more viewpoints to create the stereoscopic effect.
2. The image rendering device according to claim 1 , further comprising: a display region buffer; an out-of-region buffer; and a texture memory that stores a piece of background image data already processed by the pixel shifting, wherein of the plurality of pieces of background image data constructing the joint image data, a piece of background image data currently subjected to the pixel shifting is designated as a current piece of background image data, the pixel shifting is carried out by: storing the current piece of background image data in the display region buffer; shifting, on the display region buffer, coordinates of respective pixels in the current piece of background image data in the row direction; and extracting out-of-region pixels shifted to outside a display region of the current piece of background image data and storing the extracted out-of-region pixels in the out-of-region buffer, and the boundary processing is carried out by: reading the out-of-region pixels from the out-of-region buffer; and writing the read out-of-region pixels at the edge of the piece of already processed background image data stored in the texture memory.
The image rendering device described previously uses a display region buffer to hold the current background image being processed. Pixel shifting is performed within this buffer. Out-of-region pixels, generated by the pixel shifting, are extracted and stored in an out-of-region buffer. The boundary processing reads these out-of-region pixels from the out-of-region buffer and writes them to the edge of an adjacent background image. This adjacent image has already been processed by pixel shifting and is stored in a texture memory, allowing for seamless stitching of the panoramic image with corrected pixel offsets for stereoscopic viewing.
3. The image rendering device according to claim 1 , wherein the joint image data is joint Mercator image data obtained by stitching together one edge and another edge of a single piece of Mercator image data, and the boundary processing is carried out by: extracting out-of-region pixels which are pixels to be located outside a display region of the Mercator image data as a result of the pixel shifting applied to respective pixel in the joint Mercator image data based on the depth image data; and writing the extracted out-of-region pixels into the joint Mercator image at the opposite edge from the edge where the out-of-region pixels are extracted.
In the image rendering device described previously, the joint image data is a joint Mercator image, formed by stitching the left and right edges of a single Mercator image together. The boundary processing extracts out-of-region pixels generated by pixel shifting applied to the Mercator image based on depth data. These extracted pixels are then written to the opposite edge of the Mercator image from where they were extracted, creating a looped panoramic view suitable for stereoscopic rendering.
4. The image rendering device according to claim 1 , further comprising: a position and direction determiner that determines, in accordance with a user operation, a current viewpoint location on a map and a current line-of-sight direction; and an acquisitor that generates an image acquisition request using geographic information corresponding to the current viewpoint location on the map and transmits the image acquisition request to an image collection server, thereby to download a street view file, wherein the street view file downloaded from the image collection server has a location attribute matching the geographic information included in the image acquisition request, and the joint image data is contained in the downloaded street view file.
The image rendering device described previously includes a position and direction determiner that uses user input to determine the current viewpoint location on a map and the viewing direction. An image acquisitor then generates an image acquisition request using geographic coordinates and sends it to an image collection server. This downloads a street view file containing joint image data (the panoramic image). The downloaded file's location metadata must match the requested geographic information.
5. The image rendering device according to claim 1 , wherein the joint image data is a panoramic image and constructed of the plurality of pieces of background image data stitched together in the row direction to represent a view of surroundings as seen from a geographic location corresponding to the geographic information included in the image acquisition information.
In the image rendering device described previously, the joint image data is a panoramic image. The panoramic image is created by stitching multiple background images together horizontally. These background images represent a 360-degree view from a specific geographic location, as determined by the geographic information included in the image acquisition request used to obtain the image data.
6. An image rendering method for realizing stereoscopic viewing of joint image data, the joint image data being constructed of a plurality of pieces of background image data that are stitched together in a row direction, the image rendering method comprising: applying pixel shifting to each piece of background image data and conducting boundary processing relative to the pixel shifting, thereby to acquire two or more viewpoint textures for the joint image data; placing the two or more viewpoint textures onto inner surfaces of three-dimensional models in a three-dimensional modeling space; extracting, for each of two or more viewpoints, a viewport image from the textured three-dimensional models, wherein the pixel shifting is to shift coordinates of respective pixels in each piece of background image data in the row direction, and the amount of shift applied to each pixel is determined based on a depth of a corresponding pixel in depth image data associated with the joint image data, and the boundary processing is to extract out-of-region pixels which are pixels to be located outside a display region of a given one of the pieces of background image data as a result of the pixel shifting, and write the extracted out-of-region pixels at an edge of another piece of background image data that is adjacent to the given piece of background image data in the row direction.
The image rendering method creates a stereoscopic 3D view from a joint panoramic image. This panoramic image is constructed by stitching multiple background images together horizontally. The method shifts pixels in each background image horizontally based on the depth of the corresponding pixel in a depth map. A boundary processing step handles pixels that shift outside the image boundaries, moving them to the opposite edge of the adjacent background image. The shifted images are mapped onto the inside of 3D models, and then rendered from two or more viewpoints to create the stereoscopic effect.
7. A non-transitory recording medium recording an image rendering program for causing a computer to realize stereoscopic viewing of joint image data, the joint image data being constructed of a plurality of pieces of background image data that are stitched together in a row direction, the image rendering program comprising code that causes the computer to perform: applying pixel shifting to each piece of background image data and conducting boundary processing relative to the pixel shifting, thereby to acquire two or more viewpoint textures for the joint image data; placing the two or more viewpoint textures onto inner surfaces of three-dimensional models in a three-dimensional modeling space; extracting, for each of two or more viewpoints, a viewport image from the textured three-dimensional models, wherein the pixel shifting is to shift coordinates of respective pixels in each piece of background image data in the row direction, and the amount of shift applied to each pixel is determined based on a depth of a corresponding pixel in depth image data associated with the joint image data, and the boundary processing is to extract out-of-region pixels which are pixels to be located outside a display region of a given one of the pieces of background image data as a result of the pixel shifting, and write the extracted out-of-region pixels at an edge of another piece of background image data that is adjacent to the given piece of background image data in the row direction.
An image rendering program, stored on a non-transitory medium, enables a computer to create stereoscopic 3D views from joint panoramic images. The panoramic image is constructed by stitching multiple background images together horizontally. The program shifts pixels in each background image horizontally based on the depth of the corresponding pixel in a depth map. A boundary processing step handles pixels that shift outside the image boundaries, moving them to the opposite edge of the adjacent background image. The shifted images are mapped onto the inside of 3D models, and then rendered from two or more viewpoints to create the stereoscopic effect.
Unknown
December 9, 2014
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.