Systems and Methods for Enhanced Playback of Stereoscopic Media

This application is a continuation of U.S. patent application Ser. No. 18/787,754, filed Jul. 29, 2024 the disclosure of which is hereby incorporated by reference herein in its entirety.

This disclosure is related to providing content, and more particularly to systems and methods for providing stereoscopic media.

Stereoscopic media, which may present either an image or a video, has been used for years to present three-dimensional (3D) images and video to viewers. In general, with respect to video, stereoscopic media includes a left-eye media channel and a right-eye media channel, and each media channel includes video that is slightly different (e.g., a different viewing angle of the same scene) from the video in the other media channel. When the videos in the two media channels are viewed simultaneously as a stereoscopic video, the viewer perceives the stereoscopic video as a 3D video due to, e.g., parallax differences between the videos in each channel. Similarly, stereoscopic media based on images may be used to present 3D images to a viewer.

A problem that frequently arises with stereoscopic media is when media in one of the two channels is a different (e.g., lower) quality than media in the other channel. A quality discrepancy in the left and right channels may be due to, e.g., bandwidth constraints, compression, encryption requirements, and/or other media capturing and/or processing techniques. This difference in quality generally makes no difference to viewers who view the higher-quality media channel with their dominant eye. However, if those users view the lower-quality media channel with their dominant eye, the composite stereoscopic media is perceived to have a lower quality. With about 60-70% of the population being right-eye dominant, this can result in a substantial left-eye dominant audience perceiving any single stereoscopic media content at a lower-than-desired quality.

This quality difference between media channels of stereoscopic media typically arises from the way stereoscopic media is created, distributed, and/or transmitted. Some professionally produced stereoscopic media may have no quality differences between the videos of two channels while being produced in the studio. However, stereoscopic media is generally compressed for distribution and/or transmission to the end viewer, and common compression schemes for stereoscopic media utilize asymmetric compression techniques. Such compression techniques may improve coding efficiency by compromising the image quality of one media channel, and such compression techniques assume that the overall perceptual quality can be maintained in a viewer who can see equally well out of both eyes. Some professionally produced stereoscopic media may be created knowing that asymmetric compression techniques will be used, such that efficiencies are introduced into the creative process by starting with one higher-quality media channel and one lower-quality media channel. For such stereoscopic media, the asymmetric compression process does not significantly compromise the image quality of the media channel that is already of lower quality. Still, a substantial amount of professionally produced stereoscopic media reaches the viewer with one media channel being higher quality than the other media channel.

Non-professionals may also create stereoscopic media using smartphones that are equipped with multiple cameras, each camera having different lenses spaced apart on the body of the smartphone, e.g., allowing for depth. However, each camera on a smartphone typically has a different purpose (e.g., primary camera, telephoto camera, ultra-wide camera, and the like), and each camera frequently therefore also produces images with different pixel capabilities (e.g., 12 megapixels, 48 megapixels, etc.). When images/video from multiple cameras on a smartphone are used to create stereoscopic media, the media channel using the images/video from the lower megapixel camera will automatically be a lower quality. Moreover, since a smartphone may be used in any orientation to create images/video (e.g., vertical, upside down, horizontally left, horizontally right, or innumerable angles in between), one stereoscopic media created by the same smartphone user may have a different, higher-quality media channel than the next stereoscopic media created by the same smartphone user. If the phone is oriented where the lower quality camera is capturing for the right eye, a majority of the population may view the media at a resolution that is less than optimal. For instance, a smartphone capturing a video horizontally (e.g., in a widescreen format) may have the primary camera as the highest and farthest left in one orientation and may have the primary camera lowest and farthest right when rotated 180 degrees. Whether video is produced by professionals or non-professionals, stereoscopic media often includes unequal image quality between the two channels. Thus, for multiple reasons, the way stereoscopic media is created, distributed, and/or transmitted can have a particularly high impact on the quality of the stereoscopic media perceived by the viewer.

A need therefore exists to optimize the quality of stereoscopic media with disparate channel quality. To address this need and overcome the shortcomings introduced by existing systems that create, distribute, and/or transmit stereoscopic media, systems and methods that identify the dominant eye of a viewer and, when needed, modify the stereoscopic media to present the higher-quality media channel to the viewer's dominant eye are presented. The enhanced playback may be achieved by determining if stereoscopic media includes a higher-quality media channel and mapping that higher-quality media channel to the dominant eye of the user viewing the stereoscopic media. This mapping may be performed by the user equipment presenting the stereoscopic media to the user for viewing.

The modifications to stereoscopic media have low processing complexity, such that most viewer's equipment would be able to perform the modifications in real time to present the viewer with the highest quality stereoscopic media possible based on the initial quality of the stereoscopic media and the viewer's own sight capabilities, e.g., without disturbing the stereoscopic production and distribution processes. For instance, some embodiments may identify the viewer's dominant eye, determine if the lower-quality media channel is mapped for display to the viewer's dominant eye, and if so, perform mirror and swap functions between media in the left-eye media channel and media in the right-eye media channel. The modified stereoscopic media will therefore map the higher-quality media channel for display to the viewer's dominant eye. As discussed herein, stereoscopic media with spatial audio and/or subtitles may also be modified using low processing requirements to improve the overall quality of stereoscopic media perceived by the viewer.

Systems and methods are described herein for enhancing playback of stereoscopic media. The systems and methods may be used to improve the presentation of stereoscopic media to a viewer, particularly with high-quality displays, by providing an improved perceived picture quality from stereoscopic media. Advantageously, the systems and methods are based on display-side processing that can be achieved without upgrades to most display-side user equipment and without changes to the compression techniques used for distribution and/or communication of the stereoscopic media to user equipment. In addition, the systems and methods described herein may be leveraged to reduce the complexities of video encoding optimization for stereoscopic media.

As referred to herein, the term “content” should be understood to mean an electronically consumable asset accessed using any suitable electronic platform, such as broadcast television, pay-per-view, on-demand (as in video-on-demand (VOD) systems), network-accessible media (e.g., streaming media, downloadable media, Webcasts, etc.), video clips, information about media, images, animations, documents, playlists, websites and webpages, articles, books, electronic books, blogs, chat sessions, social media, software applications, games, virtual reality media, augmented reality media, and/or any other media or multimedia and/or any combination thereof. Extended reality (XR) content, which is a type of content, refers to augmented reality (AR) content, virtual reality (VR) content, hybrid or mixed reality (MR) content, and/or other digital content combined therewith to mirror physical-world objects, including interactions with such content.

The term “stereoscopic media,” as used herein, should be understood to be any type of content (including XR content) that includes images and/or video of the same scene separated into a left-eye media channel and a right-eye media channel. The images and/or video in the left-eye media channel may have a different viewing angle of the scene as compared to the images and/or video in the right-eye media channel, such that when the media in each of the left-eye media channel and the right-eye media channel are viewed simultaneously, the viewer may perceive the stereoscopic media as 3D media.

In general, stereoscopic media includes a left-eye media channel and a right-eye media channel, with each media channel including media content. For purposes of discussion herein, the media content in the left-eye media channel may be referred to as the left-eye media, or other similar term, and the media content in the right-eye media channel may be referred to as the right-eye media, or other similar term. The left-eye media and the right-eye media may include images/video of a scene from two different viewing perspectives to create parallax between the left-eye media and the right-eye media. To enable a viewer to view the stereoscopic media and perceive the viewed media as 3D media, the left-eye media in the left-eye media channel is mapped for viewing to the left eye of the viewer, and the right-eye media in the right-eye media channel is mapped for viewing to the right eye of the viewer. The left and right eye mappings, combined with parallax between the left-eye media and the right-eye media, enable the viewer to perceive the stereoscopic media as 3D media.

1 FIG. 100 102 100 102 104 102 102 102 Turning in detail to the drawings,shows a first example of an environmentin which enhanced playback of stereoscopic media is employed to improve the perceived quality of the viewer. In this environment, the vieweris wearing a head-mounted display (HMD)that is configured to display stereoscopic media to the viewer. In some embodiments, the stereoscopic media may be displayed to the vieweras stand-alone content. In some embodiments, the stereoscopic media may be displayed to the vieweras part of other content, such as part of extended reality (XR) content, television content, electronic book content, website content, social media content, chat content, etc.

104 102 102 In some embodiments, the HMDmay be XR glasses and/or goggles for displaying XR media to the viewer. In such embodiments, the XR media may be created entirely using stereoscopic media so that the viewerwithin the XR environment views the XR media stereoscopically. Alternatively, in such embodiments, the XR media may include a sub-part that includes the stereoscopic media so that the viewermay view the stereoscopic media in a section of and/or on demand within the XR environment.

104 In some embodiments, stereoscopic glasses may be used in combination with a display screen (e.g., a smartphone, a tablet computer, a laptop computer, a television, a computer display, and the like) in place of the HMD. In such embodiments, the display screen may be configured to display the left- and right-eye media channels of the stereoscopic media with different light polarizations, and the stereoscopic glasses may be configured to transmit one of the media channels to each eye using polarization filters. Other types of image/video displays may be used in combination with stereoscopic glasses so that a viewer perceives the stereoscopic media as 3D media. Likewise, other types of stereoscopic glasses may be used in combination with other types of displays so that a viewer perceives the stereoscopic media as 3D media.

104 100 110 110 112 114 112 116 114 118 112 116 102 114 118 102 2 FIG. The HMDin this environmentreceives stereoscopic mediafrom a media source (see). The stereoscopic mediaincludes a left-eye media channeland a right-eye media channel. The left-eye media channelincludes left-eye media, and the right-eye media channelincludes right-eye media. The left-eye media channelis mapped to display the left-eye mediato the left eye of the viewer, and the right-eye media channelis mapped to display the right-eye mediato the right eye of the viewer.

110 102 104 102 110 102 102 104 102 Before displaying the received stereoscopic mediato the viewer, the HMDidentifies whether the left eye or the right eye is the dominant eye of the viewer. In some embodiments, this identification may be done prior to receiving the stereoscopic mediaby conducting a brief visual test for the viewerto determine which eye is dominant. When done in advance, the identification of the dominant eye may be stored and associated with a user profile for the viewer, and the identification of the dominant eye may thereafter be determined simply by accessing the data stored in the user profile. In some embodiments, the HMDmay perform a brief visual test to determine the dominant eye of the viewer.

110 102 104 112 114 116 118 116 112 118 114 Also before displaying the received stereoscopic mediato the viewer, the HMDidentifies if one of the left-eye media channeland the right-eye media channelis a lower-quality media channel than the other. This identification may be performed by determining if one of the left-eye mediaand the right-eye mediais lower-quality media than the other. If the left-eye mediais determined to be the lower-quality media, then the left-eye media channelis identified as the lower-quality media channel. If the right-eye mediais determined to be the lower-quality media, then the right-eye media channelis identified as the lower-quality media channel.

116 118 116 118 116 118 110 116 118 116 118 116 118 116 118 104 116 118 In some embodiments, metadata associated with the stereoscopic media may be used to identify which of the left-eye mediaand the right-eye mediais the lower-quality media. In some embodiments, the compression type, if an asymmetric compression algorithm was employed, may be used to identify which of the left-eye mediaand the right-eye mediais the lower-quality media. Asymmetric compression algorithms, by their nature, process the left-eye mediaand the right-eye mediaof the stereoscopic mediadifferently, which results in the quality of one of the left-eye mediaand the right-eye mediabeing compromised. The decompression process may be used to determine which of the left-eye mediaand the right-eye mediawas subject to quality compromises during the compression process, thus enabling identification of the lower-quality media as between the left-eye mediaand the right-eye media. In some embodiments, a comparison of the left-eye mediaand the right-eye mediamay be performed to determine which is the lower-quality media. For example, the HMDmay compare the left-eye mediaand the right-eye mediabased on one or more factors such as the bitrates, the effective resolutions, the overall focus, the range of brightness values for pixels, and the presence and quantity of compression artifacts, among others.

104 120 With both the dominant eye and the lower-quality media channel identified, the HMDmay generate a modified stereoscopic mediain response to the lower-quality media channel being mapped to the dominant eye. In general, the modified stereoscopic media is a version of the received stereoscopic media with the higher-quality media displayed in the media channel that is mapped to the dominant eye of the viewer. With the higher-quality media displayed to the dominant eye of the viewer, the viewer perceives the stereoscopic media as having a higher quality. Advantageously, such a low-complexity display-side solution to improve the perceived display quality of stereoscopic media not only improves the 3D experience for the viewer of the stereoscopic media, but it also has the potential to introduce additional efficiencies on the back end of creation, production, and distribution of stereoscopic media.

120 116 118 116 122 118 124 116 112 124 118 114 122 120 102 104 The modified stereoscopic mediais generated by mirror and swap functions performed on the left-eye mediaand the right-eye media. These mirror and swap functions may be performed in any order, however, for purposes of this description, the mirror function is performed first. The mirror function horizontally mirrors the left-eye mediato generate a mirrored right-eye media, and the mirror function also horizontally mirrors the right-eye mediato generate a mirrored left-eye media. In the swap function, the left-eye mediain the left-eye media channelis replaced with the mirrored left-eye media, and the right-eye mediain the right-eye media channelis replaced with the mirrored right-eye media. These mirror and swap functions generate the modified stereoscopic media, which may then be displayed to the viewerusing the HMD.

Performing a swap function without also performing a mirror function results in a visual mess for the resulting stereoscopic media, such that a swap function without a mirror function is not able to enhance playback of stereoscopic media. Similarly, performing a mirror function without also performing a swap function results in stereoscopic media that presents the viewer with skewed perspectives. The combination of the mirror and swap functions, however, can, for many types of stereoscopic media, accurately reproduce the viewing experience of stereoscopic media while providing higher quality playback for viewers.

2 FIG. 200 200 200 200 202 204 206 210 212 216 206 208 200 218 220 222 200 200 shows an illustrative head-mounted display (HMD)for enabling a user to view stereoscopic media. The HMDincludes components in accordance with some embodiments of this disclosure, such that the HMDshown is intended to be non-limiting. The HMDincludes a displayenclosed within a mask, control circuitry, storage, input/output (I/O) circuitry, and a power source. The control circuitrymay include a processor. The HMDmay also include one or more integrated components such as a microphone, a speaker, and/or a camera. The HMDmay also include an input interface for communicably coupling external devices (e.g., game controllers, XR controllers, keyboards, remotes, touch-sensitive input devices, speakers, etc.) to the HMD.

200 212 206 212 206 206 212 212 206 The HMDmay access, transmit, receive, and/or retrieve content and data, including stereoscopic media, via the I/O circuitrycommunicably coupled to the control circuitry. As an illustrative example, the I/O circuitrymay provide the control circuitrywith access to content (e.g., broadcast programming, on-demand programming, internet content, content available over a local area network (LAN) or wide area network (WAN), and/or other content) and data. The control circuitrymay be used to send and receive commands, requests, and other data using the I/O circuitry. The I/O circuitrymay communicatively couple the control circuitryto other user equipment, networks, servers, and the like.

202 202 202 The displayis depicted as a generalized embodiment of a head-mounted display for viewing an XR environment. The displaymay include an optical system of one or more optical elements such as a lens in front of an eye of the viewer, one or more waveguides, or an electro-sensitive plane. The displayincludes an image source providing light output as an image to the optical element. Some non-limiting examples of a display include a tensor display, a light field display, a volumetric display, a multi-layer display, an LCD display, amorphous silicon display, low-temperature polysilicon display, electronic ink display, electrophoretic display, active matrix display, electro-wetting display, electro-fluidic display, cathode ray tube display, light-emitting diode display, organic light-emitting diode display, electroluminescent display, plasma display panel, high-performance addressing display, thin-film transistor display, organic light-emitting diode display, surface-conduction electron-emitter display (SED), laser television, carbon nanotubes, quantum dot display, interferometric modulator display, or any other suitable equipment for displaying XR content.

206 208 206 206 210 206 The control circuitrymay be based on any suitable control circuitry. As referred to herein, control circuitry should be understood to mean circuitry based on one or more microprocessors, microcontrollers, digital signal processors, programmable logic devices, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), etc., and may include a multi-core processor (e.g., dual-core, quad-core, hexa-core, or any suitable number of cores) or supercomputer. A processormay include video processing circuitry (e.g., integrated and/or a discrete graphics processor). In some embodiments, the control circuitrymay be distributed across multiple separate processors or processing units, for example, multiple of the same type of processing units (e.g., two Intel Core i7 processors) or multiple different processors (e.g., an Intel Core i5 processor and an Intel Core i7 processor). In some embodiments, the control circuitryexecutes instructions stored in memory (e.g., the storage). Specifically, the control circuitrymay be instructed to perform any of the functions described herein.

206 206 200 206 200 210 200 210 The control circuitrymay include or be communicatively coupled to video generating circuitry and tuning circuitry, such as one or more analog tuners, one or more H.265 decoders or any other suitable digital decoding circuitry, high-definition tuners, or any other suitable tuning or video circuits or combinations of such circuits. Conversion circuitry (e.g., for converting over-the-air, analog, or digital signals to MPEG signals for storage) may also be provided. The control circuitrymay also include scaler circuitry for upconverting and downconverting content into a suitable output format for the HMD. The control circuitrymay also include or be communicatively coupled to digital-to-analog converter circuitry and analog-to-digital converter circuitry for converting between digital and analog signals. The tuning and generating circuitry may be used by the HMDto receive and to display, to play, and/or to record content. The tuning and generating circuitry may also be used to receive video generating data. The circuitry described herein, including, for example, the tuning, video generating, encoding, decoding, encrypting, decrypting, scaler, and analog/digital circuitry, may be implemented using software running on one or more general purpose or specialized processors. Multiple tuners may be provided to handle simultaneous tuning functions (e.g., watch and record functions, picture-in-picture (PIP) functions, multiple-tuner recording, etc.). If the storageis provided or supplemented by a separate device from the HMD, the tuning and generating circuitry (including multiple tuners) may be associated with the storage.

210 210 206 210 200 210 210 210 The storagemay be any device for storing electronic data, such as random-access memory, solid state devices, quantum storage devices, hard disk drives, non-volatile memory or any other suitable fixed or removable storage devices, and/or any combination of the same. The storagemay be an electronic storage device that is part of the control circuitry. As referred to herein, the phrase “electronic storage device” or “storage device” should be understood to mean any device for storing electronic data, computer software, or firmware, such as random-access memory, read-only memory, hard drives, optical drives, digital video disc (DVD) recorders, compact disc (CD) recorders, BLU-RAY disc (BD) recorders, BLU-RAY 3D disc recorders, digital video recorders (DVRs, sometimes called personal video recorders, or PVRs), solid state devices, quantum storage devices, gaming consoles, gaming media, or any other suitable fixed or removable storage devices, and/or any combination of the same. The storagemay store data defining images for display by the HMD. The storagemay be used to store various types of content described herein including XR asset data. Nonvolatile memory may also be used (e.g., to launch a boot-up routine and other instructions). Cloud-based storage may be used to supplement the storageor instead of the storage.

206 212 212 212 200 212 212 200 212 The control circuitrymay include or be coupled to the I/O circuitry, which is suitable for communicating with servers, edge computing systems and devices, table or database servers, or other networks or servers. The instructions for carrying out the above-mentioned functionality may be stored on a server. Such communications may involve the internet or any other suitable communication networks. In addition, the I/O circuitrymay include circuitry that enables peer-to-peer communication of user equipment devices, or communication of user equipment devices in locations remote from each other. In some embodiments, the I/O circuitrymay include circuitry that communicatively couples the HMDto one or more other devices over a network. For example, the I/O circuitrymay include a network adaptor and associated circuitry. The I/O circuitrymay include wires and/or busses for connecting to a physical network port (e.g., an ethernet port, a wireless WiFi port, cellular communication port, or any other type of suitable physical port). Although communication paths are not shown, the HMDmay communicate directly or indirectly with other devices and/or user equipment via one or more communication paths and/or communication networks including short-range, point-to-point communication paths, such as USB cables, IEEE 1394 cables, wireless paths (e.g., Bluetooth, infrared, IEEE 802-11x, etc.), or other short-range communication via wired or wireless paths. For example, the I/O circuitrymay include a Bluetooth network adaptor.

216 216 200 216 The power sourcemay include a source of power or an interface for coupling to an external power source. The power sourcemay be coupled to other components of the HMD. Some non-limiting examples of a power sourceinclude a battery, solar generator, and/or a wired power source.

218 220 200 218 220 200 220 218 218 206 The microphoneand the speakermay be included as integrated equipment with other elements of the HMD. In some embodiments, the microphoneand the speakermay be external to the HMDas stand-alone units. An audio component of videos and other content, including stereoscopic media, may be played through the speaker(or external headphones or other external audio device). The microphonemay receive audio input such as voice commands or speech. For example, a user may speak voice commands that are received by the microphoneand recognized by control circuitry.

222 200 206 The cameramay be any suitable type of image sensor, camera, or other form of optical sensor operating in the visual spectrum that is configured to capture successive images as a video. In some embodiments, the image sensor is integrated with the HMD. In some embodiments, the image sensor may be external and communicably connected to the AR head-mounted display. In some embodiments, the image sensor may be a digital camera that includes a charge-coupled device (CCD) and/or a complementary metal-oxide semiconductor (CMOS) image sensor. In some embodiments, the image sensor may be an analog camera that converts still analog images to digital images via the control circuitryor via a video card.

200 200 200 200 In some embodiments, the HMDmay be communicatively coupled to one or more user input interfaces or devices. Some examples of input devices include a remote control, a secondary user device, a touch-sensitive display, a smartphone device, a tablet, a remote control, mouse, trackball, keypad, keyboard, touchscreen, touchpad, stylus input, joystick, voice recognition interface, and/or other user input interfaces. In some embodiments, the HMDmay include an integrated eye-tracking system or other image sensors directed at the user's eyes to enable determining the dominant eye of the user. In some embodiments, the HMDmay include one or more user interfaces (e.g., buttons, touch-sensitive bars, etc.) for a user to manually provide input to the HMD.

3 FIG. 2 FIG. 300 300 300 200 302 304 is a flowchart illustrating the steps of an exemplary processfor enhanced playback of stereoscopic media to a viewer. The processmay be implemented on the user equipment discussed herein and other systems for enhancing playback of stereoscopic media. One or more actions of the process may be incorporated into or combined with one or more actions of any other process or embodiment described herein. For purposes of clarity, this processis described in the context of being implemented on the HMDshown in. At step, the control circuitry identifies the dominant eye of the user wearing the HMD. In some embodiments, the dominant eye of the user may be identified by reference to profile data associated with the user of the HMD. In some embodiments, the HMD may instruct the user through a brief visual test to determine which eye is the user's dominant eye. At step, the HMD receives the stereoscopic media from a media content source. In some embodiments, the media content source may be other user equipment or a server on a local network. In some embodiments, the media content source may be a server, other remote system, or remote device that communicates the stereoscopic media over a wide area network.

306 Once the HMD has received the stereoscopic media, at stepthe control circuitry identifies which of the left-eye media channel and the right-eye media channel in the stereoscopic media is the lower-quality media channel. This identification is performed by determining which of the left-eye media and the right-eye media is the lower-quality media. If the left-eye media is determined to be the lower-quality media, then the left-eye media channel is identified as the lower-quality media channel. If the right-eye media is determined to be the lower-quality media, then the right-eye media channel is identified as the lower-quality media channel.

In some embodiments, metadata associated with the stereoscopic media may be used to identify which of the left-eye media and the right-eye media is the lower-quality media. In some embodiments, if an asymmetric compression algorithm was employed, the compression algorithm, or alternatively the decompression process or metadata associated with the compressed media, may be used to identify which of the left-eye media and the right-eye media is the lower-quality media. In some embodiments, a comparison of the left-eye media and the right-eye media may be performed to determine which is the lower-quality media. For example, the HMD may compare the left-eye media and the right-eye media based on one or more factors such as the bitrates, the effective resolutions the overall focus, the range of brightness values for pixels, and the presence and quantity of compression artifacts, among others.

308 310 312 314 316 312 314 316 318 300 At step, the control circuitry determines if the identified lower-quality media channel is mapped to the dominant eye. If the identified lower-quality media channel is not mapped to the dominant eye, then at step, the received stereoscopic media is displayed to the user of the HMD. If the identified lower-quality media channel is mapped to the dominant eye, then at step, the control circuitry mirrors the left-eye media to generate mirrored right-eye media, and at stepthe control circuitry mirrors the right-eye media to generate mirrored left-eye media. After the mirrored media is generated, at stepthe control circuitry generates modified stereoscopic media by (1) replacing the left eye media in the left-eye media channel of the stereoscopic media with the mirrored left-eye media and (2) replacing the right eye media in the right-eye media channel of the stereoscopic media with the mirrored right-eye media. In combination, steps,, andmirror and swap the left-eye media and right-eye media. These mirror and swap functions for the left-eye media and the right-media are performed to preserve the disparity and depth in the stereoscopic media as displayed to the viewer. At stepthe HMD displays the modified stereoscopic media to the viewer, who in this processis the user of the HMD.

4 FIG. 400 404 402 400 402 402 400 406 408 410 412 414 406 408 410 402 400 406 408 410 402 406 408 410 402 406 408 410 schematically illustrates a first exemplary processfor generating modified stereoscopic mediafrom received stereoscopic media. For purposes of clarity, the processpreviously determined that the viewer's left eye is dominant, and the received stereoscopic mediadoes not include a spatial audio track, subtitles, or other text displayed within the media. The received stereoscopic mediain this processincludes three media segments,,with a left-eye media channeland a right-eye media channelextending across all three media segments,,. Additional or fewer media segments may be included as part of the stereoscopic media. In this process, each media segment,,may each represent a different video scene in stereoscopic media. Differentiation between the media segments,,may be achieved through a variety of techniques. In some embodiments, media segments may be differentiated from each other (i.e., where one media segment ends, and another media segment begins) based on metadata associated with the received stereoscopic mediaidentifying time codes for the beginning and end of different scenes. In some embodiments, media segments may be differentiated from each other based on identifying cuts, fades, or other transitions between scenes. In some embodiments, media segments may be differentiated based on the presence of advertisement insertion indicators embedded within or associated with media content. Other techniques may be used to differentiate between the media segments,,, and the manner of differentiating between media segments is intended to be non-limiting.

412 416 406 408 410 414 418 406 408 410 400 416 418 406 408 410 416 418 406 408 410 406 418 416 406 406 400 414 408 416 418 408 408 400 412 410 416 418 410 410 400 412 414 The left-eye media channelincludes left-eye mediaextending across the three media segments,,, and similarly, the right-eye media channelincludes right-eye mediaextending across the three media segments,,. In this process, the left-eye mediaand the right-eye mediawithin each of the three media segments,,may be analyzed to determine whether a difference in quality exists between the left-eye mediaand the right-eye mediawithin each media segment,,. For example, the first media segmentmay be analyzed to determine that the right-eye mediais lower quality than the left-eye mediawithin the first media segment. Therefore, within the first media segment, the processidentifies the right-eye media channelas the lower-quality media channel. The second media segmentmay be analyzed to determine that the left-eye mediais lower quality than the right-eye mediawithin the second media segment. Therefore, within the second media segment, the processidentifies the left-eye media channelas the lower-quality media channel. The third media segmentmay be analyzed to determine that no difference in quality exists between the left-eye mediaand the right-eye mediawithin the third media segment. Therefore, within the third media segment, the processdoes not identify either of the left-eye media channelor the right-eye media channelas a lower-quality media channel.

406 408 410 408 400 404 400 408 408 406 408 410 406 400 406 410 408 400 416 420 418 422 508 404 400 416 412 422 418 414 420 404 408 402 Based on the process determining that the left eye is the dominant eye of the viewer and that of the three media segments,,, only the second media segmentincludes a lower-quality media channel mapped to the dominant eye of the viewer, the processproceeds to generate the modified stereoscopic media. In this process, mirror and swap functions are performed only on the second media segmentbecause the second media segmentis the only one of the three media segments,,that includes a lower-quality media channel mapped to the dominant eye of the viewer. Additionally, performing a mirror and swap on the first media segmentwould result in delivering lower quality stereoscopic media to the viewer. Thus, the processdoes not perform any actions on the first media segmentor the third media segment. Within the second media segment, the processmirrors the left-eye mediato generate mirrored right-eye mediaand mirrors the right-eye mediato generate mirrored left-eye media. Then, within the second media segmentin the modified stereoscopic media, the processreplaces the left-eye mediain the left-eye media channelwith the mirrored left-eye mediaand replaces the right-eye mediain the right-eye media channelwith the mirrored right-eye media. When the modified stereoscopic mediais displayed to a viewer (using suitable display and/or viewing equipment), the second media segmentwill be perceived by the viewer will perceive to have a higher quality than it would if it were displayed to the viewer in the stereoscopic media.

400 406 408 410 402 404 408 404 As can be seen from this process, only those media segments,,of the stereoscopic mediathat include a lower-quality media channel mapped to the dominant eye of the viewer are transformed by the mirror and swap functions when generating the modified stereoscopic media. In some embodiments, one or more other media segments may be transformed along with the second media segment, although it may not be desirable to perform such additional transformations if they result in reducing the perceived quality of the other media segments or otherwise creating other undesirable issues with the resulting modified stereoscopic media.

5 FIG. 500 504 502 500 502 502 500 506 508 510 512 514 506 508 510 502 500 506 508 510 502 506 508 510 schematically illustrates a second exemplary processfor generating modified stereoscopic mediafrom received stereoscopic media. For purposes of clarity, the processpreviously determined that the viewer's left eye is dominant, and the received stereoscopic mediadoes not include a spatial audio track, subtitles, or other text displayed within the media. The received stereoscopic mediain this processincludes three media segments,,with a left-eye media channeland a right-eye media channelextending across all three media segments,,. Additional or fewer media segments may be included as part of the stereoscopic media. In this process, each media segment,,may represent a different video scene in stereoscopic media. Differentiation between the media segments,,may be achieved through a variety of techniques, as described above.

512 516 506 508 510 514 518 506 508 510 500 516 518 506 508 510 516 518 506 508 510 506 516 518 506 506 500 512 508 516 518 508 508 500 512 514 510 516 518 510 510 500 512 514 The left-eye media channelincludes left-eye mediaextending across the three media segments,,, and similarly, the right-eye media channelincludes right-eye mediaextending across the three media segments,,. In this process, the left-eye mediaand the right-eye mediawithin each of the three media segments,,may be analyzed to determine whether a difference in quality exists between the left-eye mediaand the right-eye mediawithin each media segment,,. For example, the first media segmentmay be analyzed to determine that the left-eye mediais lower quality than the right-eye mediawithin the first media segment. Therefore, within the first media segment, the processidentifies the left-eye media channelas the lower-quality media channel. The second media segmentmay be analyzed to determine that no difference in quality exists between the left-eye mediaand the right-eye mediawithin the second media segment. Therefore, within the second media segment, the processdoes not identify either of the left-eye media channelor the right-eye media channelas a lower-quality media channel. The third media segmentmay be analyzed to determine that no difference in quality exists between the left-eye mediaand the right-eye mediawithin the third media segment. Therefore, within the third media segment, the processdoes not identify either of the left-eye media channelor the right-eye media channelas a lower-quality media channel.

506 508 510 506 500 504 500 504 500 506 508 510 506 508 510 504 506 506 508 510 500 516 520 518 522 506 508 510 504 500 516 512 522 518 514 520 504 506 502 508 510 502 Based on the process determining that the left eye is the dominant eye of the viewer and that, of the three media segments,,, only the first media segmentincludes a lower-quality media channel mapped to the dominant eye of the viewer, the processproceeds to generate the modified stereoscopic media. In this process, mirror and swap functions are performed to generate the modified stereoscopic media. In some embodiments, as in this process, the mirror and swap functions may be performed on all the media segments,,, as doing so would not reduce the quality of any of the media segments,,in the modified stereoscopic media. In some embodiments, the mirror and swap functions may be performed only on the first media segment. Within each of the media segments,,, the processmirrors the left-eye mediato generate mirrored right-eye mediaand mirrors the right-eye mediato generate mirrored left-eye media. Then, within each of the media segments,,in the modified stereoscopic media, the processreplaces the left-eye mediain the left-eye media channelwith the mirrored left-eye mediaand replaces the right-eye mediain the right-eye media channelwith the mirrored right-eye media. When the modified stereoscopic mediais displayed to a viewer (using suitable display and/or viewing equipment), the first media segmentwill be perceived by the viewer to have a higher quality than it would if it were displayed to the viewer in the stereoscopic media, and the viewer would perceive no difference in quality for either of the second and third media segments,if compared to the stereoscopic media.

500 506 508 510 502 504 506 508 510 As can be seen from this process, all media segments,,of the stereoscopic mediamay be transformed by the mirror and swap functions, as part of generating the modified stereoscopic media, when any one of the media segments,,includes a lower-quality media channel mapped to the dominant eye of the viewer. However, if the mirror and swap functions would reduce the perceived quality of any media segment, the process may skip performing the mirror and swap functions on such media segments.

6 FIG. 600 602 604 606 606 600 606 606 608 602 610 612 606 610 606 612 606 606 610 612 610 612 612 610 612 604 606 612 schematically illustrates a first exemplary processfor generating modified stereoscopic mediafrom stereoscopic mediacreated by a smartphone(the back side of the smartphoneis shown). For consistency and clarity, the description of this processis provided in the context of video captured by the smartphone. The smartphoneis shown in a landscape orientationthat may be used to create the stereoscopic mediausing two cameras,(each camera is represented by the respective lens on the backside of the smartphone) incorporated into the smartphone. The first camerais the primary camera for the smartphone, and the second camerais an ultra-wide-angle camera for the smartphone. In this smartphone, like most typical smartphones, the first camerautilizes an image sensor that captures video with two to four times, or more, the number of pixels as compared to the image sensor utilized by the second camera. As a result, when the cameras,are used to create stereoscopic media by simultaneously capturing video of a scene, video captured by the second (ultra-wide angle) cameraneeds to be cropped to match the framing of video captured by the first camera. This cropping process typically eliminates more than half the pixels in captured video by the second camera. Thus, when the stereoscopic mediais generated using the smartphone, the media channel containing media captured by the second camerawill generally be the lower-quality media channel.

606 608 610 612 600 610 622 620 612 626 624 620 624 602 612 610 624 With the smartphonein the landscape orientation, the first camerais on the left and the second camerais on the right. For purposes of clarity in the description of this process, the video captured by the first camerabecomes the left-eye mediain the left-eye media channel, and the video captured by the second camerabecomes the right-eye mediain the right-eye media channel. In combination, the left-eye media channeland the right-eye media channelform the stereoscopic media. As a result, because video from the second camerais the lower-quality media due to cropping needed to match the framing of video from the first camera, the right-eye media channelis the lower-quality media channel.

602 600 602 604 600 604 602 600 622 630 626 632 600 622 620 632 626 624 630 604 604 602 To enhance playback of the stereoscopic mediawhen the right eye of the viewer is dominant, the processtransforms the stereoscopic mediato generate the modified stereoscopic mediaby performing mirror and swap functions. In this process, the stereoscopic mediaincludes no segments, so the mirror and swap functions are performed on the entirety of the stereoscopic media. The processmirrors the left-eye mediato generate mirrored right-eye mediaand mirrors the right-eye mediato generate mirrored left-eye media. Then, the processreplaces the left-eye mediain the left-eye media channelwith the mirrored left-eye mediaand replaces the right-eye mediain the right-eye media channelwith the mirrored right-eye media. When the modified stereoscopic mediais displayed to a viewer (using suitable display and/or viewing equipment), the viewer would perceive the modified stereoscopic mediato have a higher quality if compared to the stereoscopic media.

7 FIG. 6 FIG. 700 704 702 706 706 700 706 706 708 702 710 712 706 710 706 712 706 706 606 706 708 606 608 606 702 706 712 schematically illustrates a second exemplary processfor generating modified stereoscopic mediafrom stereoscopic mediacreated by a smartphone(the backside of the smartphoneis shown). For consistency and clarity, the description of this processis provided in the context of video captured by the smartphone. The smartphoneis shown in a landscape orientationthat may be used to create the stereoscopic mediausing two cameras,(each camera is represented by the respective lens on the backside of the smartphone) incorporated into the smartphone. The first camerais the primary camera for the smartphone, and the second camerais an ultra-wide-angle camera for the smartphone. The smartphonemay be identical, or nearly identical, to the smartphoneof, with the only difference being the smartphonein the landscape orientationis rotated 180° as compared to the smartphonein the landscape orientation. As discussed above with respect to the smartphone, when the stereoscopic mediais generated using the smartphone, the media channel containing media captured by the second camerais the lower-quality media channel.

706 710 712 700 712 722 720 710 726 724 720 724 702 712 710 720 The smartphoneis shown with the first cameraon the right and the second cameraon the left. For purposes of clarity in the description of this process, the video captured by the second camerabecomes the left-eye mediain the left-eye media channel, and the video captured by the first camerabecomes the right-eye mediain the right-eye media channel. In combination, the left-eye media channeland the right-eye media channelform the stereoscopic media. As a result, because video from the second camerais the lower-quality media due to cropping needed to match the framing of video from the first camera, the left-eye media channelis the lower-quality media channel.

702 700 702 704 700 702 702 700 726 732 722 730 700 726 724 732 726 724 730 704 704 702 To enhance playback of the stereoscopic mediawhen the left eye of the viewer is dominant, the processtransforms the stereoscopic mediato generate the modified stereoscopic mediaby performing mirror and swap functions. In this process, the stereoscopic mediaincludes no segmenting, so the mirror and swap functions are performed on the entirety of the stereoscopic media. The processmirrors the right-eye mediato generate mirrored left-eye mediaand mirrors the left-eye mediato generate mirrored right-eye media. Then, the processreplaces the right-eye mediain the right-eye media channelwith the mirrored right-eye mediaand replaces the left-eye mediain the left-eye media channelwith the mirrored left-eye media. When the modified stereoscopic mediais displayed to a viewer (using suitable display and/or viewing equipment), the viewer perceives the modified stereoscopic mediato have a higher quality if compared to the stereoscopic media.

8 FIGS.A-C 2 FIG. 800 800 800 800 200 802 804 depict a flowchart illustrating the steps of an exemplary processfor enhanced playback of stereoscopic media to a viewer. The processmay be implemented on the user equipment discussed herein and other systems for enhancing playback of stereoscopic media. One or more actions of the processmay be incorporated into or combined with one or more actions of any other process or embodiment described herein. For purposes of clarity, this processis described in the context of being implemented on the HMDshown in. At step, the control circuitry identifies the dominant eye of the user wearing the HMD. In some embodiments, identification of the dominant eye of the user may be identified by reference to profile data associated with the user of the HMD. In some embodiments, the HMD may instruct the user through a short visual test to determine which eye is the user's dominant eye. At step, the control circuitry receives the stereoscopic media from a media content source. In some embodiments, the media content source may be another user device on a local network. In some embodiments, the media content source may be a server or other remote system that communicates the stereoscopic media over a wide area network.

806 806 806 808 808 Once the HMD has received the stereoscopic media, at stepthe control circuitry identifies and accesses a first media segment within the stereoscopic media for analysis. In general, the media segment accessed for analysis at stepis also the first media segment that is displayed at the start of the stereoscopic media. In some embodiments, such as those involving non-sequential stereoscopic media (e.g., 3D environments and the like), the media segment accessed for analysis at stepmay be any media segment included as part of the stereoscopic media. In some embodiments, the stereoscopic media may have only a single media segment, and for such stereoscopic media, the HMD continues as described herein. At step, the control circuitry determines if the lower-quality media channel (i.e., the left-eye media channel or the right-eye media channel) within the media segment accessed for analysis is mapped to the dominant eye of the viewer. As part of the evaluation at step, the control circuitry may determine which of the left-eye media in the left-eye media channel and the right-eye media in the right-eye media channel is the lower-quality media. If the left-eye media is determined to be the lower-quality media, then the left-eye media channel is identified as the lower-quality media channel. If the right-eye media is determined to be the lower-quality media, then the right-eye media channel is identified as the lower-quality media channel.

810 812 814 If, at step, the lower-quality media channel is not mapped to the dominant eye of the viewer, then at step, the control circuitry determines whether there are any additional media segments within the stereoscopic media. If there are no remaining media segments to analyze, then at step, the control circuitry may generate, as needed, the modified stereoscopic media. In embodiments with only a single media segment having the higher-quality media channel mapped to the dominant eye of the viewer, the control circuitry need not generate the modified stereoscopic media. Similarly, if all media segments have the higher-quality media channel mapped to the dominant eye of the viewer, the control circuitry need not generate the modified stereoscopic media. In all other instances, the control circuitry may generate the modified stereoscopic media in view of further evaluations of each media segment within the stereoscopic media.

812 816 816 816 800 808 If, at step, additional media segments remain to be analyzed, then at step, the control circuitry identifies and accesses the next media segment within the stereoscopic media for analysis. In general, the next media segment accessed for analysis at stepis the next media segment in the sequence of display within the stereoscopic media. In some embodiments, such as those involving non-sequential stereoscopic media, the next media segment accessed for analysis at stepmay be any media segment included as part of the stereoscopic media. In such embodiments, the control circuitry may assign identifiers to the media segments to differentiate between those media segments that have been analyzed and those media segments that have yet to be analyzed. Once the next media segment has been accessed, the processreturns to stepto determine if the lower-quality media channel within the media segment currently accessed for analysis is mapped to the dominant eye of the viewer.

810 818 820 812 Returning to step, if the lower-quality media channel is mapped to the dominant eye of the viewer, then at step, the control circuitry sets a modification flag to perform mirror and swap functions on the media segment. At step, the control circuitry determines if the media segment includes text that is not part of subtitles. If the media segment includes text that is not part of subtitles, or subtitles that are hardcoded within the media, the control circuitry proceeds to stepto determine if there is another media segment for analysis. Further analysis of media segments that include text outside of subtitles may be stopped because mirroring the media causes text within the media to read backwards when viewed, and in most instances, media with backwards text is undesirable. In some media, subtitles may be embedded or burnt in within the media such that mirroring the media would cause the subtitles to read backwards. For the same reasons, it is undesirable to mirror media with this type of embedded subtitles.

822 824 800 8 FIG.B If the media segment does not include text or subtitles that would be turned backwards by mirroring, then at stepof, the control circuitry determines if metadata associated with the stereoscopic media places limits on modifications to the media segment. Such metadata limitations may be associated with the stereoscopic media for a variety of reasons. Some media creators may include such metadata limitations with stereoscopic media so that scenes may be viewed as originally intended by the creator. Some media creators may include such metadata limitations to prevent mirroring that the creator views as undesirable for a particular scene. Some media owners and/or media distributors may include such metadata limitations for copyright or other legal reasons. Although the underlying reasons for limitations placed in the metadata may vary, it may be desirable for the process to honor the limitations to support the marketplace of creators, owners, and/or distributors of stereoscopic media. If metadata is associated with the media segment, then at step, the control circuitry sets a modification flag to perform a metadata processing function on the media segment. The metadata processing function may analyze the metadata associated with the stereoscopic media to determine whether any limitations are placed on modifications to the media segment and/or the stereoscopic media. In the event the metadata does place any such limitations (e.g., no mirroring of one or more media segments or scenes, no modifications to the audio, and if one media segment is mirrored, other specified media segments must also be mirrored, among other types of modifications), when the processmodifies media segments, the metadata modification limitations may be adhered to by the control circuitry.

822 824 826 828 If, at step, the metadata doesn't place limits on modifications to the media segment, or after step, at step, the control circuitry determines if the media segment includes spatial audio. If the media segment includes spatial audio, at step, the control circuitry sets a modification flag to perform an audio processing function on the media segment. The audio processing function may analyze the audio associated with the stereoscopic media and modify the audio to maintain coherence with the visual scenes of the stereoscopic media. For example, for stereo audio, the control circuitry may swap the spatial positions of the left and right channels. As another example, for audio such as 5.1 surround sound, the control circuitry may swap the spatial positions of the front left and right channels with each other and swap the spatial positions of the rear left and right channels with each other, leaving the front center channel and the low frequency channel unmodified.

826 828 830 820 If, at step, the media segment does not include spatial audio, or after step, at step, the control circuitry determines if the media segment includes subtitles, and particularly subtitles that are added by overlaying, blending, or compositing with the scenes within the media segment. Subtitles that are overlaid, blended, or composited may be removed or not displayed during the process of mirroring media within a segment to avoid rendering the text of the subtitles backwards in the mirrored media. The subtitles may also be copied, added back into the mirrored media, and reenabled for viewing with the modified stereoscopic media. Importantly, the subtitles are added back into the mirrored media without mirroring the subtitles themselves. This enables the viewer to view the stereoscopic media with enhanced quality and activate readable subtitles if desired. Subtitles that are embedded within the scene of media, however, cannot be processed in this manner by typical user equipment in real time. Therefore, media segments that include embedded subtitles are removed from potential processing by the mirror and swap functions at step.

830 832 834 800 834 836 8 FIG.C If, at step, the media segment does not include subtitles, or after step, at stepof, the control circuitry determines which modification flags have been set in relation to the media segment. As discussed above, the modification flags may include flags for the mirror and swap functions, the metadata processing function, the audio processing function, and the subtitles processing function. Flags for other modification functions, if previously set earlier in the processin relation to the media segment, may also be reviewed at step. Next, at step, the control circuitry modifies the media segment based on the set modification flags. The mirror and swap functions are performed on the media segment unless the metadata indicates that the media segment should not be mirrored. In addition, the metadata processing function, the audio processing function, and/or the subtitles processing function may be performed. The sequence in which each of the functions is performed depends on which of the functions are to be performed. For example, at least the first part of the subtitles processing function (removed and/or not displayed, and also copied) may be performed prior to the mirror function. As another example, the metadata processing function may be performed before any other function to determine if those other functions should be limited or not performed at all.

836 838 838 812 816 800 808 814 8 FIG.A After the media segment has been modified at step, at step, the control circuitry replaces the accessed and analyzed segment with the modified segment in the stereoscopic media as part of generating the modified stereoscopic media. In some embodiments, the replacement process may be performed using the received version of the stereoscopic media. In some embodiments, the control circuitry may make a copy of the received stereoscopic media and perform the replacement process on the copy. Following step, at stepof, the control circuitry determines if there are more media segments for processing. If there are more media segments for processing, at step, the next media segment is accessed, and the processproceeds again to step. If there are no more media segments for processing, then at step, the modified stereoscopic media is generated.

9 FIG. 2 FIG. 900 904 902 902 900 200 900 900 900 900 schematically illustrates an exemplary processfor generating modified stereoscopic mediafrom stereoscopic mediawhile also preserving the readability of subtitles that may be included with the stereoscopic media. For purposes of clarity, this processis described in the context of being implemented on the HMDshown in. The processmay be implemented using the HMD to process media with subtitles that are included by overlaying, blending, or compositing. However, user equipment such as the HMD does not typically have the processing capabilities to perform the processin real-time for media that includes embedded or burnt-in subtitles. Therefore, while the processis described in terms of processing media with subtitles included by overlaying, blending, or compositing, the processis not intended to be so limited.

910 912 914 916 904 918 912 916 918 918 912 916 918 918 912 916 918 918 920 922 920 922 924 926 924 926 928 930 902 918 930 910 928 914 904 918 The left-eye media channelincludes left-eye media, and the right-eye media channelincludes right-eye media. The stereoscopic mediaincludes subtitlesas part of the left-eye mediaand the right-eye media. The subtitlesmay be inserted into the stereoscopic mediaby overlaying, blending, or compositing. To perform mirror and swap functions on the left-eye mediaand the right-eye media, the control circuitry saves a copy of the subtitlesand removes the subtitlesfrom both the left-eye mediaand the right-eye mediaso that the subtitlesare no longer displayed. Removal of the subtitlesgenerates an intermediate left-eye mediaand an intermediate right-eye media. The intermediate left-eye mediaand the intermediate right-eye mediaare processed by the mirror and swap functions, as described herein, to generate, respectively, an intermediate mirrored right-eye mediaand an intermediate mirrored left-eye media. The control circuitry then adds the subtitles into both the intermediate mirrored right-eye mediaand the intermediate mirrored left-eye mediato generate the mirrored right-eye mediaand the mirrored left-eye media. Adding the subtitles may be performed by any one of overlaying, blending, or compositing. In some embodiments, inserting the subtitles by overlaying may be most efficient from processing perspective. In some embodiments, it may be desirable to insert the subtitles by blending or compositing to better preserve the creative presentation of the stereoscopic mediawith subtitles. With generation of the mirrored left-eye mediawithin the left-eye media channeland the mirrored right-eye mediawithin the right-eye media channel, the modified stereoscopic mediais generated with readable subtitles.

918 902 900 924 926 In some embodiments, the subtitlesin the stereoscopic mediamay be presented at varying positions with respect to the media or at varying apparent 3D positions with respect to the perceived 3D media. The process, when saving a copy of the subtitles, may also save data related to the subtitles that may be used to re-create the position of the subtitles and/or the apparent 3D position after generating the intermediate mirrored right-eye mediaand the intermediate mirrored left-eye media.

10 FIG. 2 FIG. 1004 1008 1010 1002 1004 1008 1010 206 210 212 1004 1008 1002 is an example of an illustrative system implementing the user equipment device, in accordance with embodiments of the disclosure. The user equipment devices,,may be coupled to communication network. The user equipment devices,,may include control circuitry, storage, and I/O circuitry similar to, e.g., control circuitry, storage, and I/O circuitryfrom. For some user equipment devices,, stereoscopic glasses and a polarizing screen may be needed for viewing stereoscopic media. Communication networkmay be one or more networks including the internet, a mobile phone network, mobile voice or data network (e.g., a 4G, 5G or LTE network), or other types of communication networks or combinations of communications networks.

1000 1003 1012 1016 1012 1004 1008 1010 1003 1003 1012 1004 1008 1010 1002 Systemmay comprise data source, one or more servers, and/or one or more edge computing devices. In some embodiments, the application may be executed at one or more of control circuitryof server(and/or control circuitry of user equipment,,and/or control circuitry of one or more edge computing devices). Communications with the data source, which may also be a media content source, and the user equipment devices may be exchanged over one or more communication paths. In some embodiments, the user equipment devices exchange communications with the other user equipment over one or more communication paths. In some embodiments, the data sourceand/or servermay be configured to host or otherwise facilitate communication sessions between user equipment,,and/or any other suitable user equipment, and/or host or otherwise be in communication (e.g., over communication network) with one or more network services.

1012 1016 1020 1020 1012 1018 1018 1018 1016 1020 1016 1018 1018 1016 In some embodiments, servermay include control circuitryand storage(e.g., RAM, ROM, Hard Disk, Removable Disk, etc.). Storagemay store one or more databases. Servermay also include an I/O path. In some embodiments, I/O pathis an I/O circuitry. I/O circuitry may be, e.g., a NIC card, audio output device, mouse, keyboard card, any other suitable I/O circuitry device or combination thereof. I/O pathmay provide device information, or other data, over a local area network (LAN) or wide area network (WAN), and/or other content and data to control circuitry, which may include processing circuitry, and storage. Control circuitrymay be used to send and receive commands, requests, and other suitable data using I/O path, which may comprise I/O circuitry. I/O pathmay connect control circuitryto one or more communications paths.

1016 1016 1016 1020 1020 1016 Control circuitrymay be based on any suitable control circuitry such as one or more microprocessors, microcontrollers, digital signal processors, programmable logic devices, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), etc., and may include a multi-core processor (e.g., dual-core, quad-core, hexa-core, or any suitable number of cores) or supercomputer. In some embodiments, control circuitrymay be distributed across multiple separate processors or processing units, for example, multiple of the same type of processing units (e.g., two Intel Core i7 processors) or multiple different processors (e.g., an Intel Core i6 processor and an Intel Core i7 processor). In some embodiments, control circuitryexecutes instructions for an emulation system application stored in memory (e.g., the storage). Memory may be an electronic storage device provided as storagethat is part of control circuitry. Memory may store instruction to run the application.

1003 1003 1003 1003 1 9 FIGS.- 1 9 FIGS.- Data sourcemay include one or more types of content distribution equipment including a media distribution facility, satellite distribution facility, programming sources, intermediate distribution facilities and/or servers, internet providers, on-demand media servers, and other content providers. In some embodiments, the user equipment devices access the data sourceto receive stereoscopic media. In some approaches, data sourcemay be any suitable server configured to provide any information needed for operation of the user equipment devices as described above and below (e.g., in). For example, data sourcemay provide stereoscopic media, metadata associated with stereoscopic media, applications for executing functions and operation of user equipment devices, and/or any other suitable data needed for operations of user equipment devices (e.g., as described in).

1002 Although communications paths are not drawn between user equipment, these devices may communicate directly with each other via communications paths as well as other short-range, point-to-point communications paths, such as USB cables, IEEE 1394 cables, wireless paths (e.g., Bluetooth, infrared, IEEE 702-11x, etc.), or other short-range communication via wired or wireless paths. The user equipment may also communicate with each other directly through an indirect path via communication network.

The systems and methods described herein for enhancing playback of stereoscopic media may provide still other advantages. For example, the systems and methods present a low complexity viewer-side optimization that does not require sophisticated competition imaging techniques to create higher quality images for the viewer. Instead, as discussed above, enhanced quality is achieved by directing the higher-quality media channel to the viewer's dominant eye. This improvement has very low computational costs.

An additional advantage may be found when multiple viewers are watching the same stereoscopic media using HMDs. In such circumstances, the systems and methods described herein may provide each viewer with a high-quality viewing experience, regardless of which media channel has the higher-quality media.

The systems and methods described herein may also provide advantages to creators of stereoscopic media. During the creation process, the media may be produced without regard for the creator's dominant eye, as the systems and methods described herein may be used so that the higher-quality media channel is always mapped to the creator's dominant eye during the production process. In addition, the encoding of stereoscopic video can become less complicated or constrained because optimization of media viewing can occur at the time of viewing the stereoscopic media. Asymmetric compression techniques may therefore be used to further reduce the bit rate of the lower quality video channel, such as by reducing or eliminating quality parity between media channels, as the viewing experience is significantly determined by the quality of the higher-quality media channel.

In an embodiment which presents advantages on the creative side, when a creator reviews stereoscopic media, the systems and methods described herein may be leveraged to create a comparison between viewing the stereoscopic media with media channel swapping and viewing the stereoscopic media without media channel swapping. This comparison may additionally be performed with corresponding adjustments to spatial audio, so that the creator may ensure that the viewer is getting the highest quality viewing experience that can be provided for the created stereoscopic media. Additionally, the creator may create metadata to enable creative control of scenes within the stereoscopic media at the time of viewing. This level of creative control can help present the stereoscopic media to the viewer with a high-quality viewing experience and in a manner that most closely reflects the creator's vision for the creative work.

The processes discussed above are intended to be illustrative and not limiting. One skilled in the art would appreciate that the steps of the processes discussed herein may be omitted, modified, combined and/or rearranged, and any additional steps may be performed without departing from the scope of the invention. More generally, the above disclosure is meant to be illustrative and not limiting. Only the claims that follow are meant to set bounds as to what the present invention includes. Furthermore, it should be noted that the features and limitations described in any one embodiment may be applied to any other embodiment herein, and flowcharts or examples relating to one embodiment may be combined with any other embodiment in a suitable manner, done in different orders, or done in parallel. In addition, the systems and methods described herein may be performed in real time. It should also be noted that the systems and/or methods described above may be applied to, or used in accordance with, other systems and/or methods. Throughout the specification the phrases “in response to” and “based on” shall be understood to have a broad meaning unless context requires otherwise. For example, “in response to” can refer to a step that is in direct or indirect response to a prior step, and “based on” can refer to a step that is based at least in part on a prior step.