Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A method of generating 360 degree video in a server, the method comprising: obtaining image data captured by at least one camera; stitching the image data; mapping the image data onto a packed picture; encoding the packed picture; generating one or more media tracks for the packed picture; generating timed metadata track for specifying a sphere region, wherein the timed metadata track includes first information for specifying a first rotation value related to a center of the sphere region based on a yaw axis, second information for specifying a second rotation value related to the center of the sphere region based on a pitch axis, third information for specifying a first range related to the sphere region based on the yaw axis, and fourth information for specifying a second range related to the sphere region based on the pitch axis; generating track reference box having at least one track ID (identification), wherein the timed metadata track is linked to the one or more media tracks based on the track reference box; and transmitting at least one segment including the one or more media tracks, the timed metadata track, and the track reference box.
2. The method according to claim 1 , wherein the at least one segment is transmitted to a VR (Virtual Reality) display.
A system and method for transmitting segmented data to a virtual reality (VR) display addresses the challenge of efficiently rendering high-resolution VR content with minimal latency. The invention involves dividing a data stream into multiple segments, where each segment corresponds to a portion of the VR content. These segments are then transmitted to a VR display device, which reconstructs the full VR experience by combining the received segments. The segmentation process optimizes data transmission by reducing the amount of redundant information sent, thereby improving bandwidth efficiency and reducing latency. The VR display may include head-mounted displays (HMDs) or other immersive display systems. The method ensures seamless integration of segmented data, allowing for real-time rendering of high-quality VR content without noticeable delays or distortions. This approach is particularly useful in applications requiring low-latency, high-fidelity VR experiences, such as gaming, training simulations, and virtual environments. The invention enhances the performance of VR systems by dynamically adjusting segment size and transmission parameters based on network conditions and display requirements.
3. The method according to claim 1 , wherein the one or more media tracks, the timed metadata track, and the track reference box are included in an adaptation set of DASH (Dynamic Adaptive Streaming over HTTP).
This invention relates to adaptive streaming technology, specifically improving the organization and delivery of media content in Dynamic Adaptive Streaming over HTTP (DASH). The problem addressed is the efficient packaging and referencing of media tracks, timed metadata, and associated data structures within a DASH adaptation set to enhance streaming performance and compatibility. The method involves including one or more media tracks, a timed metadata track, and a track reference box within a DASH adaptation set. The media tracks contain the primary audio or video content, while the timed metadata track provides synchronized metadata such as subtitles, captions, or other time-aligned data. The track reference box establishes relationships between these tracks, ensuring proper synchronization and playback. This structured approach allows media players to dynamically select and switch between different media representations based on network conditions while maintaining metadata alignment. The adaptation set encapsulates these components, enabling seamless adaptation to varying bandwidths and device capabilities. By integrating the track reference box, the method ensures that metadata remains correctly associated with the corresponding media tracks, improving playback quality and user experience. This solution is particularly useful for streaming services requiring synchronized metadata delivery alongside adaptive media content.
4. The method according to claim 1 , wherein the one or more media tracks, the timed metadata track, and the track reference box are included in a PES (Packetized Elementary Stream) packet of an MPEG-2 TS (Transport Stream) or an adaptation field of a TS.
This invention relates to digital media streaming, specifically methods for packaging and transmitting media content with synchronized metadata in MPEG-2 Transport Streams (TS). The problem addressed is the efficient integration of timed metadata with media tracks in a standardized format to ensure synchronization and compatibility with existing streaming systems. The method involves embedding one or more media tracks, a timed metadata track, and a track reference box within a Packetized Elementary Stream (PES) packet of an MPEG-2 TS or within an adaptation field of a TS. The media tracks contain the primary content (e.g., video or audio), while the timed metadata track provides synchronized descriptive or control data. The track reference box establishes relationships between the media tracks and the metadata, ensuring proper synchronization during playback. This approach leverages existing MPEG-2 TS structures to avoid compatibility issues while enabling metadata-driven enhancements like interactive features or adaptive streaming. The solution is particularly useful in broadcast, over-the-top (OTT) streaming, and other applications requiring synchronized metadata delivery.
5. The method according to claim 1 , wherein the one or more media tracks, the timed metadata track, and the track reference box are included in an SEI (Supplemental Enhancement Layer) message of a VCL (Video Coding layer).
This invention relates to video coding and enhancement, specifically improving the integration of media tracks, timed metadata, and track references within video data. The problem addressed is the need for efficient and standardized ways to include supplemental information alongside video streams, ensuring compatibility and interoperability across different systems. The method involves embedding one or more media tracks, a timed metadata track, and a track reference box within an SEI (Supplemental Enhancement Layer) message of the VCL (Video Coding Layer). The media tracks carry the primary video or audio data, while the timed metadata track provides additional information synchronized with the media, such as captions or annotations. The track reference box establishes relationships between different tracks, enabling coordinated playback or processing. By incorporating these elements into an SEI message, the invention ensures that the supplemental data is seamlessly integrated into the video coding structure, maintaining synchronization and reducing overhead. This approach enhances flexibility in video processing, allowing for dynamic adjustments and improved error resilience. The solution is particularly useful in applications requiring rich media experiences, such as adaptive streaming or interactive video services.
6. A device for transmitting 360 degree video, comprising: an image acquisition unit for obtaining image data captured by at least one camera; a projection unit for stitching the image data and mapping the image data onto a packed picture; an encoder for encoding the packed picture; a controller for generating one or more media tracks for the packed picture, and generating timed metadata track for specifying a sphere region, wherein the timed metadata track includes first information for specifying a first rotation value related to a center of the sphere region based on a yaw axis, second information for specifying a second rotation value related to the center of the sphere region based on a pitch axis, third information for specifying a first range related to the sphere region based on the yaw axis, and fourth information for specifying a second range related to the sphere region based on the pitch axis, wherein the controller is further configured to generate track reference box having at least one track ID (identification), wherein the timed metadata track is linked to the one or more media tracks based on the track reference box; and a transmitter for transmitting at least one segment including the one or more media tracks, the timed metadata track, and the track reference box.
This invention relates to a device for transmitting 360-degree video, addressing the challenge of efficiently encoding and transmitting immersive video content. The device includes an image acquisition unit that captures image data from one or more cameras, which is then processed by a projection unit. The projection unit stitches the captured image data and maps it onto a packed picture format suitable for encoding. An encoder compresses the packed picture, while a controller generates media tracks for the encoded video and a timed metadata track. The metadata track specifies a sphere region within the 360-degree video, including rotation values (yaw and pitch) for the center of the region and range values defining the region's extent along both axes. The controller also creates a track reference box containing track IDs to link the metadata track with the media tracks. Finally, a transmitter sends at least one segment containing the media tracks, metadata track, and track reference box, enabling efficient transmission and playback of 360-degree video with dynamic region specification. This approach ensures that only relevant portions of the immersive video are processed or displayed, optimizing bandwidth and computational resources.
7. A digital receiver for processing 360 degree video, comprising: a receiving module configured to receive at least one segment including one or more media tracks related to the 360 degree video, timed metadata track for specifying a sphere region and track reference box having at least one track ID (identification), wherein the timed metadata track includes first information for specifying a first rotation value related to a center of the sphere region based on a yaw axis, second information for specifying a second rotation value related to the center of the sphere region based on a pitch axis, third information for specifying a first range related to the sphere region based on the yaw axis, and fourth information for specifying a second range related to the sphere region based on the pitch axis, wherein the timed metadata track is linked to the one or more media tracks based on the track reference box; a decoder configured to decode the 360 degree video; and a display module configured to display the 360 degree video.
A digital receiver processes 360-degree video by receiving at least one segment containing media tracks and a timed metadata track. The timed metadata track specifies a spherical region of the 360-degree video, including rotation values for yaw and pitch axes, and range values for yaw and pitch axes. The metadata track is linked to the media tracks via a track reference box containing track identification. The receiver decodes the 360-degree video and displays it, allowing dynamic adjustment of the viewable region based on the metadata. This system enables efficient processing and rendering of immersive video content by defining and referencing specific spherical regions within the 360-degree video. The metadata ensures proper alignment and display of the video segments, enhancing user experience in virtual reality or panoramic viewing applications. The receiver's modular design supports seamless integration with existing video processing pipelines while maintaining precise control over the displayed content.
8. A method for processing 360 degree video in a digital receiver, comprising: receiving at least one segment including one or more media tracks related to the 360 degree video, timed metadata track for specifying a sphere region and track reference box having at least one track ID (identification), wherein the timed metadata track includes first information for specifying a first rotation value related to a center of the sphere region based on a yaw axis, second information for specifying a second rotation value related to the center of the sphere region based on a pitch axis, third information for specifying a first range related to the sphere region based on the yaw axis, and fourth information for specifying a second range related to the sphere region based on the pitch axis, wherein the timed metadata track is linked to the one or more media tracks based on the track reference box; decoding the 360 degree video; and displaying the 360 degree video.
This invention relates to processing 360-degree video in a digital receiver. The technology addresses the challenge of efficiently rendering and displaying immersive 360-degree video content by dynamically adjusting the viewable region of a spherical video based on metadata. The method involves receiving at least one segment containing one or more media tracks associated with the 360-degree video, along with a timed metadata track. The metadata track specifies a sphere region using rotation values for yaw and pitch axes, as well as range values defining the viewable area along these axes. The metadata track is linked to the media tracks via a track reference box containing track identification information. The digital receiver decodes the 360-degree video and displays it, applying the metadata to dynamically adjust the viewable portion of the spherical video. This approach enables real-time adaptation of the displayed region, enhancing user experience by focusing on relevant content within the immersive video. The system ensures synchronization between the video and metadata, allowing seamless navigation and interaction with the 360-degree content.
Unknown
June 16, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.