An apparatus and system to provide QoE metrics reporting mechanisms for RTP-based 360-degree video delivery in live immersive streaming and real-time immersive conversational service applications are described for both in-camera and network-based stitching. Initial and desired parameters for viewports used in a teleconference are exchanged, and the teleconference established using 360° media. RTP FoV reports sent during the teleconference each contain viewport orientation information, as well as information for the QoE metrics.
Legal claims defining the scope of protection, as filed with the USPTO.
6. The apparatus of claim 2, wherein the renderer is configured to use decoder metadata information contained in Supplemental Information Enhancement (SEI) messages contained in the elementary stream to render video and audio signals of the 3D video with a pose and a knowledge of a horizontal and vertical field of view to determine the viewport.
7. The apparatus of claim 6, wherein the renderer is configured to use the decoder metadata information for region-wise unpacking, projection de-mapping and rotation toward creating spherical content for each eye of a user. The VR Renderer uses the decoded signals and rendering metadata.
8. The apparatus of claim 2, wherein the processing circuitry is configured to control the RTP receiver, the HVEC decoder, the texture-to-sphere mapper, and the renderer based on media control information, dynamic user pose, and display and device capabilities using a display resolution, a maximum display refresh rate, a field of view in both horizontal and vertical directions, an eye-to-screen distance, a lens separation distance and operating system support.
9. The apparatus of claim 1, wherein the processing circuitry is further configured to negotiate Vital Product Data (VPD) capability, and the viewport switching latency metric includes latency and quality-related factors when viewport movement causes quality degradations, the latency and quality-related factors including viewport quality within a viewport.
10. The apparatus of claim 9, wherein the viewport quality is represented by a quality ranking (QR) value and a pixel resolution of one or more regions within the viewport.
11. The apparatus of claim 10, wherein the processing circuitry is further configured to determine whether the viewport has a comparable quality to another viewport, in response to more than one quality ranking region being visible inside the viewport, aggregated viewport quality factors are calculated as an area-weighted average QR and an area-weighted pixel resolution.
12. The apparatus of claim 11, wherein in response to movement of the viewport so that the viewport includes at least one new quality ranking region, the processing circuitry is configured to start a switch event.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
July 29, 2021
May 21, 2024
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.