In various implementations, provided are techniques, as well as systems that implement these techniques, for SEI messages that distinguish “essential” information from “non-essential” information. In various implementations, an encoder can be configured to determine whether information associated with video data is essential or is not essential. In various implementations, different types of SEI messages can be defined, where one or more of these types of SEI messages can indicate whether information associated with a particular video is essential or not essential. Essential information is required for presenting the video data, and is not required for decoding the video data.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A method of encoding, by an encoding device implemented in circuitry, video data, comprising: obtaining the video data to encode; determining, based on criteria implemented by the encoding device, whether any supplemental enhancement information (SEI) message associated with processing content of a coded video sequence (CVS) of a decoded video bitstream is considered essential to enable a desired user experience, wherein the decoded video bitstream would result from decoding of an encoded video bitstream associated with the video data; based on a determination that a set of types of SEI messages within the CVS are considered, by the encoding device, to be essential: generating a SEI message that includes information that indicates that the set of types of SEI messages within the CVS is determined, by the encoding device, to be essential to process content of the CVS to enable a desired user experience, where the set of types of SEI messages do not affect decoding of the encoded video bitstream; generating, for inclusion within the SEI message, a first syntax element indicative of a number of types of SEI messages included within the set of types of SEI messages; for each type of SEI message included within the set of types of SEI messages, generating, for inclusion within the SEI message, a second syntax element indicative of a type of SEI message; including, within the SEI message, the first syntax element and the second syntax element; encoding the video data to generate the encoded video bitstream; and transmitting the encoded video bitstream and the SEI message.
2. The method of claim 1 , further comprising including, based on a requirement, the SEI message within a first access unit of the CVS.
3. The method of claim 1 , further comprising including, based on a requirement, the SEI message in a current access unit and including the SEI message in each access unit from the current access unit to an end of the CVS.
4. The method of claim 1 , further comprising defining, based on input received from a content producer, the criteria implemented by the encoding device.
5. The method of claim 1 , further comprising: encapsulating the encoded video bitstream for streaming, wherein the encoded video bitstream is encapsulated according to a streaming format; and generating, according to the streaming format, a description of encoded video bitstream, wherein the description includes the SEI message.
6. The method of claim 1 , further comprising: encapsulating the encoded video bitstream for transmission over a network, wherein the encoded video bitstream is encapsulated according to a container format; generating, according to the container format, a first packet, wherein a payload portion of the first packet includes the SEI message as a descriptor; and generating one or more packets including the encoded video bitstream, wherein the one or more packets follow the first packet in transmission order.
7. The method of claim 1 , further comprising: encapsulating the encoded video bitstream for transmission over a network, wherein the encoded video bitstream is encapsulated according to a real-time transfer protocol; and generating, according to a session description protocol, a session description, wherein the session description includes the SEI message.
8. An encoding device, comprising: a memory configured to store video data to encode; and a processor implemented in circuitry and in communication with the memory, the processor configured to: determine, based on criteria implemented by the encoding device, whether any supplemental enhancement information (SEI) message associated with processing content of a coded video sequence (CVS) of a decoded video bitstream is considered essential to enable a desired user experience, wherein the decoded video bitstream would result from decoding of an encoded video bitstream associated with the video data; based on a determination that a set of types of SEI messages within the CVS are considered, by the encoding device, to be essential: generate a SEI message that includes information that indicates that the set of types of SEI messages within the CVS is determined, by the encoding device, to be essential to process content of the CVS to enable a desired user experience, where the set of types of SEI messages do not affect decoding of the encoded video bitstream; generate, for inclusion within the SEI message, a first syntax element indicative of a number of types of SEI messages included within the set of types of SEI messages; for each type of SEI message included within the set of types of SEI messages, generate, for inclusion within the SEI message, a second syntax element indicative of a type of SEI message; include, within the SEI message, the first syntax element and the second syntax element; encode the video data to generate the encoded video bitstream; and transmit the encoded video bitstream and the SEI message.
9. The encoding device of claim 8 , wherein the processor is further configured to include, based on a requirement, the SEI message within a first access unit of the CVS.
10. The encoding device of claim 8 , wherein the processor is further configured to include, based on a requirement, the SEI message in a current access unit and include the SEI message in each access unit from the current access unit to an end of the CVS.
11. The encoding device of claim 8 , where the processor is further configured to define, based on input received from a content producer, the criteria implemented by the encoding device.
12. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to: receive video data to encode; determine, based on criteria implemented by the encoding device, whether any supplemental enhancement information (SEI) message associated with processing content of a coded video sequence (CVS) of a decoded video bitstream is considered essential to enable a desired user experience, wherein the decoded video bitstream would result from decoding of an encoded video bitstream associated with the video data; based on a determination that a set of types of SEI messages within the CVS are considered, by the encoding device, to be essential: generate a SEI message that includes information that indicates that the set of types of SEI messages within the CVS is determined, by the encoding device, to be essential to process content of the CVS to enable a desired user experience, where the set of types of SEI messages do not affect decoding of the encoded video bitstream; generate, for inclusion within the SEI message, a first syntax element indicative of a number of types of SEI messages included within the set of types of SEI messages; for each type of SEI message included within the set of types of SEI messages, generate, for inclusion within the SEI message, a second syntax element indicative of a type of SEI message; include, within the SEI message, the first syntax element and the second syntax element; encode the video data to generated the encoded video data; and transmit the encoded video bitstream and the SEI message.
13. A method of decoding video data, comprising: receiving, at a decoding device, an encoded video bitstream and a supplemental enhancement information (SEI) message associated with the encoded video bitstream; processing the SEI message that includes information that indicates that a set of types of SEI messages within a coded video sequence (CVS) of the encoded video bitstream is determined, by an encoding device of the encoded video bitstream, to be essential to process content of the CVS to enable a desired user experience, where the set of types of SEI messages do not affect decoding of the encoded video bitstream; receiving, within the SEI message, a first syntax element indicative of a number of types of SEI messages included within the set of types of SEI messages; determining, based on the first syntax element, the number of types of SEI messages included within the set of types of SEI messages; responsive to the determination of the number of types of SEI messages included within the set of types of SEI messages, for each type of SEI message included within the set of types of SEI messages, receiving a second syntax element indicative of a type of SEI message; determining that each type of SEI message among the set of types of SEI messages have been determined, by the encoding device, to be essential to process content of the CVS to enable the desired user experience; and decoding the encoded video bitstream to generate decoded video data.
14. The method of claim 13 , further comprising receiving the SEI message in a first access unit of the CVS.
15. The method of claim 13 , further comprising receiving the SEI message in a current access unit and in each access unit from the current access unit to an end of the CVS.
16. The method of claim 13 , further comprising, based on content of the SEI message, determining whether the decoding device includes system-level processing entities configured to process the CVS to provide enable the desired user experience.
17. The method of claim 13 , wherein the encoded video bitstream and the SEI message associated with the encoded video bitstream are received in a file, wherein the file is formatted according to a file format, and wherein, according to the file format, the SEI message is in a part of the file that is read by the decoding device before a part of the file that includes the encoded video bitstream.
18. The method of claim 13 , wherein the encoded video bitstream and the SEI message associated with the encoded video bitstream are received in a data stream, wherein the data stream is formatted according to a streaming format; and reading, from the data stream, a description of encoded video bitstream, wherein the description includes the SEI message.
19. The method of claim 13 , wherein the encoded video bitstream and the SEI message associated with the encoded video bitstream are received in a plurality of network packets; reading the SEI message from a payload portion of a first packet from the plurality of network packets; and reading the encoded video bitstream from one or more packets from the plurality of network packets, wherein the one or more packets follow the first packet in transmission order.
20. The method of claim 13 , wherein the encoded video bitstream and the SEI message associated with the encoded video bitstream are received over a network, wherein the encoded video bitstream is encapsulated according to a real-time transfer protocol, wherein the SEI message is included in a session description, wherein the session description is formatted according to a session description protocol.
21. The method of claim 13 further comprising processing the decoded video data based on each type of SEI message included in the set of types of SEI messages.
22. A decoding device, comprising: a memory configured to store an encoded video bitstream and a supplemental enhancement information (SEI) message associated with the encoded video bitstream; and a processor implemented in circuitry and in communication with the memory, the processor configured to: process the SEI message that includes information that indicates that a set of types of SEI messages within a coded video sequence (CVS) of the encoded video bitstream is determined, by an encoding device of the encoded video bitstream, to be essential to process content of the CVS to enable a desired user experience, where the set of types of SEI messages do not affect decoding of the encoded video bitstream; receive, within the SEI message, a first syntax element indicative of a number of types of SEI messages included within the set of types of SEI messages; determine, based on the first syntax element, the number of types of SEI messages included within the set of types of SEI messages; responsive to the determination of the number of types of SEI messages included within the set of types of SEI messages, for each type of SEI message included within the set of types of SEI messages, receive a second syntax element indicative of a type of SEI message; determine that each type of SEI message among the set of types of SEI messages have been determined, by the encoding device, to be essential to process content of the CVS to enable the desired user experience; decode the encoded video bitstream to generate decoded video data.
23. The decoding device of claim 22 , wherein the processor is further configured to receive the SEI message in a first access unit of the CVS.
24. The decoding device of claim 22 , wherein the processor is further configured to receive the SEI message in a current access unit and in each access unit from the current access unit to an end of the CVS.
25. The decoding device of claim 22 , wherein the decoding device is further configured to, based on content of the SEI message, determine whether the decoding device includes system-level processing entities configured to process the CVS to provide enable the desired user experience.
26. The decoding device of claim 22 wherein the processor is further configured to process the decoded video data based on each type of SEI message included in the set of types of SEI messages.
27. A non-transitory, computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to: receive an encoded video bitstream and a supplemental enhancement information (SEI) message associated with the encoded video bitstream; process the SEI message that includes information that indicates that a set of types of SEI messages within a coded video sequence (CVS) of the encoded video bitstream is determined, by an encoding device of the encoded video bitstream, to be essential to process content of the CVS to enable a desired user experience, where the set of types of SEI messages do not affect decoding of the encoded video bitstream; receive, within the SEI message, a first syntax element indicative of a number of types of SEI messages included within the set of types of SEI messages; determine, based on the first syntax element, the number of types of SEI messages included within the set of types of SEI messages; responsive to the determination of the number of types of SEI messages included within the set of types of SEI messages, for each type of SEI message included within the set of types of SEI messages, receive a second syntax element indicative of a type of SEI message; determine that each type of SEI message among the set of types of SEI messages have been determined, by the encoding device, to be essential to process content of the CVS to enable the desired user experience; decode the encoded video bitstream to generate decoded video data.
28. The non-transitory, computer-readable storage medium of claim 27 , further comprising instructions that, when executed, cause the one or more processors to process the decoded video data based on each type of SEI message included in the set of types of SEI messages.
29. An apparatus, comprising: means for receiving an encoded video bitstream and a supplemental enhancement information (SEI) message associated with the encoded video bitstream; means for processing the SEI message that includes information that indicates that a set of types of SEI messages within a coded video sequence (CVS) of the encoded video bitstream is determined, by an encoding device of the encoded video bitstream, to be essential to process content of the CVS to enable a desired user experience, where the set of types of SEI messages do not affect decoding of the encoded video bitstream; means for receiving, within the SEI message, a first syntax element indicative of a number of types of SEI messages included within the set of types of SEI messages; means for determining, based on the first syntax element, the number of types of SEI messages included within the set of types of SEI messages; responsive to the determination of the number of types of SEI messages included within the set of types of SEI messages, for each type of SEI message included within the set of types of SEI messages, means for receiving a second syntax element indicative of a type of SEI message; means for determining that each type of SEI message among the set of types of SEI messages have been determined, by the encoding device, to be essential to process content of the CVS to enable the desired user experience; means for decoding the encoded video bitstream to generate decoded video data.
30. The apparatus of claim 29 further comprising means for processing the decoded video data based on each type of SEI message included in the set of types of SEI messages.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
March 20, 2018
June 30, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.