Systems and methods are provided for encoding raw image content encoded using one or more virtual intra-frames. In an exemplary method, a stream of video content may be encoded to generate compressed video data that includes an intra-frame and a plurality of corresponding inter-frames. The compressed video data may be stored within a buffer, and when the amount of data in the buffer exceeds a threshold value, a virtual intra-frame may be computed based on decoded inter-frame data. The virtual intra-frame may be output for storage in the buffer.
Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A computer-implemented method, comprising: receiving a stream of video content; encoding, using a video codec, the video content stream to generate compressed video data, the compressed video data comprising an intra-frame and a plurality of corresponding inter-frames; and storing the compressed video data within a buffer, wherein the encoding comprises: obtaining intra-frame data associated with the encoded intra-frame; encoding, based on the intra-frame data, a portion of the video content stream to generate a corresponding one of the inter-frames; determining whether an amount of compressed data stored in the buffer exceeds a first threshold value; when the amount of compressed data in the buffer exceeds the first threshold value, decoding the last generated inter-frame in the buffer and computing a viral intra-frame based on the decoded inter-frame data, the virtual intra-frame representing an out of sequence intra-frame in a group-of-pictures; and outputting the virtual intra-frame with the last generated inter-frame for storage in the buffer.
A computer system encodes video by compressing it into intra-frames (I-frames) and inter-frames (P-frames). This compressed data is stored in a buffer. The system checks if the buffer is full (exceeds a threshold). If it is, the system decodes the most recently added P-frame, generates a "virtual I-frame" (an I-frame created from a P-frame) from this decoded P-frame, and stores this virtual I-frame and the original P-frame in the buffer. The virtual I-frame is out of sequence in the Group of Pictures.
2. The method of claim 1 , wherein the encoding further comprises encoding a first portion of the video content stream to generate the intra-frame, the first portion preceding the portion of the video content stream associated with the inter-frame.
In addition to encoding video into intra-frames (I-frames) and inter-frames (P-frames) and storing compressed data in a buffer as described above, the initial I-frame is generated from a first portion of the video content stream, and the subsequent P-frame is encoded based on a later portion of the video stream that follows the I-frame's segment, thereby creating a temporal dependency for video decompression.
3. The method of claim 1 , wherein the encoding further comprises; computing a group-of-pictures length associated with the compressed video data, when the amount of buffered data fails to exceed the first threshold value; determining whether the computed group-of-pictures length exceeds a predetermined threshold length; and encoding an additional portion of the video content stream to generate an additional inter-frame, when the computed group-of-pictures length does not exceed the predetermined threshold length.
After encoding video into intra-frames (I-frames) and inter-frames (P-frames) and storing compressed data in a buffer as described above, when the buffer isn't full (doesn't exceed a threshold), the system calculates the current length of the "group of pictures" (GOP) based on the compressed video data. If this GOP length is shorter than a set maximum, the system encodes another P-frame and adds it to the compressed video data.
4. The method of claim 3 , wherein: the buffer comprises one or more previously-encoded groups of intra-frames and corresponding inter-frames; and when the computed group-of-pictures length exceeds the predetermined threshold length, discarding the previously-encoded groups of intra-frames and corresponding inter-frames.
This invention describes a method for managing video data in a buffer during encoding. A video stream is received and encoded by a video codec, producing compressed video data consisting of an initial intra-frame (I-frame) and a series of corresponding inter-frames (P/B-frames). This compressed data is stored in a buffer. During encoding, if the amount of compressed data in the buffer exceeds a *first threshold*, the most recently generated inter-frame is decoded. A *virtual intra-frame* (V-frame) is then computed from this decoded inter-frame data. This V-frame serves as an out-of-sequence I-frame within a group-of-pictures (GOP) and is subsequently outputted with the last inter-frame for storage in the buffer. However, if the buffer's data amount *does not exceed* the *first threshold*, the system computes the length of the current group-of-pictures (GOP). If this computed GOP length *does not exceed* a *predetermined threshold length*, additional inter-frames are encoded. Furthermore, the buffer is designed to hold multiple previously-encoded groups of intra-frames and their corresponding inter-frames. If the computed group-of-pictures length (as determined when the buffer data is below the *first threshold*) *exceeds* the *predetermined threshold length*, then these previously-encoded groups of intra-frames and their corresponding inter-frames are discarded from the buffer. This mechanism helps manage buffer space by removing older GOPs when the current GOP becomes too large.
5. The method of claim 1 , wherein the encoding further comprises determining whether the amount of buffered data exceeds a second threshold value, the second threshold value being larger than the first threshold value.
The video encoding system that compresses video into I-frames and P-frames, and stores this in a buffer monitors how full the buffer is. It uses two thresholds. The system checks if the buffer exceeds a "first threshold value" to generate a virtual intra-frame (as described in Claim 1). It also checks if the buffer exceeds a "second threshold value" which is larger than the first, presumably to trigger a different action based on a higher level of buffer fullness.
6. The method of claim 5 , wherein the encoding further comprises maintaining the virtual intra-frame and discarding encoding video content preceding the virtual intra-frame, when the amount of buffered data exceeds the second threshold value.
In the described video encoding system using dual buffer thresholds, if the buffer exceeds the larger, "second threshold value" the virtual I-frame is retained, and all video content encoded prior to that virtual I-frame is discarded from the buffer. This action ensures that the buffer only contains recent video data, with the virtual I-frame providing a starting point for decoding.
7. The method of claim 1 , further comprising: receiving information indicative of a triggering incident; and generating, based on the received information, an instruction to transmit at least a portion of the buffered data to a recipient.
The described video encoding system also includes the ability to respond to external events. The system can receive information signaling a "triggering incident," and based on that information, it generates a command to send a portion of the buffered video data to a recipient.
8. The method of claim 7 , wherein the triggering incident comprises at least one of a request for video content, an occurrence of a predetermined event within the received video stream, a receipt of predetermined data through an interface, or a detection of a predetermined event by a sensor.
In the event-driven video system, the "triggering incident" that causes the system to send video data can be any of the following: a direct request for the video content, the detection of a specific event within the video stream itself, receiving specific data via an interface, or detecting an event through an attached sensor.
9. The method of claim 7 , wherein generating the instruction further comprises: determining whether the buffered data includes the intra-frame and the virtual intra-frame; discarding the virtual intra-frame and maintaining the corresponding inter-frame, when the buffered data includes the intra-frame and the virtual intra-frame; and generating the buffered data portion based on the intra-frame.
When a "triggering incident" occurs, the video data transmission mechanism analyzes the buffer content: If the buffer contains both a normal I-frame and a virtual I-frame, the virtual I-frame is discarded, the original P-frames are kept, and the portion of data sent is created starting from the original I-frame. This prioritizes standard I-frame decoding.
10. The method of claim 7 , further comprising: determining whether the buffered data includes the virtual intra-frame and fails to include the intra-frame; and generating the buffered data portion based on the virtual intra-frame, when the buffered data includes the virtual intra-frame and fails to include the intra-frame.
When a "triggering incident" occurs, the video data transmission mechanism analyzes the buffer content: If the buffer ONLY contains a virtual I-frame (meaning the original I-frame is missing), the portion of data transmitted is created starting from the virtual I-frame. This allows transmission even without the original I-frame, leveraging the virtual I-frame for decoding.
11. The method of claim 1 , further comprising: receiving at least one frame of compressed video data; decoding the received frame; determining whether the decoded frame corresponds to an intra-frame; and storing at least the decoded frame in a data repository, when the decoded frame corresponds to an intra-frame.
The system can also receive and decode compressed video data. It determines if a received frame is an I-frame. If the received frame is identified as an I-frame, it stores at least this decoded I-frame in a data repository for later use.
12. The method of claim 11 , further comprising: determining that the decoded frame corresponds to an inter-frame; when the decoded frame is an inter-frame, computing a group-of-pictures length associated with the received data; determining whether the computed group-of-pictures length exceeds a predetermined threshold length; and storing at least the decoded inter-frame within the data repository, when the computer group-of-pictures length fails to exceed the predetermined threshold length.
This extension to the video receiving and decoding process examines inter-frames (P-frames). When a received frame is determined to be a P-frame, the system calculates a GOP (group of pictures) length. If this GOP length is within the threshold, the decoded P-frame is stored in the data repository.
13. The method of claim 12 , further comprising: generating an additional intra-frame based on the decoded inter-frame data, when the computed group-of-pictures length exceeds the predetermined threshold length; and storing at least the additional intra-frame within the data repository.
This extends the P-frame handling process. If, when processing a P-frame, the calculated GOP length exceeds a predetermined threshold, an additional I-frame is generated based on the decoded P-frame data and this newly created I-frame is stored in the data repository. This allows the system to create keyframes as needed.
14. An apparatus, comprising: a video codec configured to encode a stream of video content to generate compressed video data, the compressed video data comprising an intra-frame and a plurality of corresponding inter-frames; and a storage device couple to the video codec unit and configured to store the compressed video data within a buffer, wherein the video codec is further configured to: obtain intra-frame data associated with the encoded intra-frame; encode, based on the intra-frame data, a portion of the video content stream to generate a corresponding one of the inter-frames; determine whether an amount of compressed data stored in the buffer exceeds a first threshold value; when the amount of compressed data in the buffer exceeds the first threshold value, decode the last generated inter-frame in the buffer and compute a virtual intra-frame based on the decoded inter-frame data, the virtual intra-frame representing an out of sequence intra-frame in a group-of-pictures; and provide the virtual intra-frame with the last generated intra-frame as output for storage in the buffer.
A video encoding apparatus includes a video codec and storage. The video codec encodes video into I-frames and P-frames, which are stored in a buffer on the storage device. The codec analyzes the buffer fullness. If the buffer usage exceeds a threshold, the last P-frame added is decoded. The codec then creates a virtual I-frame from the decoded P-frame and outputs both the virtual I-frame and the last P-frame for storage in the buffer. The virtual I-frame represents an out of sequence intra-frame in the Group of Pictures.
15. The apparatus of claim 14 , wherein the video codec is further configured to: encode a first portion of the video content stream to generate the intra-frame, the first portion preceding the portion of the video content stream associated with the inter-frame.
The video encoding apparatus as described above creates the initial I-frame from the beginning of a video stream, and then encodes subsequent P-frames based on later portions of the video stream. This ensures that I-frames are created first and the following P-frames are encoded based on the I-frame.
16. The apparatus of claim 14 , wherein the video codec is further configured to: compute a group-of-pictures length associated with the compressed video data, when the amount of buffered data fails to exceed the first threshold value; determine whether the computed group-of-pictures length exceeds a predetermined threshold length; and encode an additional portion of the video content stream to generate an additional inter-frame, when the computed group-of-pictures length fails to exceed the predetermined threshold length.
The video encoding apparatus as described above, when the buffer isn't full (doesn't exceed a threshold), calculates the current length of the "group of pictures" (GOP) based on the compressed video data. If this GOP length is shorter than a set maximum, the apparatus encodes another P-frame and adds it to the compressed video data.
17. The apparatus of claim 16 , wherein: the buffer comprises one or more previously-encoded groups of intra-frames and corresponding inter-frames; and when the computed group-of-pictures length exceeds the predetermined threshold length, the video codec is further configured to discard the previously-encoded groups of inter-frames and corresponding inter-frames.
The apparatus uses a buffer to store video data encoded in groups of I-frames and P-frames. When the calculated group of pictures (GOP) length exceeds a predetermined maximum, older groups of I-frames and P-frames are discarded.
18. The apparatus of claim 14 , wherein the video codec is further configured to: determine whether the amount of buffered data exceeds a second threshold value, the second threshold value being larger than the first threshold value.
This video encoding apparatus from above manages video encoding by tracking how full the buffer is. It uses two thresholds. The codec checks if the buffer exceeds a "first threshold value" to generate a virtual intra-frame (as described in Claim 14). It also checks if the buffer exceeds a "second threshold value" which is larger than the first.
19. The apparatus of claim 18 , wherein the vide coded is further configured to: maintain the virtual intra-frame and discard encoded video content preceding the virtual intra-frame, when the amount of buffered data exceeds the second threshold value.
The described video encoding apparatus using dual buffer thresholds, if the buffer exceeds the larger, "second threshold value" the virtual I-frame is maintained, and all video content encoded prior to that virtual I-frame is discarded from the buffer. This ensures that the buffer only contains recent video data, with the virtual I-frame providing a starting point for decoding.
20. The apparatus of claim 14 , wherein the video codec is further configured to: receive information indicative of a triggering incident; and generate, based on the received information, an instruction to transmit at least a portion of the buffered data to a recipient.
The video encoding apparatus also incorporates external events. The system can receive information signaling a "triggering incident," and the video codec, based on that information, generates a command to send a portion of the buffered video data to a recipient.
21. The apparatus of claim 20 , wherein the video codec is further configured to: determine that the buttered data includes the intra-frame and the virtual intra-frame; discard the virtual intra-frame and maintain the corresponding inter-frame, when the buffered data includes the intra-frame and the virtual intra-frame; and generate the buffered data portion based on the intra-frame.
The apparatus that sends video based on an event first checks the buffer: If the buffer has both an I-frame and a virtual I-frame, the virtual I-frame is removed, the original P-frames are kept, and the data is created based on the original I-frame. This allows the system to ensure that standard I-frame decoding is prioritized.
22. The apparatus of claim 20 , wherein the video is codec is further configured to: determine whether the buffered data includes the virtual intra-frame and fails to include the intra-frame; and generate the buffered data portion based on the virtual intra-frame, when the buffered data includes the virtual intra-frame and fails to include the intra-frame.
The apparatus that sends video based on an event, if the buffer ONLY contains a virtual I-frame (the original I-frame is missing), generates the data to send starting from the virtual I-frame. The system then allows the transmission of video using the virtual I-frame for decoding.
23. The apparatus of claim 14 , wherein the video codec is further configured to: receive at least one frame of compressed video data; decode the received frame; determine whether the decoded frame corresponds to an intra-frame; and generate an instruction to store at least the decoded frame in the storage device, when decoded frame corresponds to an intra-frame.
The video codec of the apparatus can also receive compressed video data, determines if a received frame is an I-frame, and, if it is, instructs the storage device to store this decoded I-frame.
24. The apparatus of claim 23 , wherein the video codec is further configured to: determine that the decoded frame corresponds to an inter-frame; when the decoded frame is an inter-frame, compute a group-of-pictures length associated with the received data; determine whether the computed group-of-pictures length exceeds a predetermined threshold length; and generate an instruction to store at least the decoded inter-frame within the storage device, when the computed group-of-pictures length fails to exceed the predetermined threshold length.
When the codec receives a P-frame, the video codec calculates the GOP length. If the GOP length is within a threshold, the codec instructs the storage device to store the decoded P-frame.
25. The apparatus of claim 24 , wherein the video codec is further configured to: generate an additional intra-frame based on the decoded inter-frame data, when the computed group-of-pictures length exceeds the predetermined threshold length; and generate an instruction to store at least the additional intra-frame within the storage device.
In the apparatus when the calculated GOP length exceeds a threshold upon receiving a P-frame, the video codec generates a new I-frame from that P-frame's data, and instructs the storage device to store this new I-frame.
26. A tangible, non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform the steps of: encoding a stream of video content to generate compressed video data, the compressed video data comprising an intra-frame and a plurality of corresponding inter-frames; and storing the compressed video data within a buffer, wherein the encoding comprises: obtaining intra-frame data associated with the encoded intra-frame; encoding, based on the intra-frame data, a portion of the video content stream to generate a corresponding one of the inter-frames; determining whether an amount of compressed data stored in the buffer exceeds a first threshold value; when the amount of compressed data in the buffer exceeds the first threshold value, decoding the last generated inter-frame in the buffer and computing a virtual intra-frame based on the decoded inter-frame data, the virtual intra-frame representing an out of sequence intra-frame in a group-of-pictures; and outputting the virtual intra-frame with the last generated inter-frame for storage in the buffer.
A computer-readable medium stores instructions for encoding video into I-frames and P-frames. This compressed data is stored in a buffer. The system checks if the buffer is full (exceeds a threshold). If it is, the system decodes the most recently added P-frame, generates a "virtual I-frame" (an I-frame created from a P-frame) from this decoded P-frame, and stores this virtual I-frame and the original P-frame in the buffer. The virtual I-frame is out of sequence in the Group of Pictures.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
June 26, 2013
November 7, 2017
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.