A video transcoding method is provided for transcoding a first signal stream compressed by a first coding scheme to a second signal stream compressed by a second coding scheme. The method employs an adaptive frame rate and a joint temporal-spatial rate control technique, such that the overall quality of compressed MPEG video can be significantly enhanced when the transcoding is controlled in the joint temporal (picture or frame rate) and spatial (quantization) domains. One embodiment considers transcoding from high bit rate video with larger image size (e.g. 4CIF/4SIF, CIF) coded by one coding technique, e.g., MPEG-2 to a lower bit rate video with smaller image size (e.g. CIF, QCIF) coded by the same or another coding technique, e.g., MPEG-4.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A method for transcoding an input compressed video signal stream to an output compressed signal stream comprising: a) performing adaptive frame rate transcoding for a sub-GOP (Group of Pictures), said sub-GOP comprising one or more types of frames of said input compressed signal stream to be transcoded; and, b) using a joint temporal-spatial rate distortion optimization within said sub-GOP to adjust the output rate of the compressed output signal stream, wherein for said sub-GOP, said joint temporal-spatial rate distortion optimization comprises: determining which frames of said sub-GOP to be either skipped or transcoded; computing a rate-distortion contribution resulting from each transcoded frame and from each skipped frame; and, minimizing a total distortion for a target rate of said compressed output signal stream based on said transcoded frame and skipped frame rate-distortion contributions.
2. The method of claim 1 , wherein a frame set S is defined to indicate certain frames to be either skipped or transcoded, said set S=[S 1 , S 2 , . . . S N ), S i ε[0,1], I=1, . . . N, with 0 denoting a frame to be skipped and 1 denoting a frame to be transcoded and N denoting a total number of frames in said sub-GOP, said adaptive frame rate transcoding further comprising: determining a status of a frame in said sub-GOP based on its corresponding value in set S.
3. The method of claim 2 , wherein the signal stream is a video stream compressed pursuant to the standards of the Moving Picture Experts Group (MPEG), and within a Group of Pictures (GOP) three types of pictures are distinguished according to the compression method which is used, Intra-mode pictures (I pictures) which are compressed independently of any other picture, Predictively motion-compensated pictures (P pictures) which are reconstructed from the compressed data in that picture and two most recently reconstructed fields from previously displayed I or P pictures, and Bidirectionally motion-compensated pictures (B pictures) which are reconstructed from the compressed data in that picture plus two reconstructed fields from previously displayed I or P pictures and two reconstructed fields from I or P pictures that will be displayed in the future, wherein I and P pictures are termed reference pictures because they are used to reconstruct other pictures.
4. The method of claim 2 , further including transcoding all reference I and P pictures because of the reuse of motion vectors and to maintain the prediction sequential order, while not transcoding and skipping some or all B pictures that are non-reference pictures.
5. The method of claim 4 , wherein skipped B pictures are reconstructed at the decoder to ensure a full frame rate playback.
6. The method of claim 3 , further including: determining whether a picture is an I picture, a B picture or a P picture type from the picture header information; transcoding all I pictures; and if a series of pictures comprises a (B, P) pair, which starts with a B picture, and having zero or more intermediate B pictures, and ends with a P picture, applying adaptive picture rate transcoding with joint temporal-spatial rate distortion optimization to the series of pictures of the (B, P) pair.
7. The method of claim 6 , wherein the adaptive picture rate transcoding with joint temporal-spatial rate distortion optimization comprises the following steps: a. transcoding a P picture to obtain its rate denoted by R(P), which is the bit count consumed by transcoding the P picture, and its distortion denoted by D(P), such as PNR (Peak Signal-to-Noise Ratio) or SAD (Sum of Absolute Differences), in comparison with the input down-sampled video picture; b. transcoding the following B picture to obtain its rate denoted by R(B), which is the bit count consumed by transcoding the B picture, and its distortion denoted by D(B), such as PNR (Peak Signal-to-Noise Ratio) or SAD (Sum of Absolute Differences), in comparison with the input down-sampled video picture; c. summing R(B) and R(P) as a target bit count for the P picture, re-transcode it at one half of the incoming picture rate to obtain R(P 2 ) and D(P 2 ); d. reconstructing a skipped picture and calculate its distortion D(S), while the rate R(S) is zero, wherein the B picture is the skipped picture (S) and is reconstructed by averaging the previous I or P picture, which is the last coded picture of the previous sub-GOP, and picture P 2 obtained in step c; e. comparing the sum of distortion D(B)+D(P) with the sum of distortion D(S)+D(P 2 ).
8. The method of claim 7 , wherein following the comparing step, selecting the smaller sum of distortion, and if the sum of distortion D(B)+D(P) is chosen, both frames are transcoded, otherwise, only the P frame is transcoded, and storing the P frame in a reference frame buffer, and proceed to the next sub-GOP.
10. The method of claim 1 , wherein said adaptive frame rate transcoding processes on the basis of one sub-GOP at a time.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
January 17, 2008
October 30, 2012
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.