Methods for processing a set of successive video frames in two passes to determine the number of bidirectional (B) and unidirectional (P) motion compensated frames to be encoded in a video coding system. During the first pass, motion vectors and motion costs are computed for each frame and a derived cost value is computed based on the motion cost of at least one frame. The derived cost value is used to determine the number (NB) of B-frames to be encoded in the set of successive frames. In the second pass, the set of successive frames are encoded where NB frames are encoded as B-frames and some or all motion vectors computed in the first pass are re-used in the second pass. A scene cut detection method is also provided where an impulse-like increase in a ratio of motion costs is monitored.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A method for processing a plurality of video pictures, the method comprising: computing motion vectors for the plurality of video pictures as if all video pictures after a first video picture in the plurality of video pictures are going to be encoded as unidirectional motion compensated (P) video pictures; computing a plurality of costs for the computed motion vectors; computing at least one value based on the plurality of costs; and determining whether the value satisfies a threshold value in order to determine a number of video pictures in the plurality of video pictures to be encoded as bidirectional motion compensated (B) video pictures.
2. The method of claim 1 further comprising encoding the plurality of video pictures using the determined number of B-video pictures.
3. The method of claim 2 , wherein (i) computing the motion vectors, (ii) computing the costs, (iii) computing the value, and (iv) determining whether the value satisfies the threshold value are performed in a first set of operations during a first processing pass of the plurality of video pictures, wherein encoding the plurality of video pictures using the determined number of B-video pictures is performed in a second set of operations during a second processing pass of the plurality of video pictures.
4. The method of claim 1 , wherein computing the value comprises computing an average of costs for motion vectors of at least one video picture in the plurality of video pictures.
5. A method for processing a sequence of video pictures, each video picture comprising at least one set of pixels, the method comprising: for each particular video picture after a first video picture in the sequence of video pictures, (i) computing a cost for a motion vector for at least one set of pixels of the particular video picture, each set of pixels of the particular video picture associated with at most one motion vector, each motion vector based only on the particular video picture and a preceding video picture and (ii) computing a value based on at least one of the computed costs; and determining a number of video pictures in the sequence of video pictures to be encoded as bidirectional motion compensated (B) video pictures based on at least one of the computed values.
6. The method of claim 5 , wherein determining the number of video pictures to be encoded as B-video pictures comprises determining whether the value satisfies a threshold value.
7. The method of claim 6 , wherein the value is further based on a ratio that is based on two different costs for motion vectors of two different video pictures.
8. The method of claim 6 , wherein the value is further based on an average cost for motion vectors of a set of video pictures in the sequence of video pictures.
9. The method of claim 6 , wherein the value is an average cost for motion vectors of a particular video picture.
10. The method of claim 6 further comprising encoding the sequence of video pictures based on the number of video pictures to be encoded as B-video pictures, wherein the encoding of the sequence of video pictures comprises using all of the computed motion vectors.
11. The method of claim 6 , wherein at least one of the computed motion vectors is based on the first video picture in the sequence of video pictures, wherein the first video picture is encoded as an I-picture.
12. The method of claim 5 , wherein a set of pixels is a partition of a macroblock.
13. The method of claim 5 , wherein the motion vector for each set of pixels of the particular video picture is based on another set of pixels of another video picture that immediately precedes the particular video picture in a sequence for displaying the video pictures.
14. The method of claim 6 further comprising encoding the sequence of video pictures based on the number of B-video pictures that is determined by using the computed values, wherein the encoding of the sequence of video pictures comprises using at least one of the computed motion vectors.
15. The method of claim 5 , wherein determining the number of video pictures to be encoded as B-video pictures comprises determining whether there is a scene cut based on at least one of the computed values.
16. The method of claim 15 , wherein the value is a ratio, wherein determining whether there is the scene cut comprises determining whether the ratio satisfies a threshold value.
17. The method of claim 6 , wherein the threshold value is based on a type of content of the video pictures to be encoded, wherein different threshold values are used for different types of content.
18. The method of claim 17 , wherein the type of content comprises one of a sports content, soap opera content, news report content, and videoconference content.
19. The method of claim 6 , wherein the threshold value varies based on a number of video pictures that have been processed in the sequence of video pictures.
20. The method of claim 6 , wherein the threshold value varies monotonically based on a number of video pictures that have been processed in the sequence of video pictures, wherein the threshold value decreases as the number of video pictures that have been processed increases.
21. The method of claim 6 , wherein the cost is based on a number of bits required for encoding at least one motion vector.
22. The method of claim 21 , wherein the cost is further based on (i) a motion compensation error for the motion vector and (ii) a Lagrangian multiplier.
23. A method comprising: computing a motion vector for a first unencoded video picture; computing a motion vector for a second unencoded video picture; computing a particular value based on a cost value for the motion vector for the first video picture and a cost value for the motion vector for the second video picture, wherein a cost value for a motion vector for a particular video picture is based on a number of bits required for encoding the motion vector for the particular video picture; determining whether there is a scene cut between the first video picture and the second video picture based on the particular value; and encoding the first and second video pictures.
24. The method of claim 23 , wherein determining whether there is the scene cut comprises determining whether the particular value satisfies a threshold value.
25. The method of claim 23 , wherein the cost value for each video picture is further based on (i) a motion compensation error associated with the motion vector for the video picture and (ii) a Lagrangian multiplier.
26. The method of claim 24 , wherein the particular value is a ratio between the cost values of the motion vectors of the first and second video pictures, wherein determining whether the particular value satisfies the threshold value comprises determining whether the ratio is less than the threshold value.
27. The method of claim 24 , wherein the threshold value varies monotonically based on a number of video pictures that have been processed in a sequence of video pictures comprising the first and second video pictures, wherein the threshold value decreases as the number of video pictures that have been processed increases.
28. The method of claim 24 , wherein determining whether there is the scene cut comprises determining that there is no scene cut when the particular value does not satisfy the threshold value.
29. The method of claim 24 , wherein determining whether there is the scene cut comprises determining that the particular value satisfies the threshold value.
30. The method of claim 29 further comprising marking the second video picture as a video picture after the scene cut.
31. The method of claim 30 , wherein marking the second video picture comprises marking the second video picture as an I-picture.
32. The method of claim 29 further comprising marking the first video picture as a video picture before the scene cut.
33. The method of claim 32 further comprising marking the first video picture as a P-picture.
34. The method of claim 23 , wherein the cost value for the motion vector for the first video picture is computed as a cost of encoding the first video picture using the motion vector for the first video picture and the cost value for the motion vector for the second video picture is computed as a cost of encoding the second video picture using the motion vector for the second video picture.
35. The method of claim 23 , wherein the first video picture comprises a plurality of sets of pixels, each set of pixels of the first video picture associated with at most one motion vector, wherein computing the particular value is further based on cost values for motion vectors of the plurality of sets of pixels of the first video picture.
36. The method of claim 35 , wherein the second video picture comprises a plurality of sets of pixels, each set of pixels of the second video picture associated with at most one motion vector, wherein computing the particular value is further based on cost values for motion vectors of the plurality of sets of pixels of the second video picture.
37. The method of claim 23 , wherein the motion vectors for the first and second video pictures are computed as if the first video picture and the second video picture are going to be encoded as unidirectional motion compensated video pictures.
38. The method of claim 37 , wherein a unidirectional motion compensated video picture is a P-picture.
39. The method of claim 36 , wherein computing the particular value is further based on a sum of the cost values for motion vectors of the plurality of sets of pixels of the first video picture and a sum of the cost values for motion vectors of the plurality of sets of pixels of the second video picture.
40. A method comprising: computing a value based on a cost value for a motion vector for a first video picture and a cost value for a motion vector for a second video picture, said cost values based on numbers of bits required for encoding said motion vectors; and determining whether the value satisfies a threshold value to (i) determine whether there is a scene cut between the first video picture and the second video picture, and (ii) determine a number of bidirectional motion compensated video pictures in a video sequence to be encoded, the video sequence comprising the first and second video pictures.
41. The method of claim 40 , wherein the cost values for the motion vectors for the first and second video pictures are computed as if all video pictures after an initial video picture in the video sequence are going to be encoded as unidirectional motion compensated video pictures.
42. The method of claim 40 further comprising encoding the video sequence using the determined number of bidirectional motion compensated video pictures.
43. The method of claim 42 , wherein computing the value comprises computing at least one motion vector for the first video picture and at least one motion vector for the second video picture, wherein the motion vectors are computed in a first set of operations during a first processing pass of the video sequence, wherein encoding is performed in a second set of operations during a second processing pass of the video sequence, wherein encoding comprises using at least one motion vector that was computed during the first set of operations.
44. The method of claim 42 , wherein computing the value comprises computing at least one motion vector for the first video picture and at least one motion vector for the second video picture, wherein the motion vectors are computed in a first set of operations during a first processing pass of the video sequence, wherein encoding is performed in a second set of operations during a second processing pass of the video sequence, wherein encoding comprises using all of the motion vectors that were computed during the first set of operations.
45. The method of claim 40 , wherein a bidirectional motion compensated video picture is a B-picture.
46. The method of claim 40 , wherein computing the value further comprises computing a sum of a plurality of costs for a plurality of motion vectors for the first video picture and a sum of a plurality of costs for a plurality of motion vectors for the second video picture.
47. A non-transitory computer readable medium storing a computer program that is executable by one or more processing units, the computer program for processing a plurality of video pictures, the computer program comprising sets of instructions for: computing motion vectors for the plurality of video pictures as if all video pictures after a first video picture in the plurality of video pictures are going to be encoded as unidirectional motion compensated (P) video pictures; computing a plurality of costs for the computed motion vectors; computing at least one value based on the plurality of costs; and determining whether the value satisfies a threshold value in order to determine whether there is a scene cut in the plurality of video pictures.
48. The non-transitory computer readable medium of claim 47 further comprising a set of instructions for encoding the plurality of video pictures using the determined scene cut.
49. The non-transitory computer readable medium of claim 48 , wherein the sets of instructions for (i) computing the motion vectors, (ii) computing the costs, (iii) computing the value, and (iv) determining whether the value satisfies the threshold value are executed in a first set of operations during a first processing pass of the plurality of video pictures, wherein the set of instructions for determining whether there is the scene cut is executed in a second set of operations during a second processing pass of the plurality of video pictures.
50. The non-transitory computer readable medium of claim 47 , wherein the set of instructions for computing the value comprises a set of instructions for computing an average of costs for motion vectors of at least one video picture in the plurality of video pictures.
51. The non-transitory computer readable medium of claim 47 , wherein the cost value for each motion vector in each video picture is further based on (i) a motion compensation error associated with the motion vector for the video picture and (ii) a Lagrangian multiplier.
52. The non-transitory computer readable medium of claim 47 , wherein the threshold value varies monotonically based on a number of video pictures that have been processed in a sequence of video pictures comprising said plurality of video pictures, wherein the threshold value decreases as the number of video pictures that have been processed increases.
53. The non-transitory computer readable medium of claim 47 , wherein the value is a ratio between cost values of the motion vectors of two successive video pictures in a sequence of video pictures comprising said plurality of video pictures, wherein the set of instructions for determining whether the value satisfies the threshold value comprises a set of instructions for determining whether the ratio is less than the threshold value.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
October 22, 2007
December 21, 2010
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.