Reference processing may be used in a video encoder or decoder to derive reference pictures that are better correlated with a source image to be encoded or decoded, which generally yields better coding efficiency. Methods for filter selection for a reference processing unit adapted for use in a video codec system are discussed. Specifically, methods for filter selection based on performing motion estimation and obtaining distortion/cost information by comparing reference pictures, either processed or non-processed, with the source image to be encoded are discussed.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A method for selecting a particular filter from among a plurality of filters, the particular filter adapted for use in a reference picture processing unit of a computer processor in a multi-layered video coding system, the multi-layered video coding system comprising a base layer and one or more enhancement layers, the method comprising: a) providing a reference picture and an enhancement layer source picture; b) filtering copies of the reference picture using at least one filter from the plurality of filters to obtain at least one filtered reference picture, wherein each filter is applied to a corresponding copy of the reference picture; c) performing disparity estimation based on the enhancement layer source picture and a full set or subset of the at least one filtered reference picture, wherein the disparity estimation is adapted to generate disparity information, the disparity information being further based on temporal reference pictures from an enhancement layer picture buffer; d) obtaining distortion information based on the disparity information; e) selecting the particular filter based on comparing the distortion information generated in the distortion information obtaining step d); and f) for each filtered reference picture on which disparity estimation has been performed in accordance with step c), selecting between said reference picture and a corresponding temporal reference picture from the enhancement layer picture buffer, wherein the disparity information comprises a function of at least one of motion vectors, prediction distortion, intra prediction information, illumination parameters, luma components, chroma components, and tone mapping parameters.
2. The method as recited in claim 1 , wherein the reference picture is encoded after the filtering step.
3. The method as recited in claim 1 , wherein the reference picture comprises a base layer reference picture or an inter-layer reference picture, wherein an inter-layer comprises a layer from among the one or more enhancement layers.
4. The method as recited in claim 1 , wherein the reference picture comprises a spatial reference picture or a temporal reference picture from the enhancement layer.
5. The method as recited in claim 1 , wherein the step of providing further comprises processing the enhancement layer source picture, wherein the processing removes noise in the enhancement layer source picture.
6. The method as recited in claim 1 , wherein the step of providing further comprises processing the enhancement layer source picture, wherein the processing involves applying at least one of filtering, motion transformation, motion compensation, illumination compensation, scaling, inverse and forward tone mapping, color format conversion, and gamma correction.
7. The method as recited in claim 1 , wherein at least one of the filtered reference pictures comprises the reference picture.
8. The method as recited in claim 1 , wherein the step of performing disparity estimation further comprises obtaining cost information for the at least one filter from among the plurality of filters used in the step of filtering, and wherein the step of selecting is further based on the cost information; and wherein the cost information comprises a function of the disparity information, number of bits to be used in signaling filter parameters of each filter, number of bits to be used in signaling the motion vectors corresponding to each filtered reference picture, number of bits to be used in signaling the prediction distortion corresponding to each filtered reference picture, computational complexity in applying each filter, and power consumption of each filter.
9. The method as recited in claim 1 , further comprising, between the step of performing and the step of selecting, a step of performing disparity compensation on the enhancement layer source picture to obtain a final reconstructed picture, wherein the disparity compensation is based on the step of disparity estimation.
10. The method as recited in claim 1 , wherein each of the steps is performed on regions of the reference picture.
11. The method as recited in claim 10 , wherein the reference picture is decomposed using a plurality of region sizes and region shapes to obtain a plurality of reconstructed reference pictures, and wherein each of the steps are performed on the plurality of reconstructed reference pictures.
12. The method as recited in claim 10 , wherein the region sizes and the region shapes are determined based on performing edge detection on the reference picture.
13. The method as recited in claim 1 , wherein the disparity estimation comprises block-based motion estimation.
14. The method as recited in claim 13 , wherein motion vectors corresponding to a particular block are adapted to be predicted by motion vectors of blocks neighboring the particular block.
15. The method as recited in claim 13 , wherein block size is based on image characteristics of the reference picture, wherein the image characteristics comprise a function of at least one of a luma component, a chroma component, and edge characteristics of the reference picture and texture, color, shape, and size of elements in the reference picture.
16. The method as recited in claim 13 , wherein the step of performing disparity estimation or the step of obtaining distortion information also determines at least one of block size and block shape.
17. The method as recited in claim 1 , wherein the disparity estimation comprises integer pixel motion estimation.
18. The method as recited in claim 1 , wherein the disparity estimation comprises sub-pixel accurate motion estimation.
19. The method as recited in claim 1 , wherein the disparity estimation is further based on one or more luma and chroma components.
20. The method as recited in claim 19 , wherein the disparity estimation is further based on a subset of luma and chroma components, and wherein distortion of remaining luma and chroma components are computed based on the disparity information obtained from the subset of luma and chroma components.
21. The method as recited in claim 20 , further comprising a plurality of weighting factors, wherein one weighting factor is applied to each luma component and each chroma component.
22. The method as recited in claim 1 , wherein the step of providing comprises providing a plurality of reference pictures and the step of filtering is performed on a full set or subset of the plurality of reference pictures to obtain at least one filtered reference picture, and wherein each filter is applied to each reference picture of the full set or subset of the plurality of reference pictures.
23. A system for selecting a particular filter from among a plurality of filters, the particular filter adapted for use in a reference picture processing unit in a multi-layered video coding system, the multi-layered video coding system comprising a base layer and one or more enhancement layers, the system comprising: a) means for providing a reference picture and an enhancement layer source picture; b) means for filtering copies of the reference picture using at least one filter from the plurality of filters to obtain at least one filtered reference picture, wherein each filter is applied to a corresponding copy of the reference picture; c) means for performing disparity estimation based on the enhancement layer source picture and a full set or subset of the at least one filtered reference picture; and wherein the disparity estimation is adapted to generate disparity information, the disparity information being further based on temporal reference pictures from an enhancement layer picture buffer; d) means for obtaining distortion information based on the disparity information; e) means for selecting the particular filter based on comparing the distortion information generated with the distortion information obtaining means d); and f) means for selecting, for each filtered reference picture on which disparity estimation has been performed in accordance with step c), between said reference picture and a corresponding temporal reference picture from the enhancement layer picture buffer, wherein the disparity information comprises a function of at least one of motion vectors, prediction distortion, intra prediction information, illumination parameters, luma components, chroma components, and tone mapping parameters.
24. A non-transitory computer readable storage medium, comprising instructions that are tangibly encoded therewith, which when executed with a processor, cause the processor to cause, control, program or configure, at least in part, a process for selecting a particular filter from among a plurality of filters, the particular filter adapted for use in a reference picture processing unit in a multi-layered video coding system, the multi-layered video coding system comprising a base layer and one or more enhancement layers, the process comprising: a) providing a reference picture and an enhancement layer source picture; b) filtering copies of the reference picture using at least one filter from the plurality of filters to obtain at least one filtered reference picture, wherein each filter is applied to a corresponding copy of the reference picture; c) performing disparity estimation based on the enhancement layer source picture and a full set or subset of the at least one filtered reference picture, wherein the disparity estimation is adapted to generate disparity information, the disparity information being further based on temporal reference pictures from an enhancement layer picture buffer; d) obtaining distortion information based on the disparity information; e) selecting the particular filter based on comparing the distortion information generated in the distortion information obtaining step d); and f) for each filtered reference picture on which disparity estimation has been performed in accordance with step c), selecting between said reference picture and a corresponding temporal reference picture from the enhancement layer picture buffer, wherein the disparity information comprises a function of at least one of motion vectors, prediction distortion, intra prediction information, illumination parameters, luma components, chroma components, and tone mapping parameters.
25. A method for selecting a particular filter from among a plurality of filters, the particular filter adapted for use in a reference picture processing unit of a computer processor in a multi-layered video coding system, the multi-layered video coding system comprising a base layer and one or more enhancement layers, the method comprising: a) providing a reference picture and an enhancement layer source picture; b) filtering copies of the reference picture using at least one filter from the plurality of filters to obtain at least one filtered reference picture, wherein each filter is applied to a corresponding copy of the reference picture; c) performing disparity estimation based on the enhancement layer source picture, a full set or subset of the at least one filtered reference picture, and motion information from a particular layer, wherein the disparity estimation is adapted to generate disparity information, the disparity information being further based on temporal reference pictures from an enhancement layer picture buffer; d) obtaining distortion information based on the enhancement layer source picture, the full set or subset of the at least one filtered reference picture, and motion information from the particular layer; e) selecting the particular filter based on comparing the distortion information acquired in the distortion information obtaining step d); and f) for each filtered reference picture on which disparity estimation has been performed in accordance with step c), selecting between said reference picture and a corresponding temporal reference picture from the enhancement layer picture buffer, wherein the disparity information comprises a function of at least one of motion vectors, prediction distortion, intra prediction information, illumination parameters, luma components, chroma components, and tone mapping parameters, and wherein the motion information is based on temporal reference pictures of the particular layer.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
September 19, 2011
February 23, 2016
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.