Methods, apparatuses and systems may provide for technology that performs global motion estimation. More particularly, implementations relate to technology that provides accurate global motion compensation in order to improve video processing efficiency. In some implementations, a highly adaptive and accurate approach may be used to address the problem of estimation and compensation of global motion in video scenes. The solution may be content adaptive as it uses adaptive modeling of motion using best of multiple models that are used to estimate global motion.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A system to perform efficient motion based video processing using global motion, comprising: a global motion analyzer, the global motion analyzer including one or more substrates and logic coupled to the one or more substrates, wherein the logic is to: obtain a plurality of block motion vectors for a plurality of blocks of a current frame with respect to a reference frame; modify the plurality of block motion vectors, wherein the modification of the plurality of block motion vectors includes one or more of the following operations: smoothing of one or more of the plurality of block motion vectors, merging of one or more of the plurality of block motion vectors, and discarding of one or more of the plurality of block motion vectors; restrict the modified plurality of block motion vectors by excluding a portion of the frame; compute a plurality of candidate global motion models based on the restricted-modified plurality of block motion vectors for the current frame with respect to the reference frame, wherein each candidate global motion model comprises a set of candidate global motion model parameters representing global motion of the current frame, wherein the computation of the plurality of candidate global motion models further comprises operations to: choose a global motion region and a local motion region segmentation for selection of a valid region for choosing candidate motion vectors for global motion model computation, wherein the global motion region includes an area of blocks whose motion corresponds to globally moving objects, wherein the local motion region includes an area of blocks whose motion corresponds to locally moving objects, and choose a set of global motion models in a first mode selected from among four parameter models, six parameter models, and eight parameter models as well as in a second mode selected from among six parameter models, eight parameter models, and twelve parameter models, wherein the first mode is selected for first definition scene sequences and the second mode is selected for second definition scene sequences wherein the first definition scene sequences and the second definition scene sequences have different levels of resolution; determine a final global motion model from the computed plurality of candidate global motion models on a frame-by-frame basis, wherein each final global motion model comprises a set of final global motion model parameters representing global motion of the current frame; modify a precision of the final global motion model parameters in response to one or more application parameters, wherein the application parameters, include one or more of the following application parameter types: coding bit-rate, resolution, and required visual quality, wherein the modification of the precision of the final global motion model parameters includes assigning a different accuracy to one or more model parameter of the final global motion model parameters; map the modified-precision final global motion model parameters to a pixel-based coordinate system to determine a plurality of mapped global motion warping vectors for a plurality of reference frame control-grid points; predict and encode the plurality of mapped global motion warping vectors for the current frame with respect to a plurality of previous mapped global motion warping vectors; determine a final sub-pel filter to use for interpolation at an ⅛ th pel location or a 1/16 th pel location from among two or more sub-pel filter choices per frame; apply the plurality of mapped global motion warping vectors at sub-pel locations to the reference frame and perform interpolation of pixels based on the determined final sub-pel filter to generate a global motion compensated warped reference frame; and a power supply to provide power to the global motion analyzer.
2. The system of claim 1 , wherein the computation of the plurality of candidate global motion models further comprises operations to: choose a method for computing each individual global motion model of the set of global motion models selected from among least square and Levenberg Marquardt (LMA); and choose one or more convergence parameters for the chosen least square and Levenberg Marquardt method.
3. The system of claim 2 , further comprising operations to: select a method for computing each individual global motion model depending on the order of each individual global motion model of the set of global motion models including for four and six parameter model using the least square method, and for eight and twelve parameter model using the Levenberg Marquardt method; perform computation of the each global motion model using the related chosen method; and select a final model based on lowest distortion.
4. The system of claim 2 , further comprising operations to: select a method for computing each individual global motion model depending on the order of each individual global motion model of the set of global motion models including for four and six parameter model using the least square method, and for eight and twelve parameter model using the Levenberg Marquardt method; perform computation of the each global motion model using the related chosen method; and select a final model from the set of global motion models based on a Rate Distortion Optimization tradeoff that takes into account distortion.
5. A method to perform efficient motion based video processing using global motion, comprising: obtaining a plurality of block motion vectors for a plurality of blocks of a current frame with respect to a reference frame; modifying the plurality of block motion vectors, wherein the modification of the plurality of block motion vectors includes one or more of the following operations: smoothing of one or more of the plurality of block motion vectors, merging of one or more of the plurality of block motion vectors, and discarding of one or more of the plurality of block motion vectors; restricting the modified plurality of block motion vectors by excluding a portion of the frame; computing a plurality of candidate global motion models based on the restricted-modified plurality of block motion vectors for the current frame with respect to the reference frame, wherein each candidate global motion model comprises a set of candidate global motion model parameters representing global motion of the current frame, wherein the computation of the plurality of candidate global motion models further comprises: choosing a global motion region and a local motion region segmentation for selection of a valid region for choosing candidate motion vectors for global motion model computation, wherein the global motion region includes an area of blocks whose motion corresponds to globally moving objects, wherein the local motion region includes an area of blocks whose motion corresponds to locally moving objects, and choosing a set of global motion models in a first mode selected from among four parameter models, six parameter models, and eight parameter models as well as in a second mode selected from among six parameter models, eight parameter models, and twelve parameter models, wherein the first mode is selected for first definition scene sequences and the second mode is selected for second definition scene sequences wherein the first definition scene sequences and the second definition scene sequences have different levels of resolution; determining a final global motion model from the computed plurality of candidate global motion models on a frame-by-frame basis, wherein each final global motion model comprises a set of final global motion model parameters representing global motion of the current frame; modifying a precision of the final global motion model parameters in response to one or more application parameters, wherein the application parameters, include one or more of the following application parameter types: coding bit-rate, resolution, and required visual quality, wherein the modification of the precision of the final global motion model parameters includes assigning a different accuracy to one or more model parameter of the final global motion model parameters; mapping the modified-precision final global motion model parameters to a pixel-based coordinate system to determine a plurality of mapped global motion warping vectors for a plurality of reference frame control-grid points; predicting and encoding the plurality of mapped global motion warping vectors for the current frame with respect to a plurality of previous mapped global motion warping vectors; determining a final sub-pel filter to use for interpolation at an ⅛ th pel location or a 1/16 th pel location from among two or more sub-pel filter choices per frame; applying the plurality of mapped global motion warping vectors at sub-pel locations to the reference frame and perform interpolation of pixels based on the determined final sub-pel filter to generate a global motion compensated warped reference frame.
6. The method of claim 5 , wherein the computation of the plurality of candidate global motion models further comprises: choosing a method for computing each individual global motion model of the set of global motion models selected from among least square and Levenberg Marquardt (LMA), including for four and six parameter model using the least square method, and for eight and twelve parameter model using the Levenberg Marquardt method; performing computation of the each global motion model using the related chosen method; and selecting a final model from the set of global motion models based on a select one of a lowest distortion and a Rate Distortion Optimization tradeoff that takes into account distortion.
7. At least one non-transitory computer readable storage medium comprising a set of instructions, which when executed by a computing system, cause the computing system to: obtain a plurality of block motion vectors for a plurality of blocks of a current frame with respect to a reference frame; modify the plurality of block motion vectors, wherein the modification of the plurality of block motion vectors includes one or more of the following operations: smoothing of one or more of the plurality of block motion vectors, merging of one or more of the plurality of block motion vectors, and discarding of one or more of the plurality of block motion vectors; restrict the modified plurality of block motion vectors by excluding a portion of the frame; compute a plurality of candidate global motion models based on the restricted-modified plurality of block motion vectors for the current frame with respect to the reference frame, wherein each candidate global motion model comprises a set of candidate global motion model parameters representing global motion of the current frame, wherein the computation of the plurality of candidate global motion models further comprises operations to: choose a global motion region and a local motion region segmentation for selection of a valid region for choosing candidate motion vectors for global motion model computation, wherein the global motion region includes an area of blocks whose motion corresponds to globally moving objects, wherein the local motion region includes an area of blocks whose motion corresponds to locally moving objects, and choose a set of global motion models in a first mode selected from among four parameter models, six parameter models, and eight parameter models as well as in a second mode selected from among six parameter models, eight parameter models, and twelve parameter models, wherein the first mode is selected for first definition scene sequences and the second mode is selected for second definition scene sequences wherein the first definition scene sequences and the second definition scene sequences have different levels of resolution; determine a final global motion model from the computed plurality of candidate global motion models on a frame-by-frame basis, wherein each final global motion model comprises a set of final global motion model parameters representing global motion of the current frame; modify a precision of the final global motion model parameters in response to one or more application parameters, wherein the application parameters, include one or more of the following application parameter types: coding bit-rate, resolution, and required visual quality, wherein the modification of the precision of the final global motion model parameters includes assigning a different accuracy to one or more model parameter of the final global motion model parameters; map the modified-precision final global motion model parameters to a pixel-based coordinate system to determine a plurality of mapped global motion warping vectors for a plurality of reference frame control-grid points; predict and encode the plurality of mapped global motion warping vectors for the current frame with respect to a plurality of previous mapped global motion warping vectors; determine a final sub-pel filter to use for interpolation at an ⅛th pel location or a 1/16th pel location from among two or more sub-pel filter choices per frame; apply the plurality of mapped global motion warping vectors at sub-pel locations to the reference frame and perform interpolation of pixels based on the determined final sub-pel filter to generate a global motion compensated warped reference frame.
8. The at least one non-transitory computer readable storage medium of claim 7 , wherein the instructions, when executed, cause the computing system to: choose a method for computing each individual global motion model of the set of global motion models selected from among least square and Levenberg Marquardt (LMA), including for four and six parameter model using the least square method, and for eight and twelve parameter model using the Levenberg Marquardt method; perform computation of the each global motion model using the related chosen method; and select a final model from the set of global motion models based on a select one of a lowest distortion and a Rate Distortion Optimization tradeoff that takes into account distortion.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
June 29, 2018
July 20, 2021
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.