Stabilizing a Sequence of Image Frames

1. A machine-implemented method of processing a sequence of image frames, comprising: computing respective sets of motion vectors for pairs of the image frames; classifying the computed motion vectors into motion classes; identifying motion clusters in the image frames based at least in part on the motion classes; determining for each of the identified motion clusters a respective spatiotemporal consistency value indicating persistence of the motion cluster in a respective spatial region across neighboring ones of the image frames; selecting one of the identified motion clusters as a motion stabilization reference based on the spatiotemporal consistency values; determining a motion model describing motion of the motion stabilization reference in the image frame sequence; and producing a motion-stabilized version of the sequence of image frames based on the motion model.

2. The method of claim 1 , wherein the computing comprises generating for pairs of the image frames respective dense motion models describing motion at pixel locations with respective sets of parameter values in a motion parameter space.

3. The method of claim 2 , wherein the identifying comprises iteratively clustering ones of the motion vectors from a coarse image frame resolution level to a fine image frame resolution level.

4. The method of claim 3 , wherein at each image frame resolution level ones of the motion vectors are classified into motion clusters, and a respective one of the spatiotemporal consistency values is determined for each of the clusters identified in a given image frame based on a projection of the motion cluster into a neighboring image frame using computed inter-frame motion.

5. The method of claim 3 , wherein motion vectors are clustered iteratively in accordance with a clustering method.

6. The method of claim 4 , wherein each of the respective spatiotemporal consistency values is determined based on degree of overlap between the respective motion cluster projected from the given image frame and a corresponding one of the motion clusters identified in the neighboring image frame.

7. The method of claim 4 , wherein ones of the motion vectors are re-classified with a modified clustering parameter in response to a determination that the respective spatiotemporal consistency values are below a consistency threshold.

8. The method of claim 1 , wherein the selecting comprises projecting each motion cluster from image frames to respective neighboring image frames, and computing respective measures of spatiotemporal consistency for the projected motion clusters.

9. The method of claim 1 , wherein the selecting comprises selecting as the motion stabilization reference for a given reference image frame the motion cluster having a greater spatiotemporal consistency value than the spatiotemporal consistency values of other ones of the motion clusters across multiple image frames neighboring the given reference image frame.

10. A system for processing a sequence of image frames, comprising: a motion estimation module configured to compute respective sets of motion vectors for pairs of image frames; a motion classification module configured to classify the computed motion vectors into motion classes; a motion-based spatial clustering module configured to identify motion clusters in the image frames based at least in part on the motion classes, and determine for each of the identified motion clusters a respective spatiotemporal consistency value indicating persistence of the motion cluster in a respective spatial region across neighboring ones of the image frames; a motion stabilization reference selection module configured to select one of the identified motion clusters as a motion stabilization reference based on the spatiotemporal consistency values; and a motion stabilization module configured to determine a motion model describing motion of the motion stabilization reference in the image frame sequence, and produce a motion-stabilized version of the sequence of image frames based on the motion model.

11. The system of claim 10 , wherein the motion estimation module is configured to compute motion vectors by generating for pairs of the image frames respective dense motion models describing motion at pixel locations with respective sets of parameter values in a motion parameter space.

12. The system of claim 11 , wherein the motion-based spatial clustering module is configured to identify motion clusters by iteratively clustering ones of the motion vectors from a coarse image frame resolution level to a fine image frame resolution level.

13. The system of claim 12 , wherein at each image frame resolution level ones of the motion vectors are classified by the motion classification module into motion clusters, and a respective one of the spatiotemporal consistency values is determined for each of the clusters in a given image frame based on a projection of the motion cluster into a neighboring image frame using computed inter-frame motion.

14. The system of claim 12 , wherein the motion classification module clusters motion vectors iteratively in accordance with a clustering method.

15. The system of claim 13 , wherein each of the respective spatiotemporal consistency values is determined based on degree of overlap between the respective motion cluster projected from the given image frame and a corresponding one of the motion clusters identified in the neighboring image frame.

16. The system of claim 13 , wherein the motion classification module re-classifies ones of the motion vectors with a modified clustering parameter in response to a determination that the respective spatiotemporal consistency values are below a consistency threshold.

17. The system of claim 10 , wherein the motion stabilization reference selection module selects a motion cluster as the motion stabilization reference by projecting each motion cluster from image frames to respective neighboring image frames and computing respective measures of spatiotemporal consistency for the projected motion clusters.

18. The system of claim 10 , wherein the motion stabilization reference selection module selects as the motion stabilization reference for a given reference image frame the motion cluster having a greater spatiotemporal consistency value than the spatiotemporal consistency values of other ones of the motion clusters across multiple image frames neighboring the given reference image frame.

19. A machine-readable medium storing machine-readable instructions for causing a machine to perform operations comprising: computing respective sets of motion vectors for pairs of image frames; classifying the computed motion vectors into motion classes; identifying motion clusters in the image frames based at least in part on the motion classes; determining for each of the identified motion clusters a respective spatiotemporal consistency value indicating persistence of the motion cluster in a respective spatial region across neighboring ones of the image frames; selecting one of the identified motion clusters as a motion stabilization reference based on the spatiotemporal consistency values; determining a motion model describing motion of the motion stabilization reference in the image frame sequence; and producing a motion-stabilized version of the sequence of image frames based on the motion model.

20. The machine-readable medium of claim 19 , wherein the machine-readable instructions cause the machine to compute motion vectors by generating for pairs of the image frames respective dense motion models describing motion at pixel locations with respective sets of parameter values in a motion parameter space.

21. The machine-readable medium of claim 20 , wherein the machine-readable instructions cause the machine to identify motion clusters by iteratively clustering ones of the motion vectors from a coarse image frame resolution level to a fine image frame resolution level.

22. The machine-readable medium of claim 21 , wherein at each image frame resolution level ones of the motion vectors are classified into motion clusters, and a respective one of the spatiotemporal consistency values is determined for each of the clusters in a given image frame based on a projection of the motion cluster into a neighboring image frame using computed inter-frame motion.

23. The machine-readable medium of claim 21 , wherein the machine-readable instructions cause the machine to cluster motion vectors iteratively in accordance with a clustering method.

24. The machine-readable medium of claim 22 , wherein each of the respective spatiotemporal consistency values is determined based on degree of overlap between the respective motion cluster projected from the given image frame and a corresponding one of the motion clusters identified in the neighboring image frame.

25. The machine-readable medium of claim 22 , wherein the machine-readable instructions cause the machine to re-classify ones of the motion vectors with a modified clustering parameter in response to a determination that the respective spatiotemporal consistency values are below a consistency threshold.

26. The machine-readable medium of claim 19 , wherein the machine-readable instructions cause the machine to select a motion cluster as the motion stabilization reference by projecting each motion cluster from image frames to respective neighboring image frames and computing respective measures of spatiotemporal consistency for the projected motion clusters.

27. The machine-readable medium of claim 19 , wherein the machine-readable instructions cause the machine to select as the motion stabilization reference for a given reference image frame the motion cluster having a greater spatiotemporal consistency value than the spatiotemporal consistency values of other ones of the motion clusters across multiple image frames neighboring the given reference image frame.