A video sequence having a plurality of frames is received. A feature in a first frame from the plurality of frames and a first position of the feature in the first frame are detected. The position of the feature in a second frame from the plurality of frames is estimated to determine a second position. A displacement vector between the first position and the second position is also computed. A plurality of content characteristics is determined for the first frame and the second frame. The displacement vector is spatially diffused with a spatial filter over a frame from the plurality of frames to generate a spatially diffused displacement vector field. The spatial filter utilizes the plurality of content characteristics. A temporal filter temporally diffuses over a video volume the spatially diffused displacement vector field to generate a spatiotemporal vector field. The temporal filter utilizes the plurality of content characteristics.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A computer program product comprising a computer readable storage device having a computer readable program stored thereon, wherein the computer readable program when executed on a computer causes the computer to: receive a video sequence having a plurality of frames; detect a feature in a first frame from the plurality of frames and a first position of the feature in the first frame; estimate the position of the feature in a second frame from the plurality of frames to determine a second position; compute a displacement vector between the first position and the second position; determine a plurality of content characteristics for the first frame and the second frame; spatially diffuse the displacement vector with a spatial filter over a frame from the plurality of frames to generate a spatially diffused displacement vector field, the spatial filter utilizing the plurality of content characteristics; and temporally diffuse over a video volume, with a temporal filter, the spatially diffused displacement vector field to generate a spatiotemporal vector field, the temporal filter utilizing the plurality of content characteristics.
2. The computer program product of claim 1 , wherein the spatially diffused displacement vector field is temporally diffused along a motion path determined by the spatially diffused displacement vector field.
3. The computer program product of claim 1 , wherein the plurality of content characteristics comprises a plurality of edges.
4. The computer program product of claim 1 , wherein the plurality of content characteristics is calculated.
5. The computer program product of claim 1 , wherein the plurality of content characteristics is received through as at least one input.
6. The computer program product of claim 1 , wherein the spatial filter is a two-dimensional filter.
7. The computer program product of claim 1 , wherein the spatial filter comprises a one-dimensional x axis filter and one-dimensional y axis filter.
8. The computer program product of claim 1 , wherein the temporal filter comprises a one-dimensional time axis filter.
9. The computer program product of claim 1 , where the spatial and temporal filters are weighted with a confidence value that is assigned to the feature.
10. The computer program product of claim 1 , wherein the temporal filter is an edge-aware filter.
11. A computer program product comprising a computer readable storage device having a computer readable program stored thereon, wherein the computer readable program when executed on a computer causes the computer to: receive a video sequence having a plurality of frames; receive a spatiotemporal displacement vector field; receive data; determine a plurality of content characteristics for a first frame and a second frame in the plurality of frames; spatially diffuse the data with a spatial filter a frame to generate a spatially diffused data field, the spatial filter utilizing the plurality of content characteristics; and temporally diffuse over a video volume, with a temporal filter, the spatially diffused data field along a motion path determined by the spatiotemporal displacement vector field to generate a spatiotemporal data field, the temporal filter utilizing the plurality of content characteristics.
12. The computer program product of claim 11 , wherein the data is selected from the group consisting of optical flow, depth/disparity estimation, colorization, low resolution data upscaling, and scribble propagation.
13. A method comprising: receiving a video sequence having a plurality of frames; detecting a feature in a first frame from the plurality of frames and a first position of the feature in the first frame; estimating the position of the feature in a second frame from the plurality of frames to determine a second position; computing a displacement vector between the first position and the second position; determining a plurality of content characteristics for the first frame and the second frame; spatially diffusing the displacement vector with a spatial filter over a frame from the plurality of frames to generate a spatially diffused displacement vector field, the spatial filter utilizing the plurality of content characteristics; and temporally diffusing over a video volume, with a temporal filter, the spatially diffused displacement vector field to generate a spatiotemporal vector field, the temporal filter utilizing the plurality of content characteristics.
14. The method of claim 13 , wherein the spatially diffused displacement vector field is temporally diffused along a motion path determined by the spatially diffused displacement vector field.
15. The method of claim 13 , wherein the plurality of content characteristics comprises a plurality of edges.
16. The method of claim 13 , wherein the plurality of content characteristics is calculated.
17. The method of claim 13 , wherein the plurality of content characteristics is received through as at least one input.
18. The method of claim 13 , wherein the spatial filter is a two-dimensional filter.
19. The method of claim 13 , wherein the spatial filter comprises a one-dimensional x axis filter and one-dimensional y axis filter.
20. The method of claim 13 , wherein the temporal filter comprises a one-dimensional time axis filter.
21. The method of claim 13 , where the spatial and temporal filters are weighted with a confidence value that is assigned to the feature.
22. The method of claim 13 , wherein the temporal filter is an edge-aware filter.
23. A system comprising: a processor that receives a video sequence having a plurality of frames, detects a feature in a first frame from the plurality of frames and a first position of the feature in the first frame, estimates the position of the feature in a second frame from the plurality of frames to determine a second position, computes a displacement vector between the first position and the second position, determines a plurality of content characteristics for the first frame and the second frame, spatially diffuses the displacement vector with a spatial filter over a frame from the plurality of frames to generate a spatially diffused displacement vector field, and temporally diffuses over a video volume, with a temporal filter, the spatially diffused displacement vector field to generate a spatiotemporal vector field, the spatial filter utilizing the plurality of content characteristics, the temporal filter utilizing the plurality of content characteristics.
24. The system of claim 23 , wherein the spatially diffused displacement vector field is temporally diffused along a motion path determined by the spatially diffused displacement vector field.
25. The system of claim 23 , wherein the plurality of content characteristics comprises a plurality of edges.
26. The system of claim 23 , wherein the plurality of content characteristics is calculated.
27. The system of claim 23 , wherein the plurality of content characteristics is received through as at least one input.
28. The system of claim 23 , wherein the spatial filter is a two-dimensional filter.
29. The system of claim 23 , wherein the spatial filter comprises a one-dimensional x axis filter and one-dimensional y axis filter.
30. The system of claim 23 , wherein the temporal filter comprises a one-dimensional time axis filter.
31. The system of claim 23 , where the spatial and temporal filters are weighted with a confidence value that is assigned to the feature.
32. The system of claim 23 , wherein the temporal filter is an edge-aware filter.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
November 27, 2012
May 19, 2015
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.