One embodiment of the present invention sets forth a technique that enables a user to reverse through video content based on scene transitions. By employing a graphics processing unit to compute one or more frame-to-frame correlation coefficients that measure the consistency of sequential images and a central processing unit to analyze the one or more correlation coefficients, a list of scene transitions may be generated in real-time. The list of scene transitions forms the basis for a content-based reverse user control within the playback application. The content-based reverse user control enables a more natural mechanism for reversing through video content, providing the user with a superior overall viewing experience.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A system for generating a list of content-based scene transitions within a stream of video data, the system comprising: a video decoder engine configured to receive the video data and to generate a plurality of video frames based on the video data; a frame buffer for storing the plurality of video frames; a frame correlation engine configured to generate at least one correlation coefficient based on a comparison between at least two video frames in the plurality of video frames, wherein the at least one correlation coefficient is used to determine whether a scene boundary exists between any of the at least two video frames; and a data transport engine configured to transfer the at least one correlation coefficient to a playback application executing on a central processing unit (CPU) to allow the playback application to determine whether a scene boundary exists between any of the at least two video frames based on the at least one correlation coefficient and to generate an entry in a list of scene transitions when a determination is made that a scene boundary exists.
2. The system of claim 1 , wherein the frame correlation engine includes a pixel difference engine that applies a pixel difference function between each pixel in a first video frame of the at least two video frames and a corresponding pixel in a second video frame of the at least two video frames to generate a set of difference pixel values.
3. The system of claim 2 , wherein the pixel difference function is applied using the luminance value associated with each pixel.
4. The system of claim 2 , wherein the frame correlation engine further includes a pixel difference summing engine that adds together the difference pixel values to produce the at least one correlation coefficient.
5. The system of claim 1 , wherein the frame correlation engine includes a frame histogram engine configured to generate a first histogram of the luminance values associated with the pixels in a first video frame of the at least two video frames and a second histogram of the luminance values associated with the pixels in a second video frame of the at least two video frames.
6. The system of claim 5 , wherein the frame correlation engine further includes a histogram difference engine that applies a histogram difference function between an element in the first histogram and a corresponding element in the second histogram to generate a set of histogram difference values, and a histogram difference summing engine that adds together the histogram difference values to generate the at least one correlation coefficient.
7. The system of claim 1 , wherein the frame correlation engine includes a pixel luminance correlation engine configured to: apply a pixel difference function between the luminance value of each pixel in a first video frame of the at least two video frames and the luminance value of a corresponding pixel in a second video frame of the at least two video frames to generate a set of luminance difference values; and add together the luminance difference values to produce a first correlation coefficient.
8. The system of claim 7 , wherein the frame correlation engine further includes a luminance histogram correlation engine configured to: generate a first histogram of the luminance values associated with the pixels in the first video frame and a second histogram of the luminance values associated with the pixels in the second video frame; apply a histogram difference function between an element in the first histogram and a corresponding element in the second histogram to generate a set of histogram difference values; and add together the histogram difference values to generate a second correlation coefficient.
9. The system of claim 1 , wherein the entry indicates the time at which the scene boundary is located within the video data.
10. The system of claim 9 , wherein the playback application is configured to receive user input to view the video data in a reverse mode that allows the user to view the video data beginning at the scene boundary.
11. The system of claim 10 , further comprising a display engine configured to generate a video output signal to display the video data beginning at the scene boundary based on a playback time control signal generated by the playback application in response to the user input.
12. A computing device configured to generate a list of content-based scene transitions within a stream of video data, the computing device comprising: a graphics processing unit (GPU) configured to: receive the video data and generate a plurality of video frames based on the video data; generate at least one correlation coefficient based on a comparison between at least two video frames in the plurality of video frames, wherein the at least one correlation coefficient is used to determine whether a scene boundary exists between any of the at least two video frames; and transfer the at least one correlation coefficient to a playback application executing on a central processing unit (CPU); and the playback application configured to: determine whether a scene boundary exists between any of the at least two video frames based on the at least one correlation coefficient; and generate an entry in a list of scene transitions when a determination is made that a scene boundary exists.
13. The computing device of claim 12 , wherein the playback application is configured to receive user input to view the video data in a reverse mode and to generate a playback time control signal in response to the user input that causes the GPU to display the video data beginning at the scene boundary.
14. The computing device of claim 12 , wherein the GPU computes the at least one correlation coefficient using a pixel difference function.
15. The computing device of claim 12 , wherein the GPU computes the at least one correlation coefficient using a histogram difference function.
16. A method for generating a list of content-based scene transitions within a stream of video data, the method comprising: receiving the video data and generating a plurality of video frames based on the video data; generating at least one correlation coefficient based on a comparison between at least two video frames in the plurality of video frames, wherein the at least one correlation coefficient is used to determine whether a scene boundary exists between any of the at least two video frames; and transferring the at least one correlation coefficient to a playback application executing on a central processing unit (CPU) to allow the playback application to determine whether a scene boundary exists between any of the at least two video frames based on the at least one correlation coefficient and to generate an entry in a list of scene transitions when a determination is made that a scene boundary exists.
17. The method of claim 16 , wherein the at least one correlation coefficient is generated using a pixel difference function or a histogram difference function.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
April 10, 2007
January 22, 2013
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.