A method for determining a relatedness between a query video and a database video is provided. A processor extracts an audio stream from the query video to produce a query audio stream, extracts an audio stream from the database video to produce a database audio stream, produces a first-sized snippet from the query audio stream, and produces a first-sized snippet from the database audio stream. An estimation is made of a first most probable sequence of latent evidence probability vectors generating the first-sized audio snippet of the query audio stream. An estimation is made of a second most probable sequence of latent evidence probability vectors generating the first-sized audio snippet of the database audio stream. A similarity is measured between the first sequence and the second sequence producing a score of relatedness between the two snippets. Finally a relatedness is determined between the query video and a database video.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A method for determining a relatedness between a query video and a database video, the method comprising the steps of: extracting an audio stream from the query video to produce a query audio stream; extracting an audio stream from the database video to produce a database audio stream; producing a first-sized snippet from the query audio stream; producing a first-sized snippet from the database audio stream; generating a collection of single state HMM (Hidden Markov Model) for sounds within the snippet from the query audio stream and the snippet from the database audio stream; estimating a first most probable sequence of a HMM state probability vectors generating the first-sized audio snippet of the query audio stream; estimating a second most probable sequence of HMM state probability vectors generating the first-sized audio snippet of the database audio stream; measuring a similarity between the first sequence and the second sequence to produce a score of relatedness between the first-sized query audio snippet and the first-sized database audio snippet, wherein the step of measuring the similarity between the between the first sequence and the second sequence to produce a score of relatedness comprises the step of using a Viterbi algorithm and Kernalized Locality-Sensitive Hashing to determine whether two videos are related; and determining a relatedness between the query video and a database video based on the measure of similarity.
2. The method of claim 1 further comprising the steps of: producing a second-sized snippet from the query audio stream; producing a second-sized snippet from the database audio stream; estimating a first most probable sequence of HMM state probability vectors generating the second-sized audio snippet of the query audio stream; estimating a second most probable sequence of HMM state probability vectors generating the second-sized audio snippet of the database audio stream; and measuring a similarity between the first sequence and the second sequence to produce a score of relatedness between the second-sized query audio snippet and the second-sized database audio snippet.
3. The method of claim 2 further comprising the steps of: sorting the relatedness scores; and wherein the step of determining the relatedness comprises the step of selecting N best relatedness scores as the relatedness scores between the query video and the database video.
4. The method of claim 2 wherein the step of producing a relatedness score comprises the step of using a Hamming distance to produce the relatedness score.
5. The method of claim 2 wherein second-sized snippet has a starting time that differs from the starting time of the first-sized snippet.
6. The method of claim 5 wherein the second-sized snippet contains a same portion of audio as the first-sized snippet.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
November 19, 2012
February 17, 2015
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.