Method and Apparatus for the Use of Cross Modal Association to Isolate Individual Media Sources

1. Apparatus for cross-modal association of events from a complex source having at least a first and a second modality, multiple objects, and events, the apparatus comprising: an input for receiving first data from a first recording device, said first data relating to said first modality; an input for receiving second data from a second recording device, said second data relating to said second modality; an associator configured for iteratively associating event-related changes recorded in said first mode and event-related changes recorded in said second mode according to a predetermined maximum likelihood criterion, said likelihood criterion, over said iteration, obtaining a score for respective event related changes in said first mode and reinforcing respective associations where event related changes are repeated and reducing respective associations where event related changes are not repeated, said associator configured to provide an association between events belonging to said changes using a result of said iteration, by selecting a best score, thereby not pregrouping said event-related changes into different coherent groups expected to repeat themselves; a first output connected to said associator, configured to indicate ones of the multiple objects in the second modality being associated with respective ones of the multiple events in the first modality.

2. The apparatus of claim 1 , wherein said event-related change is any one of the group comprising a maximum rate of acceleration, and an onset.

3. The apparatus of claim 1 , wherein said associator is configured to make said association based on respective timings of said onsets.

4. The apparatus of claim 1 , further comprising a second output associated with said first output configured to group together events in the first modality that are all associated with a selected object in the second modality; thereby to isolate a stream associated with said object.

5. The apparatus of claim 1 , wherein said first modality is an audio mode and said first recording device is one or more microphones, and said second modality is a visual mode, and said second recording device is one or more cameras.

6. The apparatus of claim 1 , further comprising event change detectors placed between respective recording devices and said associator, to provide event change indications for use by said associator.

7. The apparatus of claim 1 , wherein said maximum likelihood detector is configured to refine said likelihood based on repeated occurrences of said given event in said second modality.

8. The apparatus of claim 7 , wherein said maximum likelihood detector is configured to calculate a confirmation likelihood based on association of said event in said second modality with repeated occurrence of said event in said first mode.

9. Method for isolation of a media stream for respected detected objects of a first modality from a complex media source having at least two media modalities, multiple objects, and events, the method comprising: obtaining first data of said first modality; obtaining second data of a second modality; detecting events and respective changes of said events; iteratively associating between events recorded in said first modality and events recorded in said second modality according to a predetermined maximum likelihood criterion, said associating comprising obtaining a score for respective event related changes in said first mode based at least partly on timings of respective changes and providing an association output using a best score result of said iteration, said maximum likelihood criterion, over said iteration, reinforcing respective associations where event related changes are repeated and reducing respective associations where event related changes are not repeated, said scoring using said predetermined maximum likelihood criterion thereby obviating a need for pregrouping said event-related changes into different coherent groups expected to repeat themselves; and isolating those events in said first modality associated with events in said second modality associated with a predetermined object, thereby to isolate an isolated media stream associated with said predetermined object.

10. The method of claim 9 , wherein said first modality is an audio modality, and said second modality is a visual modality.

11. The method of claim 9 , providing event change indications for use in said association.

12. The method of claim 11 , wherein said maximum likelihood criterion comprises calculating a likelihood that a given event in said first modality is associated with a given event of a specific object in said second modality.

13. The method of claim 12 , wherein said maximum likelihood criterion further comprises refining said likelihood based on repeated occurrences of said given event in said second modality.

14. The method of claim 13 , wherein said maximum likelihood criterion further comprises calculating a confirmation likelihood based on association of said event in said second modality with repeated occurrence of said event in said first modality.