10140088

Visual Spatial Audio

PublishedNovember 27, 2018
Assigneenot available in USPTO data we have
Technical Abstract

Patent Claims
24 claims

Legal claims defining the scope of protection, as filed with the USPTO.

1

1. A method comprising: receiving at least two audio signals from one or more sound sources by two or more microphones of an apparatus; determining at least one user interface element associated with the one or more sound sources being displayed by a display of the apparatus based on the at least two audio signals, wherein the at least one user interface element is configured to be modified by an indicator; determining at least one spatial component of the one or more sound sources comprising at least one directional component; generating a visualization of the one or more sound sources, wherein the visualization is configured to rotate or change the at least one directional component of the one or more sound sources dependent on the indicator for the at least one user interface element, the at least one directional component based on the at least two audio signals, wherein generating the visualization further comprises the at least one directional component for the one or more sound sources being mapped to the visualization, wherein the at least one directional component comprises direction and magnitude of the one or more sound sources; and rendering the at least two audio signals to generate a modified version of the at least one spatial component of the one or more sound sources after being mapped to the visualization dependent on the indicator, wherein the visualization comprises a rotation direction indication of a swipe gesture for the visualization of the one or more sound sources to cause to rotate said at least one directional component of the modified version of the at least one spatial component during rendering together with said magnitude of the one or more sound sources in such a way that the one or more sound sources is reproduced with the modified version of the at least one spatial component based on the swipe gesture.

2

2. The method as claimed in claim 1 , wherein receiving the at least two audio signals further comprises at least one of: receiving the at least two audio signals from the at least two or more microphones of the apparatus; receiving the at least two audio signals from a memory; or receiving the at least two audio signals from at least one external apparatus.

3

3. The method as claimed in claim 1 , wherein determining the at least one user interface element further comprises: receiving the at least one user interface element from a touch screen; determining an action associated with the at least one user interface element; and outputting the indicator so as to represent the action.

4

4. The method as claimed in claim 3 , wherein generating the visualization of the one or more sound sources further comprises: receiving the indicator; and modifying the visualization of the at least two audio signals dependent on the indicator.

5

5. The method as claimed in claim 3 , wherein rendering the at least two audio signals to generate the modified version of the at least one spatial component further comprises: receiving the indicator; and modifying the rendered at least two audio signals dependent on the indicator to follow the visualization of the at least two audio signals.

6

6. The method as claimed in claim 3 , wherein the action associated with the at least one user interface element comprises at least one of: a clockwise rotation; a counter-clockwise rotation; a translational motion; a rotation about an external point; a focus on a determined position; or a tilt.

7

7. The method as claimed in claim 1 , wherein generating the visualization of the one or more sound sources further comprises: determining the at least one directional component of the one or more sound sources; and determining a radial value for a visualization sector dependent on a frequency of directional components of the at least two audio signals in a sector range.

8

8. The method as claimed in claim 7 , wherein determining the at least one spatial component of the one or more sound sources comprising the at least one directional component further comprises: grouping time domain samples of the at least two audio signals into frames; time to frequency domain converting the frames of the at least two audio signals; dividing the frequency domain representation of the at least two audio signals into at least two sub-bands; and comparing pairs of audio signal sub-bands to determine the at least one directional component.

9

9. The method as claimed in claim 7 , wherein generating the visualization of the one or more sound sources further comprises: determining at least one magnitude component for the at least one directional component of the one or more sound sources; and determining a color value for the visualization sector dependent on the at least one magnitude component for the at least one directional component of the one or more sound sources.

10

10. An apparatus comprising: at least one processor and at least one memory including computer code for one or more programs, the at least one memory and the computer code configured to with the at least one processor cause the apparatus to at least: receive at least two audio signals from one or more sound sources by two or more microphones of a device; determine at least one user interface element associated with the one or more sound sources being displayed by a display of the device based on the at least two audio signals, wherein the at least one user interface element is configured to be modified by an indicator; determine at least one spatial component of the one or more sound sources comprising at least one directional component; generate a visualization of the one or more sound sources, wherein the visualization is configured to rotate or change the at least one directional component of the one or more sound sources dependent on the indicator for the at least one user interface element, the at least one directional component based on the at least two audio signals, wherein generating the visualization further comprises the at least one directional component for the one or more sound sources being mapped to the visualization, wherein the at least one directional component comprises direction and magnitude of the one or more sound sources; and render the at least two audio signals to generate a modified version of the at least one spatial component of the one or more sound sources after being mapped to the visualization dependent on the indicator, wherein the visualization comprises a rotation direction indication of a swipe gesture for the visualization of the one or more sound sources to cause to rotate the at least one directional component of the modified version of the at least one spatial component during rendering together with said magnitude of the one or more sound sources in such a way that the one or more sound sources is reproduced with the modified version of the at least one spatial component based on the swipe gesture.

11

11. The apparatus as claimed in claim 10 , wherein the received at least two audio signals further causes the apparatus to at least one of: receive the at least two audio signals from the at least two or more microphones of the device; receive the at least two audio signals from a memory; or receive the at least two audio signals from at least one external apparatus.

12

12. The apparatus as claimed in claim 10 , wherein the determined at least one user interface element further causes the apparatus to: receive the at least one user interface element from a touch screen; determine an action associated with the at least one user interface element; and output the indicator so as to represent the action.

13

13. The apparatus as claimed in claim 10 , wherein the generated visualization of the one or more sound sources further causes the apparatus to: receive the indicator; and modify the visualization of the at least two audio signals dependent on the indicator.

14

14. The apparatus as claimed in claim 10 , wherein the rendered at least two audio signals further causes the apparatus to: receive the indicator; and modify the rendered at least two audio signals dependent on the indicator to follow the visualization of the at least two audio signals.

15

15. The apparatus as claimed in claim 10 , wherein the action associated with the at least one user interface element comprises at least one of: a clockwise rotation; a counter-clockwise rotation; a translational motion; a rotation about an external point; a focus on a determined position; or a tilt.

16

16. The apparatus as claimed in claim 10 , wherein the generated spatial visualization of the one or more sound sources further causes the apparatus to: determine the at least one directional component of the one or more sound sources; and determine a radial value for a visualization sector dependent on a frequency of directional components of the at least two audio signals in a sector range.

17

17. The apparatus as claimed in claim 16 , wherein the determined at least one spatial component of the one or more sound sources comprising the at least one directional component further causes the apparatus to: group time domain samples of the at least two audio signals into frames; time to frequency domain convert the frames of the at least two audio signals; divide the frequency domain representation of the at least two audio signals into at least two sub-bands; and compare pairs of audio signal sub-bands to determine the at least one directional component.

18

18. The apparatus as claimed in claim 16 , wherein the generated visualization of the one or more sound sources further causes the apparatus to: determine at least one magnitude component for the at least one directional component of the one or more sound sources; and determine a color value for the visualization sector dependent on the at least one magnitude component for the at least one directional component of the one or more sound sources.

19

19. The apparatus as claimed in claim 16 , wherein the rendered at least two audio signals to generate the modified version of the at least one spatial component causes the apparatus to: generate a head related transfer function; and apply the head related transfer function to the at least two audio signals.

20

20. The method as claimed in claim 1 , wherein the at least one user interface element further comprises a synchronize timing of the rotation direction indication with a timing of the spatial visualization of the one or more sound sources.

21

21. The apparatus as claimed in claim 10 , wherein the at least one user interface element further comprises a synchronize timing of the rotation direction indication with a timing of the spatial visualization of the one or more sound sources.

22

22. A computer program product comprising a non-transitory computer readable storage medium, the non-transitory computer readable storage medium comprising program code instructions that, when executed by a processor, configure the processor to: receive at least two audio signals from one or more sound sources by two or more microphones of an apparatus; determine at least one user interface element associated with the one or more sound sources being displayed by a display of the apparatus based on the at least two audio signals, wherein the at least one user interface element is configured to be modified by an indicator; determine at least one spatial component of the one or more sound sources comprising at least one directional component; generate a visualization of the one or more sound sources, wherein the visualization is configured to rotate or change the at least one directional component of the one or more sound sources dependent on the indicator for the at least one user interface element, the at least one directional component based on the at least two audio signals, wherein generating the visualization further comprises the at least one directional component for the one or more sound sources being mapped to the visualization, wherein the at least one directional component comprises direction and magnitude of the one or more sound sources; and render the at least two audio signals to generate a modified version of the at least one spatial component of the one or more sound sources after being mapped to the visualization dependent on the indicator, wherein the visualization comprises a rotation direction indication of a swipe gesture for the visualization of the one or more sound sources to cause to rotate the at least one directional component of the modified version of the at least one spatial component during rendering together with said magnitude of the one or more sound sources in such a way that the one or more sound sources is reproduced with the modified version of the at least one spatial component based on the swipe gesture.

23

23. The computer program product as claimed in claim 22 , wherein the at least one user interface element further comprises a synchronize timing of the rotation direction indication with a timing of the spatial visualization of the one or more sound sources.

24

24. The method as claimed in claim 1 , wherein the at least one directional component further comprises the direction and the magnitude of the one or more sound sources at each frequency band and time frame of the at least two audio signals.

Patent Metadata

Filing Date

Unknown

Publication Date

November 27, 2018

Inventors

Sampo Vesa
Erika Reponen
Anssi Rämö
Ravi Shenoy
Mikko Tammi

Want to explore more patents?

Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.

Citation & reuse

Analysis on this page is generated by Patentable — an AI-powered patent intelligence platform. AI-generated summaries, explanations, and analysis may be reused with attribution and a visible link back to the canonical URL below. Patent abstracts and claims are USPTO public domain.

Cite as: Patentable. “VISUAL SPATIAL AUDIO” (10140088). https://patentable.app/patents/10140088

© 2026 Patentable. All rights reserved.

Patentable is a research and drafting-assistant tool, not a law firm, and does not provide legal advice. Documents we generate are drafts for review by a licensed patent attorney.

VISUAL SPATIAL AUDIO — Sampo Vesa | Patentable