Patentable/Patents/US-10560752
US-10560752

Apparatus and associated methods

PublishedFebruary 11, 2020
Assigneenot available in USPTO data we have
Inventorsnot available in USPTO data we have
Technical Abstract

An apparatus configured to: in respect of virtual reality content comprising video imagery and spatial audio wherein the perceived direction to an audio event related to a visual event in the video imagery corresponds to a visual location of said visual event as presented in the virtual reality space; based on a change in the viewing direction of the virtual reality view by the user from a first viewing direction to a second viewing direction, and wherein said second viewing direction corresponds to, within a direction threshold, a direction of a particular audio event; providing for display of missed event content thereby allowing a user to see a missed visual event for which they heard the associated audio event and reacted thereto by changing the viewing direction.

Patent Claims
20 claims

Legal claims defining the scope of protection, as filed with the USPTO.

1

1. An apparatus comprising: at least one processor; and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following: in respect of virtual reality content comprising video imagery configured to provide a virtual reality space for viewing in virtual reality, wherein a virtual reality view presented to a user provides for viewing of the virtual reality content, the virtual reality view comprising a spatial portion of the video imagery that forms the virtual reality space and being smaller in spatial extent than the spatial extent of the video imagery of the virtual reality space, the virtual reality content including spatial audio wherein the perceived direction to an audio event related to a visual event in the video imagery corresponds to a visual location of said visual event as presented in the virtual reality space; based on a change in the viewing direction of the virtual reality view by the user from a first viewing direction to a second viewing direction, and wherein said second viewing direction corresponds to, within a direction threshold, a direction of a particular audio event, a start of which occurred prior to the change in the viewing direction; provide for display of missed event content, the missed event content comprising at least visual imagery extracted from the video imagery of the virtual reality content that shows the visual event corresponding to the particular audio event, a temporal start point of the missed event content being prior to a temporal point in the virtual reality content when the change in the viewing direction occurred and based on the particular audio event.

2

2. An apparatus according to claim 1 , wherein the visual event related to the particular audio event was not visible in the virtual reality view when directed in the first viewing direction or, in the virtual reality view when directed in the first viewing direction, the user's gaze did not lie upon said visual event when the corresponding particular audio event occurred.

3

3. An apparatus according to claim 1 , wherein the temporal start point of the missed event content is based on the time at which the particular audio event is deemed to have begun based on one or more of analysis of the particular audio event relative to any other audio presented or predetermined audio event data that defines at least the timing of audio events in the virtual reality content.

4

4. An apparatus according to claim 1 , wherein the provision of the missed event content is further based on the rate of change in viewing direction between the first viewing direction and the second viewing direction exceeding a direction-change threshold.

5

5. An apparatus according to claim 1 , wherein the provision of the missed event content is further based on one or more of the volume or frequency content of the particular audio event relative to an audio threshold.

6

6. An apparatus according to claim 1 , wherein determination of the particular audio event is based on one or more of; predetermined audio event data comprising the direction to one or more audio events occurring in the virtual reality content and a correlation between the second viewing direction, within the direction threshold, and the direction of one of the one or more audio events; or audio analysis of the spatial audio in response to the change in viewing direction to identify the particular audio event the user reacted to.

7

7. An apparatus according to claim 1 , wherein the provision of display of the missed event content includes providing for temporarily pausing of the virtual reality content and display of the missed event content in the virtual reality view.

8

8. An apparatus according to claim 1 , wherein in response to the change in viewing direction towards the direction of the particular audio event, provide for display of the virtual reality content in reverse with respect to time such that the virtual reality content appears to the user as being re-wound to a time point at which the missed event content begins.

9

9. An apparatus according to claim 1 , wherein the missed event content comprises one or more of; missed event video imagery extracted from the video imagery of the virtual reality content; missed event still imagery extracted from the video imagery of the virtual reality content; missed event virtual reality content extracted from the video imagery of the virtual reality content wherein at least an initial viewing direction of the missed event virtual reality content is aligned with the direction of the particular audio event; or missed event virtual reality content extracted from the video imagery of the virtual reality content wherein the missed event virtual reality content has a spatial extent less than the spatial extent of the virtual reality content and is focussed in the direction of the particular audio event.

10

10. An apparatus according to claim 1 , wherein the missed event content is provided for display as picture in picture in the virtual reality view with the virtual reality content.

11

11. An apparatus according to claim 1 , wherein upon completion of the display of the missed event content the display of the virtual reality content is resumed from a time point corresponding to the time of the change in viewing direction from the first viewing direction to the second viewing direction.

12

12. An apparatus according to claim 1 , wherein upon completion of the display of the missed event content the virtual reality content is provided for display, at least temporarily, at a playback speed greater than real-time.

13

13. A method, the method comprising: in respect of virtual reality content comprising video imagery configured to provide a virtual reality space for viewing in virtual reality, wherein a virtual reality view presented to a user provides for viewing of the virtual reality content, the virtual reality view comprising a spatial portion of the video imagery that forms the virtual reality space and being smaller in spatial extent than the spatial extent of the video imagery of the virtual reality space, the virtual reality content including spatial audio wherein the perceived direction to an audio event related to a visual event in the video imagery corresponds to a visual location of said visual event as presented in the virtual reality space; based on a change in the viewing direction of the virtual reality view by the user from a first viewing direction to a second viewing direction, and wherein said second viewing direction corresponds to, within a direction threshold, a direction of a particular audio event, a start of which occurred prior to the change in the viewing direction; providing for display of missed event content, the missed event content comprising at least visual imagery extracted from the video imagery of the virtual reality content that shows the visual event corresponding to the particular audio event, a temporal start point of the missed event content being prior to a temporal point in the virtual reality content when the change in the viewing direction occurred and based on the particular audio event.

14

14. A method according to claim 13 , wherein the visual event related to the particular audio event was not visible in the virtual reality view when directed in the first viewing direction or, in the virtual reality view when directed in the first viewing direction, the user's gaze did not lie upon said visual event when the corresponding particular audio event occurred.

15

15. A method according to claim 13 , wherein the temporal start point of the missed event content is based on the time at which the particular audio event is deemed to have begun based on one or more of analysis of the particular audio event relative to any other audio presented or predetermined audio event data that defines at least the timing of audio events in the virtual reality content.

16

16. A method according to claim 13 , wherein the provision of the missed event content is further based on the rate of change in viewing direction between the first viewing direction and the second viewing direction exceeding a direction-change threshold.

17

17. A method according to claim 13 , wherein the provision of the missed event content is further based on one or more of the volume or frequency content of the particular audio event relative to an audio threshold.

18

18. A method according to claim 13 , wherein determination of the particular audio event is based on one or more of; predetermined audio event data comprising the direction to one or more audio events occurring in the virtual reality content and a correlation between the second viewing direction, within the direction threshold, and the direction of one of the one or more audio events; or audio analysis of the spatial audio in response to the change in viewing direction to identify the particular audio event the user reacted to.

19

19. A method according to claim 13 , wherein the provision of display of the missed event content includes providing for temporarily pausing of the virtual reality content and display of the missed event content in the virtual reality view.

20

20. A non-transitory computer readable medium comprising program instructions stored thereon for performing at least the following: in respect of virtual reality content comprising video imagery configured to provide a virtual reality space for viewing in virtual reality, wherein a virtual reality view presented to a user provides for viewing of the virtual reality content, the virtual reality view comprising a spatial portion of the video imagery that forms the virtual reality space and being smaller in spatial extent than the spatial extent of the video imagery of the virtual reality space, the virtual reality content including spatial audio wherein the perceived direction to an audio event related to a visual event in the video imagery corresponds to a visual location of said visual event as presented in the virtual reality space; based on a change in the viewing direction of the virtual reality view by the user from a first viewing direction to a second viewing direction, and wherein said second viewing direction corresponds to, within a direction threshold, a direction of a particular audio event, a start of which occurred prior to the change in the viewing direction; providing for display of missed event content, the missed event content comprising at least visual imagery extracted from the video imagery of the virtual reality content that shows the visual event corresponding to the particular audio event, a temporal start point of the missed event content being prior to a temporal point in the virtual reality content when the change in the viewing direction occurred and based on the particular audio event.

Classification Codes (CPC)

Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.

Patent Metadata

Filing Date

June 12, 2017

Publication Date

February 11, 2020

Want to explore more patents?

Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.

Citation & reuse

Analysis on this page is generated by Patentable — an AI-powered patent intelligence platform. AI-generated summaries, explanations, and analysis may be reused with attribution and a visible link back to the canonical URL below. Patent abstracts and claims are USPTO public domain.

Cite as: Patentable. “Apparatus and associated methods” (US-10560752). https://patentable.app/patents/US-10560752

© 2026 Patentable. All rights reserved.

Patentable is a research and drafting-assistant tool, not a law firm, and does not provide legal advice. Documents we generate are drafts for review by a licensed patent attorney.