Head-Wearable Augmented Vision Apparatus

The present application claims the priority benefits of International Patent Application No. PCT/GB2023/052121, filed on Aug. 10, 2023, and claims benefit of GB 2211923.4, filed on Aug. 15, 2022, which are hereby incorporated herein by reference in their entireties.

The present disclosure relates to head-wearable augmented vision apparatus, particularly in the context of providing augmented display output for people with sensory, perceptual or cognitive disorders that affect their ability to process and interpret visual and auditory stimuli.

Dedicated virtual reality (VR) headsets are known but relatively expensive and heavy. Head-mountable cradles exist with optics that allow a smartphone to be held just in front of the eyes and viewed by a user. Both approaches satisfy requirements for creating a virtual reality environment but are not typically suitable for long-term wear or for use in a situation where a user needs to move around in the real world, such as might be the case for a user with some sensory, perceptual or cognitive condition who wears the device to augment awareness of the surroundings. Such devices may capture information about the environment via multiple sensors but it is difficult to convey this information to a user in a natural way. A user may have difficulty keeping balance for example relying solely on visual information provided by such displays. Furthermore, known devices are difficult to wear for long periods and can cause headache, nausea or fatigue.

It is an object of the present disclosure to at least partially address one, some or all of the shortcomings with the prior art discussed above and/or other problems.

According to an aspect of the invention, there is provided a head-wearable apparatus, comprising: a display configured to display information; a sensor configured to sense an environment outside of the apparatus; a data processing system configured to control the display using an output from the sensor; a lens system configured to allow a wearer to focus on the information displayed by the display when the apparatus is worn on a head of the wearer; and a mounting arrangement configured to allow the apparatus to be worn on the head of the wearer, the mounting arrangement comprising: a head engagement portion configured to fit over and/or around the head; and a projecting portion mechanically attached to the head engagement portion and configured to extend away from the head in a generally forwards direction relative to the face of the wearer when the apparatus is worn on the head of the wearer, wherein the projecting portion is configured to support a weight of at least the display and the lens system.

Thus, an arrangement is provided that allows a user to view displayed information via a display that is worn by a user but while the weight of at least the display and a lens system is supported by a projecting portion that in turn is supported by a head engagement portion that fits over and/or around (e.g., encircling) the head. The head engagement portion may be, or have substantially the same form as, a hat crown, the hat crown being open or closed. The projecting portion may be, or have substantially the same form as, a hat brim, the hat brim extending only in the forwards direction or in all directions. The approach of the invention has been found to provide a significantly higher degree of comfort relative to known head-worn devices having active displays, which are typically secured to the face area of the head of the wearer via harnesses that typically apply pressure in the vicinity of eyes, ears and nose and/or to the nose, cheekbones, temples, ears and forehead, which, with prolonged use, can lead to headache, nausea or fatigue. Where it is desired to open up peripheral vision, for example by switching from a virtual reality form factor to a glasses-like form factor, the shortcomings of the prior art are made even worse because there are now typically fewer points of contact while the weight remains largely unchanged. Some use cases, such as visual aid, require the user to wear the device for many hours a day.

In an embodiment, the apparatus comprises a face contacting member supported by the projecting portion, the face contacting member configured to engage against a face of the wearer. The projecting portion may be configured to pivot under gravity to press the face contacting member against the face of the wearer and thereby provide stable positioning of the lens system relative to the face of the wearer. This approach has been found to provide a simple and effective way of ensuring correct positioning of the lens system without compromising long term wear comfort.

In an embodiment, the apparatus further comprises an abutment member configured such that, when the wearer is looking in a horizontal direction: the face contacting member extends substantially horizontally; and the abutment member extends substantially downwardly from the face contacting member and limits a range of the pivoting of the projecting portion by pressing against the face below the face contacting member. This approach has been found to allow particularly precise positioning of the lens system with minimal negative impact on long term wear comfort. This feature furthermore reduces mechanical demands on the projecting portion of the mounting arrangement, allowing for example a wider range of hat brims, including weaker hat brims, to be used to implement the projecting portion.

In an embodiment, the abutment member is configured to allow adjustment of the angle between the face of the wearer and axes of lenses of the lens system. This approach may allow the wearer to vertically control which portion of a scene is sensed by the sensor without the wearer needing to change an orientation of his/her head.

In an embodiment, the apparatus is configured such that when the face contacting member is engaged against the face the wearer has peripheral vision of the environment outside of the apparatus. Allowing peripheral vision makes it easier for a user to keep balance in comparison to alternative arrangements in which the user has to rely solely on central vision and/or where peripheral vision is blocked. Peripheral vision is important for spatial navigation (e.g., left and right monocular temporal crescents are used in spatial awareness and spatial learning).

In an embodiment, the apparatus comprises an actuatable shrouding arrangement configured to allow controllable variation of an extent of the peripheral vision. This feature provides flexibility to adapt to different use cases or scenarios and/or further improve comfort. Neither glasses-like systems (with peripheral vision uninhibited) nor VR-based system (peripheral vision fully blocked) cater optimally for all use cases: each form factor has advantages and disadvantages when it comes to specific applications. For example, glasses that ordinarily allow some use of peripheral vision for navigation could benefit from decrease of brightness of the scenes available to peripheral vision if ambient light is too strong. VR-based systems could benefit from “opening up” to both let peripheral vision be used in navigation like with glasses but also so that more of the face of the wearer is exposed to the person the wearer is interacting with (or to a front facing camera if interacting remotely), allowing to better capture facial expressions and emotions.

These and other objects, advantages and features of the invention will become apparent upon review of the following specification in conjunction with the drawings.

The present disclosure relates to a head-wearable apparatus. Example arrangements are discussed below.

The apparatus comprises a display, a sensor, and a data processing system.

The display is configured to display information. The display may comprise an electronic display for example. The display may be opaque or transparent.

The sensor is configured to sense an environment outside of the apparatus. For example, the sensor may be configured to perform one or more of the following in any combination: capture visual scenes; record audio data; acquire multi-point distance information across a field-of-view, optionally by performing light detection and ranging, LiDaR; measure linear acceleration; measure intensity of ambient light; measure magnetic field or magnetic dipole moment; and measure angular velocity.

The data processing system is configured to control the display using an output from the sensor. Any of various known configurations may be provided for providing the required data processing functionality (e.g., including CPUs, GPUs, memory, power, etc.). For example, the data processing system may be configured such that the control of the display using the output from the sensor comprises one or more of the following in any combination: segmentation and matting; localization and mapping; enhancement of colour; adjustment of brightness; adjustment of contrast; tracking of objects; estimation of poses; recognition and/or parsing of textual information; recognition of objects and/or attributes of objects; measurement of distance to objects; location of objects and boundaries of objects; estimation of a change in position; detection of obstacles; parsing of spoken language and/or translation; and detection of faces, emotions and/or actions of people.

1 2 FIGS.and 2 The display, sensor and data processing system may be provided by a portable computing apparatus such as a smartphone. Arrangements of this type are exemplified in the figures. Referring to, for example, the apparatus may comprise a smartphone holderand a smartphone supported by the smartphone holder.

2 2 2 3 2 2 2 2 2 FIG. 2 FIG. The smartphone holderis configured to hold the smartphone. As depicted in, the smartphone holdermay, for example, comprise a cage defining an internal volume into which a smartphone may be placed and held securely.shows the cage in a closed state without a smartphone in place. The cage may be opened by a user and a smartphone placed inside. Any of various known techniques may be used to allow a range of different sizes and shapes of smartphone to be held appropriately in the internal volume, including specially dimensioned adaptors and/or resilient members. The smartphone holderdefines at least one openingor transparent portion configured to allow a camera (an example of a sensor) of the smartphone to be able to capture images outside of the smartphone holderwhen the smartphone is held in the smartphone holder. Images may, for example, be captured of a region in front of the smartphone holder(on the opposite side of the smartphone holder from the wearer) and/or of a region behind the smartphone holder(e.g., of the wearer himself/herself, for example to capture emotions etc. of the wearer).

4 10 12 4 2 18 4 4 The apparatus further comprises a lens system, and a mounting arrangement comprising a head engagement portionand a projecting portion. The lens systemis configured to allow a wearer of the apparatus to focus on the information displayed by the display (e.g., to focus on the display of a smartphone held in the smartphone holder) when the apparatus is worn on the head of the wearer. Lens systems configured to allowing focusing on objects closer to the eye than would be possible without lenses are well known and any suitable configuration of lenses may be used. Lenses for each eye may be provided in respective lens housings. Lenses of the lens systemwill typically be separate from the display but this is not essential. The display could be partly or entirely integrated (e.g., embedded) into one or more lenses of the lens system.

10 12 10 12 10 2 12 2 4 10 1 FIG. 11 FIG. The mounting arrangement formed by the head engagement portionand the projecting portionallow the apparatus to be worn on the head. The engagement portionis configured to fit over and/or around (e.g., encircling) the head in such a way that the projecting portion, which is mechanically attached to the head engagement portion, can support the weight of at least the display and the lens system, optionally also the sensor and data processing system, without the head engagement portion disengaging from or shifting significantly on the head. In arrangements comprising a smartphone held in a smartphone holder, the projecting portionis configured to be able to support the smartphone, the smartphone holder, and the lens systemwithout the head engagement portion disengaging from or shifting significantly on the head. The engagement portionmay have substantially the same form as the crown of a hat (a hat crown). The hat crown may be closed, as in the example of. In other arrangements, the hat crown may be open. An example of a mounting arrangement having a head engagement portion in the form of an open hat crown is shown in.

12 10 12 12 12 FIG. The projecting portionis mechanically attached to the head engagement portionand configured to extend away from the head in a generally forwards direction relative to the face of the wearer when the apparatus is worn on the head of the wearer. The projecting portionmay have substantially the same form as the brim of a hat (a hat brim). The hat brim may extend in the forwards direction only, as in the example of Figure I. In other arrangements, the hat brim may extend in all directions. An example of a mounting arrangement having a projecting portionin the form of a hat brim extending in all directions is shown in.

12 12 The display, sensor, data processing system and lens system may be provided in a unit connected to the projecting portion. The unit may be detachably connected to the projecting portion.

14 14 12 14 12 14 4 2 12 12 10 14 4 14 14 1 FIG. In some arrangements, the apparatus comprises a face contacting member. The face contacting membermay be supported by the projecting portionof the mounting arrangement. The face contacting memberis configured to engage against a face of the wearer, typically against an upper portion of the face such as the forehead. The projecting portionis configured to pivot under gravity to press the face contacting memberagainst the face of the wearer and thereby provide stable positioning of the lens systemrelative to the face of the wearer. Thus, the weight of the display and the lens system (and, optionally, the sensor and data processing system, such as when these elements are provided by a smartphone in a smartphone holder) may apply a torque to the projecting portionthat causes it to bend downwards (e.g., to pivot about an axis in the vicinity of where the projecting portionconnects to the head engagement portion) until the face contacting memberpresses against the face with sufficient force to balance the torque. This arrangement ensures that the apparatus can be quickly and reliably mounted in such a way that the lens systemis positioned appropriately in front of the wearer's eyes with no or a minimum of time-consuming adjustments needing to be made by the wearer (e.g., to align the lens system and/or adjust focusing of the lenses). It has been found that this functionality can be achieved particularly effectively by arranging for the face contacting memberto contact the face along an elongate path conforming with the head of the wearer. The face contacting membermay in particular be configured such that the elongate path has an axis of elongation lying substantially in a horizontal plane when the apparatus is worn on the head of the wearer and the wearer is looking in a horizontal direction, as exemplified infor example.

14 15 14 12 2 4 15 14 15 4 15 4 15 12 15 15 14 12 In some arrangements, the face contacting memberis arranged to extend substantially horizontally when a wearer is looking in the horizonal direction and the apparatus further comprises an abutment memberextending substantially downwardly from the face contacting memberand configured to limit a range of pivoting of the projecting portionunder the weight of elements attached to the projecting portion (e.g., smartphone holder, smartphone, and lens system). The pivoting is limited by the abutment memberexerting pressure (i.e., pressing) against the face at a position below the face contacting member. The abutment memberthus helps to reliably fix the position and alignment of the lens systemrelative to the eyes of the wearer. In some arrangements, the abutment memberis configured to allow adjustment of the angle between the face of the wearer and axes of lenses of the lens system. The abutment membercan thereby precisely control the angle between the face of the wearer and the projecting portion, allowing the user to better control which part of the scene is being captured by the device (e.g., sensed by the sensor) without changing an orientation of the head. In some arrangements, the abutment memberis configured to substantially conform with the external shape of bone structure between the eyes of the wearer. The abutment membermay be connected to and/or supported by the face contacting memberand/or the projecting portionof the mounting arrangement.

14 In some arrangements, the apparatus is configured such that when the face contacting memberis engaged against the face the wearer has peripheral vision of the environment outside the apparatus. The apparatus thus fits on the wearer in such a way as to allow a degree of peripheral vision. Allowing peripheral vision improves comfort for a wearer, particularly where the apparatus is configured to provide support for a visually impaired person interacting with the environment, for example by augmenting vision as the wearer moves through and/or interacts with the environment. The smartphone may, for example, be configured to capture visual information using a camera of the smartphone and display a processed version of the captured visual information on a display of the smartphone. The processed version of the captured visual information may be configured to be more easily interpretable by the visually impaired wearer than the captured visual information. Allowing peripheral vision in this context may make it easier for the wearer to maintain balance, thereby enhancing safety.

captured images can be subject to colour enhancement, e.g. through adjustment of the contrast based on the lighting conditions of a scene; captured images can be subject to edge enhancement by detecting the edges in a scene and highlighting them in a way that facilitates their identification by the user; captured images can be subject to object recognition, that is, specific objects in a scene are detected and the nature and attributes of those objects are conveyed to the user via some output mechanism; captured images can be subject to motion tracking of relevant objects, that is, the direction in which a specific object in a scene moves is highlighted in a way that makes it easier to identify this motion; motion tracking together with object recognition can be used for obstacle avoidance by conveying information about an obstacle and its potential direction to the user; captured textual information present in images can be subject to optical character recognition in order to help the user understand the content of a block of text; captured images can be subject to stabilization, that is, in case that the user's head is unstable or shaking, the captured frames are stabilized to compensate for the head motion and present a more stable and more consistent scene to the user; the distance measure module can provide information about the distance to different objects; this together with object recognition can help the user have an estimation of the distance to an object of interest or can help them identify the distance to a specific obstacle; audio data can be captured via the device's microphones; this data can be used to parse human spoken language and trigger actions based on the processed input; directional information in audio stream can be extracted, combined with inputs from other sensors and assist in spatial cognition and spatial awareness. The data processing system may be configured to augment information displayed by the display in a variety of different ways. These may include one or more of the following:

12 The peripheral vision may be allowed in all peripheral directions or in a selected subset of the available directions. In some implementations, peripheral vision may be allowed in lateral directions and/or in a downwards direction. Peripheral vision in the upwards direction maybe blocked by the projecting portionof the mounting arrangement.

In some arrangements, a degree to which peripheral vision is allowed may be controlled by the wearer and/or by the data processing system. The apparatus may thus be switchable between different modes. In some situations, for example, it may be desirable to completely block peripheral vision and allow a wearer to focus entirely on an output from the display. This may be appropriate where the wearer is sat down or otherwise physically inactive. Alternatively, the surrounding environment may be excessively bright or otherwise distracting, such that it would be more comfortable to suppress peripheral vision. In other situations, for example where the wearer is interacting more actively with the surroundings and/or moving about, it may be desirable to switch the apparatus to a mode allowing peripheral vision or allowing peripheral vision to a greater extent. These operations may be performed manually by the wearer or automatically by the data processing system. For example, the data processing system may detect when the wearer switches from an inactive state to an active state using a motion sensor (e.g., accelerometer) and respond by increasing peripheral vision. Alternatively or additionally, the data processing system may detect changes in the intensity of ambient light and respond by modifying the peripheral vision (reducing peripheral vision when an increase in intensity is detected, such as when the sun comes out, and increasing peripheral vision when a decrease in intensity is detected). The apparatus can thus automatically seek to provide an optimal balance between light from the display and ambient light entering via peripheral vision.

3 5 FIGS.to In some arrangements, the apparatus comprises an actuatable shrouding arrangement configured to allow controllable variation of an extent of the peripheral vision. The variation may be controlled by the data processing system, by the wearer, or may happen automatically for other reasons (e.g., via materials that respond to different intensities of ambient light). The variation may be achieved at least partly by varying a transparency (e.g., transmittance) of a material of variable transparency. The shrouding arrangement may thus comprise a material having variable transparency. Alternatively, or additionally, as exemplified in, the variation may be achieved mechanically (by movement and/or rotation of one of more elements).

3 5 FIGS.to 3 5 FIGS.- 16 2 16 16 16 16 16 depict an example of a class of arrangement in which the apparatus can control peripheral vision using an actuatable shrouding arrangement. For ease of illustration, the smartphone holderis not shown in. The shrouding arrangementmay be actuated manually via direct manual manipulation from the wearer. Alternatively, the shrouding arrangementmay be actuated electrically, for example via a motor or any other suitable powered mechanism. Such actuation may be controlled by the data processing system, for example in response to output from the sensor. The actuatable shrouding arrangementis configured to allow the shrouding arrangementto be selectively switched between an open state and one or more peripheral vision inhibiting states. In the particular example shown, the shrouding arrangementcomprises L-shaped members, one for each eye, that block peripheral vision laterally and in a downwards direction. Various other arrangements are possible.

16 14 16 3 4 FIGS.and The or each peripheral vision inhibiting state of the shrouding arrangement, exemplified in, is such that when the face contacting memberis engaged against the face of the user the user is able to focus on the information displayed by the display and the shrouding arrangementinhibits peripheral viewing of the surrounding environment relative to the open state.

16 14 5 FIG. The open state of the shrouding arrangement, exemplified in, is such that when the face contacting memberis engaged against the face of the wearer the wearer can focus on the information displayed by the display and peripherally view a portion of the environment.

3 5 FIGS.to 3 4 FIGS.and 16 4 16 The one or more peripheral vision inhibiting states may comprise a plurality of peripheral vision inhibiting states, with each peripheral vision inhibiting state inhibiting peripheral vision to a different extent. In the arrangement of, for example, the shrouding arrangementis actuatable to position the L-shaped members at a plurality of different positions along a longitudinal displacement axis parallel to optical axes of the lens system. Different peripheral vision inhibiting states may be provided by allowing the L-shaped members to be positionable at one or more positions that are intermediate between a fully blocking state (e.g., as shown in) where peripheral vision is completely blocked and a fully open state where the L-shaped members do not inhibit peripheral vision at all. At such intermediate positions the shrouding arrangementmay reduce peripheral vision but does not block peripheral vision completely. Providing such a plurality of peripheral vision inhibiting states provides enhanced control for the wearer. For example, the wearer could adapt the degree of blocking of peripheral vision as a function of a brightness of the surrounding environment, e.g., to provide a higher level of blocking when the wearer is outside, particularly in sunny weather, and a lower level of blocking when the wearer is inside or where the weather is overcast or when the sun is not at a high level in the sky etc.

16 16 16 16 3 5 FIGS.- The shrouding arrangementmay be configured in a range of different ways to achieve the desired functionality. In some arrangements, at least a portion of the shrouding arrangementis configured to move and/or rotate so as to be positioned closer to the face of the wearer in the one or more peripheral vision inhibiting states than in the open state. As described above, in the example of, L-shaped members of the shrouding arrangementmove longitudinally. In an alternative arrangement, the shrouding arrangementmay comprise a hinged shrouding element that is rotatable about an axis of the hinge from a blocking position to an open position. Alternatively or additionally, the shrouding arrangement may comprise a plurality of pins that are individually moveable along mutually parallel pin axes. Each pin blocks a portion of the peripheral vision when in a longitudinally advanced position. By selectively advancing available pins it is possible to vary the extent and directionality of peripheral vision blocking in a highly flexible manner Alternatively or additionally, the shrouding arrangement can be installed in a permanently closed state but made from material of variable transparency, with the level of transparency being controlled by the data processing system based on the use case (e.g., viewing photos vs navigating environment) or based on sensor input (e.g., amount of light as read by ambient light sensor).

4 18 18 4 18 18 18 16 16 18 16 18 5 FIG. 5 FIG. 3 4 FIGS.and In some arrangements the lens systemcomprises two tubular lens housings. Each lens housingcontains one or more of the lenses of the lens systemand is aligned such that the wearer can look axially through the lens housing, and through the lenses contained by the lens housing, with a respective eye. Thus, a left eye would look through one of the lens housingsand the right eye would look through the other one. In the open state of the shrouding arrangement, as exemplified in, the shrouding arrangementis positioned at a same distance, or further (as shown in), from the face of the wearer than each lens housing. In the or each peripheral vision inhibiting state, as exemplified in, the shrouding arrangementis positioned closer to the face of the wearer than each tubular lens housing.

4 4 4 4 12 12 In some arrangements, the lens systemis configured to be switchable between a viewing mode and a storage mode. The viewing mode is such that the lens systemis in a directly forwards line of sight of the wearer of the apparatus. The storage mode is such that the lens systemis outside of the directly forwards line of sight of the wearer of the apparatus, optionally with lenses of the lens systemfolded towards the projecting portionof the mounting arrangement, optionally so as to be parallel and/or flush with (e.g., directly adjacent to) the projecting portionof the mounting arrangement.

6 10 FIGS.- 6 FIG. 7 10 FIGS.- 8 10 FIGS.- 18 18 4 4 20 18 2 18 In some arrangements, as exemplified in, the switching between the viewing mode and the storage mode may be facilitated by arranging for the lens housingsto be switchable between an axially extended state (shown in) and an axially contracted state (shown in). This may be achieved by providing the lens housingswith walls formed from a malleable/deformable material or by configuring the walls to be compressible longitudinally in the manner of a bellows or concertina. This allows an overall thickness of the lens system, in a direction parallel to the optical axes, to be reduced when required. The reduction in thickness facilitates folding away of the lens system, for example when the wearer does not wish to use the apparatus. As depicted in, the apparatus may comprise a pivotable support memberthat allows the lens housings(and smartphone holderin the arrangement shown) to be pivoted to the storage position when the lens housingsare in the axially contracted state.

Changes and modifications in the specifically-described embodiments may be carried out without departing from the principles of the present invention, which is intended to be limited only by the scope of the appended claims as interpreted according to the principles of patent law including the doctrine of equivalents.