Methods, Systems, Apparatuses, and Devices for Facilitating Provisioning of a Virtual Experience in a Racing Environment

The present patent application claims the benefit of U.S. Provisional Patent Application 63/664,607, filed Jun. 26, 2024, and the present patent application is a continuation-in-part of U.S. Provisional patent application Ser. No. 18/087,042, filed Dec. 22, 2022, which is a continuation-in-part of PCT/US2022/027665 filed May 4, 2022, which claims the benefit of U.S. Provisional Patent Application 63/183,951, filed May 4, 2021, U.S. Provisional Patent Application 63/234,261, filed Aug. 18, 2021, U.S. Provisional Patent Application 63/234,866, filed Aug. 19, 2021, and U.S. Provisional Patent Application 63/335,977, filed Apr. 28, 2022; the entire disclosures of each of which are hereby incorporated herein by reference.

Generally, the present disclosure relates to the field of data processing. More specifically, the present disclosure relates to methods, systems, apparatuses, and devices for facilitating provisioning of a virtual experience.

Display devices are used for various types of training, such as in simulators. Such display devices may display virtual reality and augmented reality content.

However, in some situations, movement of a display device with respect to a user using the display device may alter a perception of the content that may be displayed. For instance, due to a movement of the display device due to external forces, such as movement of display devices in flight helmets due to acceleration of aircraft, the user's perception of the displayed content may change, which is not desired.

Therefore, there is a need for improved methods, systems, apparatuses and devices for facilitating provisioning of a virtual experience that may overcome one or more of the above-mentioned problems and/or limitations.

This summary is provided to introduce a selection of concepts in a simplified form, that are further described below in the Detailed Description. This summary is not intended to identify key features or essential features of the claimed subject matter. Nor is this summary intended to be used to limit the claimed subject matter's scope.

In accordance with exemplary and non-limiting embodiments, a computer system comprises a memory in communication with a processor, the memory storing instructions that when executed by the processor cause the processor to maintain a virtual environment, manage a virtual asset within the virtual environment, map a geospatial position of a real vehicle into a position within the virtual environment representative of the geospatial position. determine a measurement between the position of the real vehicle in the virtual environment to the virtual asset and communicate content representative of the virtual asset to a mixed reality optical system configured for an operator of the real vehicle, wherein the content is presented at a time and position based, at least in part, on the measurement.

As a preliminary matter, it will readily be understood by one having ordinary skill in the relevant art that the present disclosure has broad utility and application. As should be understood, any embodiment may incorporate only one or a plurality of the above-disclosed aspects of the disclosure and may further incorporate only one or a plurality of the above-disclosed features. Furthermore, any embodiment discussed and identified as being “preferred” is considered to be part of a best mode contemplated for carrying out the embodiments of the present disclosure. Other embodiments also may be discussed for additional illustrative purposes in providing a full and enabling disclosure. Moreover, many embodiments, such as adaptations, variations, modifications, and equivalent arrangements, will be implicitly disclosed by the embodiments described herein and fall within the scope of the present disclosure.

Accordingly, while embodiments are described herein in detail in relation to one or more embodiments, it is to be understood that this disclosure is illustrative and exemplary of the present disclosure, and are made merely for the purposes of providing a full and enabling disclosure. The detailed disclosure herein of one or more embodiments is not intended, nor is to be construed, to limit the scope of patent protection afforded in any claim of a patent issuing here from, which scope is to be defined by the claims and the equivalents thereof. It is not intended that the scope of patent protection be defined by reading into any claim a limitation found herein that does not explicitly appear in the claim itself.

Thus, for example, any sequence(s) and/or temporal order of steps of various processes or methods that are described herein are illustrative and not restrictive. Accordingly, it should be understood that, although steps of various processes or methods may be shown and described as being in a sequence or temporal order, the steps of any such processes or methods are not limited to being carried out in any particular sequence or order, absent an indication otherwise. Indeed, the steps in such processes or methods generally may be carried out in various different sequences and orders while still falling within the scope of the present invention. Accordingly, it is intended that the scope of patent protection is to be defined by the issued claim(s) rather than the description set forth herein.

Additionally, it is important to note that each term used herein refers to that which an ordinary artisan would understand such term to mean based on the contextual use of such term herein. To the extent that the meaning of a term used herein—as understood by the ordinary artisan based on the contextual use of such term—differs in any way from any particular dictionary definition of such term, it is intended that the meaning of the term as understood by the ordinary artisan should prevail.

Furthermore, it is important to note that, as used herein, “a” and “an” each generally denotes “at least one,” but does not exclude a plurality unless the contextual use dictates otherwise. When used herein to join a list of items, “or” denotes “at least one of the items,” but does not exclude a plurality of items of the list. Finally, when used herein to join a list of items, “and” denotes “all of the items of the list.”

The following detailed description refers to the accompanying drawings. Wherever possible, the same reference numbers are used in the drawings and the following description to refer to the same or similar elements. While many embodiments of the disclosure may be described, modifications, adaptations, and other implementations are possible. For example, substitutions, additions, or modifications may be made to the elements illustrated in the drawings, and the methods described herein may be modified by substituting, reordering, or adding stages to the disclosed methods. Accordingly, the following detailed description does not limit the disclosure. Instead, the proper scope of the disclosure is defined by the appended claims. The present disclosure contains headers. It should be understood that these headers are used as references and are not to be construed as limiting upon the subjected matter disclosed under the header.

The present disclosure includes many aspects and features. Moreover, while many aspects and features relate to, and are described in the context of facilitating provisioning of a virtual experience, embodiments of the present disclosure are not limited to use only in this context.

is an illustration of an online platformconsistent with various embodiments of the present disclosure. By way of non-limiting example, the online platformto facilitate provisioning of a virtual experience may be hosted on a centralized server, such as, for example, a cloud computing service. The centralized servermay communicate with other network entities, such as, for example, an augmented and virtual reality display device, a sensor systemof an aircraft, database(such as 3D model database) over a communication network, such as, but not limited to, the Internet. Further, users of the online platformmay include relevant parties such as, but not limited to, trainees, trainers, pilots, administrators, and so on.

A user, such as the one or more relevant parties, may access online platformthrough a web based software application or browser. The web based software application may be embodied as, for example, but not be limited to, a website, a web application, a desktop application, and a mobile application compatible with a computing device.

shows a wearable display devicefor facilitating provisioning of a virtual experience. In some embodiments, the wearable display devicemay be utilized in conjunction with and/or to effectuate and/or facilitate operation of any element described elsewhere herein or illustrated in any figure herein. Further, the wearable display devicemay include a support memberconfigured to be mounted on a user. Further, the support membermay include a structure allowing the support memberto be easily mountable on the user. For instance, the wearable display devicemay include a head mounted device (HMD). Further, the wearable display devicemay include a display deviceattached to the support member. For instance, if the wearable display deviceis an HMD, the HMD may include a display device in front of one eye of the user, (a monocular HMD), in front of both eyes of the user, (a binocular HMD), an optical display device (which may reflect projected images), and so on. Further, the display devicemay be configured for displaying at least one display data. Further, the display data may include virtual reality data related to a simulation, such as a training simulation. For instance, the training simulation may correspond to vehicular racing, such as Formula 1®, and may be used by race car drivers to train for race events. Further, in an instance, the training simulation may correspond to flight training, and may be used by air force pilots for flight training in fighter aircraft. Further, in some embodiments, the display data may include augmented reality data. Accordingly, the display data may include one or more augmented reality components overlaid on top of live image. For instance, the augmented reality data may be related to flight training including a first aircraft training simultaneously with a plurality of aircrafts in different locations. Accordingly, the augmented reality data may include augmented reality components displaying the plurality of plurality of aircrafts in different locations to a display device associated with a pilot of the first aircraft. Further, the wearable display devicemay include at least one disturbance sensorconfigured for sensing a disturbance in a spatial relationship between the display deviceand the user. Further, the spatial relationship between the display deviceand the usermay include at least one of a distance and an orientation. For instance, the spatial relationship may include an exact distance, and an orientation, such as a precise angle between the display deviceand the eyes of the user.

Further, the disturbance in the spatial relationship may include a change in at least one of the distance and the orientation between the display deviceand the user. Further, the disturbance in the spatial relationship may lead to an alteration in how the usermay view the at least one display data. For instance, if the disturbance in the spatial relationship leads to a reduction in the distance between the display deviceand the user, the usermay perceive one or more objects in the at least one display data to be closer. For instance, if the spatial relationship between the display deviceand the userspecifies a distance of “x” centimeters, and the disturbance in the spatial relationship leads to a reduction in the distance between the display deviceand the userto “y” centimeters, the usermay perceive the at least one display data to be closer by “x-y” centimeters.

Further, the wearable display devicemay include a processing devicecommunicatively coupled with the display device. Further, the processing devicemay be configured for receiving the at least one display data. Further, the processing devicemay be configured for analyzing the disturbance in the spatial relationship. Further, the processing devicemay be configured for generating a correction data based on the analyzing. Further, the processing devicemay be configured for generating a corrected display data based on the at least one display data and the correction data. Further, the correction data may include an instruction to shift a perspective view of the at least one display data to compensate for the disturbance in the spatial relationship between the display deviceand the user. Accordingly, the correction data may be generated contrary to the disturbance in the spatial relationship.

For instance, the disturbance may include an angular disturbance, wherein the display devicemay undergo an angular displacement as a result of the angular disturbance. Accordingly, the correction data may include an instruction of translation of the display data to compensate for the angular disturbance. Further, the display data may be translated along a horizontal axis of the display data, a vertical axis of the display data, a diagonal axis of the display data, and so on, to negate the angular displacement of the display data.

Further, in an instance, the disturbance may include a longitudinal disturbance, wherein the display devicemay undergo a longitudinal displacement as a result of the longitudinal displacement. Accordingly, the correction data may include an instruction of translation of the display data to compensate for the longitudinal disturbance. Further, the display data may be projected along a distance perpendicular to a line of sight of the userto negate the angular displacement of the display data. For instance, the display data may be projected along a distance perpendicular to the line of sight of the useropposite to a direction of the longitudinal disturbance to compensate for the longitudinal disturbance.

Further, the support membermay include a head gear configured to be mounted on a head of the user. Further, the head gear may include a helmet configured to be worn over a crown of the head. Further, the head gear may include a shell configured to accommodate at least a part of a head of the user. Further, a shape of the shell may define a concavity to facilitate accommodation of at least the part of the head. Further, the shell may include an interior layer, an exterior layerand a deformable layerdisposed in between the interior layerand the exterior layer. Further, the deformable layermay be configured to provide cushioning. Further, the display devicemay be attached to at least one of the interior layerand the exterior layer.

Further, the disturbance in the spatial relationship may be based on a deformation of the deformable layerdue to an acceleration of the head gear. Further, the spatial relationship may include at least one vector representing at least one position of at least one part of the display devicein relation to at least one eye of the user. Further, a vector of the at least one vector may be characterized by an orientation and a distance. For instance, the spatial relationship between the display deviceand the usermay include at least one of a distance and an orientation. For instance, the spatial relationship may include an exact distance, and an orientation, such as a precise angle between the display deviceand the eyes of the user. Further, the spatial relationship may describe an optimal arrangement of the display devicewith respect to the user. Further, so that the optimal arrangement of the display devicewith respect to the usermay allow the user to clearly view the display data without perceived distortion.

Further, in some embodiments, the at least one disturbance sensormay include an accelerometer configured for sensing the acceleration. Further, in some embodiments, the at least one disturbance sensormay include at least one proximity sensor configured for sensing at least one proximity between the at least one part of the display deviceand the user. Further, in some embodiments, the at least one disturbance sensormay include a deformation sensor configured for sensing a deformation of the deformable layer.

Further, in some embodiments, the display devicemay include a see-through display deviceconfigured to allow the userto view a physical surrounding of the wearable device.

Further, in some embodiments, the at least one display data may include at least one object model associated with at least one object. Further, in some embodiments, the generating of the corrected display data may include applying at least one transformation to the at least one object model based on the correction data.

Further, the applying of the at least one transformation to the at least one object model based on the correction data may include translation of the display data to compensate for the angular disturbance. For instance, the correction data may include one or more instructions to translate the display data along a horizontal axis of the display data, a vertical axis of the display data, a diagonal axis of the display data, and so on, to negate the angular displacement of the display data. Accordingly, the applying of the at least one transformation to the at least one object model based on the correction data may include translation of the display data along the horizontal axis, the vertical axis, and the diagonal axis of the display data, to negate the angular displacement of the display data. Further, in an instance, if the correction data includes an instruction of translation of the display data to compensate for the longitudinal disturbance, the applying of the at least one transformation to the at least one object model based on the correction data may include translation may include projection of the display data along a distance perpendicular to a line of sight of the userto negate the angular displacement of the display data. For instance, the applying of the at least one transform may include projection of the display data along a distance perpendicular to the line of sight of the useropposite to a direction of the longitudinal disturbance to compensate for the longitudinal disturbance.

Further, in some embodiments, the at least one disturbance sensormay include a camera configured to capture an image of each of a face of the userand at least a part of the head gear. Further, the spatial relationship may include disposition of at least the part of the head gear in relation to the face of the user.

Further, in some embodiments, the at least one disturbance sensormay include a camera disposed on the display device. Further, the camera may be configured to capture an image of at least a part of a face of the user. Further, the wearable display devicemay include a calibration input device configured to receive a calibration input. Further, the camera may be configured to capture a reference image of at least the part of the face of the userbased on receiving the calibration input. Further, the calibration input may be received in an absence of the disturbance. For instance, the calibration input device may include a button configured to be pushed by the userin absence of the disturbance whereupon the reference image of at least the part of the face of the usermay be captured. Further, the analyzing of the disturbance may include comparing the reference image with a current image of at least the part of the face of the user. Further, the current image may be captured by the camera in a presence of the disturbance. Further, determining the correction data may include determining at least one spatial parameter change based on the comparing. Further, the at least one spatial parameter change may correspond to at least one of a displacement of at least the part of the face relative to the camera and a rotation about at least one axis of at least the part of the face relative to the camera.

Further, in some embodiments, the generating of the corrected display data may include applying at least one image transform on the at least one display data based on the at least one spatial parameter change.

Further, in some embodiments, the wearable display devicemay include at least one actuator coupled to the display deviceand the support member. Further, the at least one actuator may be configured for modifying the spatial relationship based on a correction data.

Further, the spatial relationship between the display deviceand the usermay include at least one of a distanceand an orientation. Further, the disturbance in the spatial relationship between the display deviceand the usermay include a change in at least one of the distance, the angle, the direction, and the orientation. Further, the distancemay include a perceived distance between the userand the at least one display data. For instance, as shown in, the disturbance in the spatial relationship may originate due to a forward accelerationof the userand the wearable display device. Accordingly, the deformation of the deformable layermay lead to a disturbance in the spatial relationship leading to a change in the distanceto a reduced distancebetween the display deviceand the user. Accordingly, the correction data may include transforming of the at least one display data through object level processing and restoring the at least one display data to the distancefrom the user. Further, the object level processing may include projecting one or more objects in the display data at the distanceinstead of the distanceto oppose the disturbance in the spatial relationship. Further, the disturbance in the spatial relationship may include a change in the angle between the display deviceand the user. Further, the angle between the display deviceand the userin the spatial relationship may be related to an original viewing angle related to the display data. Further, the original viewing angle related to the display data may be a viewing angle at which the usermay view the display data through the display device. Further, the disturbance in the spatial relationship may lead to a change in the original viewing angle related to the display data. Accordingly, the at least one display data may be transformed through pixel level processing to restore the original viewing angle related to the display data. Further, the pixel level processing may include translation of the display data to compensate for the change in the angle in the spatial relationship. Further, the display data may be translated along a horizontal axis of the display data, a vertical axis of the display data, a diagonal axis of the display data, and so on, to negate the angular displacement of the display data to compensate for the change in the angle in the spatial relationship, and to restore the original viewing angle related to the display data.

Further, in some embodiments, the actuator may be configured for modifying the spatial relationship based on the correction data. Further, the correction data may include at least one operational instruction corresponding to the actuator to oppose the disturbance in the spatial relationship, such as, but not limited to, modification of the distance, such as increasing of the distanceto the distance. Further, the correction data may include at least one operational instruction corresponding to the actuator to oppose the disturbance in the spatial relationship such as, but not limited to, the orientation opposing the disturbance in the spatial relationship.

shows a wearable display devicefor facilitating provisioning of a virtual experience, in accordance with some embodiments. In some embodiments, the wearable display devicemay be utilized in conjunction with and/or to effectuate and/or facilitate operation of any element described elsewhere herein or illustrated in any figure herein. Further, the wearable display devicemay include a support memberconfigured to be mounted on a user. Further, the support membermay include a deformable member.

Further, the wearable display devicemay include a display deviceattached to the support member. Further, the display devicemay be configured for displaying at least one display data.

Further, the wearable display devicemay include at least one disturbance sensorconfigured for sensing a disturbance in a spatial relationship between the display deviceand the support member.

Further, the spatial relationship between the display deviceand the usermay include at least one of a distance and an orientation. For instance, the spatial relationship may include an exact distance, and an orientation, such as a precise angle between the display deviceand the eyes of the user. Further, the disturbance in the spatial relationship may include a change in the at least of the distance and the orientation between the display deviceand the user. Further, the disturbance in the spatial relationship may lead to an alteration in how the usermay view the at least one display data. For instance, if the disturbance in the spatial relationship leads to a reduction in the distance between the display deviceand the user, the usermay perceive one or more objects in the at least one display data to be closer. For instance, if the spatial relationship between the display deviceand the userspecifies a distance of “x” centimeters, and the disturbance in the spatial relationship leads to a reduction in the distance between the display deviceand the userto “y” centimeters, the usermay perceive the at least one display data to be closer by “x-y” centimeters.

Further, the wearable display devicemay include at least one actuatorcoupled to the display deviceand the support member. Further, the at least one actuatormay be configured for modifying the spatial relationship between the display deviceand the user. Further, in an embodiment, the at least one actuatormay be configured for modifying the spatial relationship to oppose the disturbance in the spatial relationship. Further, in an embodiment, the at least one actuatormay be configured for modifying the spatial relationship based on the correction data. For instance, the at least one actuatormay be configured for actuating a connected motor, such as an AC motor or a DC motor controlling an extendable rail mechanism connecting the display deviceand the support member. For instance, if the disturbance in the spatial relationship leads to a reduction in the distance between the display deviceand the user, the usermay perceive one or more objects in the at least one display data to be closer. For instance, if the spatial relationship between the display deviceand the userspecifies a distance of “x” centimeters, and the disturbance in the spatial relationship leads to a reduction in the distance between the display deviceand the userto “y” centimeters, the usermay perceive the at least one display data to be closer by “x-y” centimeters. Accordingly, the at least one actuatormay transmit an actuating signal to the connected motor to increase the distance between the display deviceand the userby “x-y” centimeters to the distance of “x” centimeters.

Further, in an embodiment, the at least one actuatormay be connected to a servo motor configured to control the angle in the spatial relationship through a 6-axis rotary mechanism. Accordingly, if the disturbance in the spatial relationship leads to a change in the angle between the display deviceand the user, the usermay perceive the at least one display data to be skewed. For instance, if the spatial relationship between the display deviceand the userspecifies the display deviceto be significantly parallel to the user, and the disturbance in the spatial relationship leads the display deviceto be skewed by an angle of 30 degrees towards the user, the at least one actuatormay transmit an actuating signal to the connected servo motor, which may alter the angle in the spatial relationship by 30 degrees oppositely to the disturbance in the spatial relationship through the 6-axis rotary mechanism.

Further, the wearable display devicemay include a processing devicecommunicatively coupled with the display device. Further, the processing devicemay be configured for receiving the at least one display data. Further, the processing devicemay be configured for analyzing the disturbance in the spatial relationship. Further, the processing devicemay be configured for generating the actuation data based on the analyzing.

shows a wearable display devicefor facilitating provisioning of a virtual experience, in accordance with some embodiments. In some embodiments, the wearable display devicemay be utilized in conjunction with and/or to effectuate and/or facilitate operation of any element described elsewhere herein or illustrated in any figure herein. Further, the wearable display devicemay include a head gearincluding a shell configured to accommodate at least a part of a head of the user. Further, a shape of the shell may define a concavity to facilitate accommodation of at least the part of the head. Further, the shell may include an interior layer, an exterior layerand a deformable layerdisposed in between the interior layerand the exterior layer. Further, the deformable layermay be configured to provide cushioning.

Further, the wearable display devicemay include a display deviceattached to at least one of the interior layerand the exterior layer. Further, the display devicemay be configured for displaying at least one display data.

Further, the wearable display devicemay include at least one disturbance sensorconfigured for sensing a disturbance in a spatial relationship between the display deviceand the at least one of the interior layerand the exterior layer.

Further, the wearable display devicemay include a processing devicecommunicatively coupled with the display device. Further, the processing devicemay be configured for receiving the at least one display data.

Further, the processing devicemay be configured for analyzing a disturbance in the spatial relationship. Further, the processing devicemay be configured for generating a correction data based on the analyzing. Further, the processing devicemay be configured for generating a corrected display data based on the at least one display data and the correction data. Further, the display devicemay be configured to display the corrected display data.

shows a methodfor facilitating provisioning of a virtual experience through a wearable display device, such as the wearable display device, in accordance with some embodiments.

At, the methodmay include receiving, using a communication device, a disturbance data from at least one disturbance sensor. Further, the at least one disturbance sensor may be configured for sensing a disturbance in a spatial relationship between a display device and a user. At, the methodmay include analyzing, using a processing device, the disturbance in the spatial relationship. At, the methodmay include generating, using the processing device, a correction data based on the analyzing. At, the methodmay include generating, using the processing device, a corrected display data based on at least one display data and the correction data. At, the methodmay include transmitting, using the communication device, the corrected display data to the wearable display device. Further, the wearable display device may be configured to be worn by the user. Further, the wearable display device may include a display device. Further, the display device may be configured for displaying the corrected display data.

shows a methodfor determining a spatial parameter change, in accordance with some embodiments. At, the methodmay include receiving, using the communication device, a reference image of at least a part of the face of the user. Further, the at least one disturbance sensor may include a camera disposed on the display device. Further, the camera may be configured to capture an image of at least the part of a face of the user. Further, the wearable display device may include a calibration input device configured to receive a calibration input. Further, the camera may be configured to capture the reference image of at least the part of the face of the user based on receiving the calibration input. Further, the calibration input may be received in an absence of the disturbance.

At, the methodmay include receiving, using the communication device, a current image of at least the part of the face of the user. Further, the current image may be captured by the camera in a presence of the disturbance. At, the methodmay include comparing, using the processing device, the reference image with the current image. At, the methodmay include determining using the processing device, at least one spatial parameter change based on the comparing. Further, the at least one spatial parameter change may correspond to at least one of a displacement of at least the part of the face relative to the camera and a rotation, about at least one axis, of at least the part of the face relative to the camera. Further, the generating of the corrected display data may include applying at least one image transform on the at least one display data based on the at least one spatial parameter change. Further, the part of the face may include the eyes of the user. Further, the reference image may include at least one reference spatial parameter corresponding to the eyes. Further, the current image may include at least one current spatial parameter corresponding to the eyes. Further, the at least one spatial parameter change may be independent of a gaze of the eyes.

is a block diagram of a systemfor facilitating provisioning of a virtual experience in accordance with some embodiments. The systemmay include a communication device, a processing deviceand a storage device.