Augmented Reality Task Flow Optimization Systems

This application is a continuation of U.S. patent application Ser. No. 18/459,160, filed on Aug. 31, 2023, now abandoned, which is a continuation of U.S. Ser. No. 17/498,971, filed on Oct. 12, 2021, now U.S. Pat. No. 11,747,909, which is a continuation of U.S. patent application Ser. No. 16/422,013, filed on May 24, 2019, now U.S. Pat. No. 11,169,613, which claims the benefit of U.S. provisional application No. 62/678,211, filed on May 30, 2018. The entire contents of all of these applications are hereby incorporated in their entirety by reference for all purposes.

An increasing number of workers in a variety of industries are assisted by hand-held or wearable computer systems. Rather than using computer kiosks or workstations at locations throughout the work environment, the hand-held or wearable computers allow the worker to move freely about the workspace and retrieve information from computer networks accessible at their fingertips, while they maintain in proximity to the object of the task. An augmented reality system may be used to further enhance task flows where users need information to complete a task.

The disclosed augmented reality task flow optimization systems will become better understood through a review of the following detailed description in conjunction with the figures. The detailed description and figures provide merely examples of the various embodiments described herein. Those skilled in the art will understand that the disclosed examples may be varied, modified, and altered and not depart from the scope of the embodiments described herein. Many variations are contemplated for different applications and design considerations; however, for the sake of brevity, the contemplated variations may not be individually described in the following detailed description.

Throughout the following detailed description, examples of various augmented reality task flow optimization systems are provided. Related features in the examples may be identical, similar, or dissimilar in different examples. For the sake of brevity, related features will not be redundantly explained in multiple examples. Instead, the use of related feature names will cue the reader that the feature with a related feature name may be similar to the related feature in an example explained previously. Features specific to a given example will be described in that particular example. The reader is to understand that a given feature need not be the same or similar to the specific portrayal of a related feature in any given figure or example.

Individuals that work in industries where their hands must remain free to perform tasks may use wearable computer systems to retrieve and/or view information needed to perform the tasks. For example, individuals working in manufacturing facilities may be required to maintain a clean and hygienic manufacturing area to prevent cross-contamination. Additionally, in the manufacturing environment and other environments, it may be required to record that all the steps and instructions in a defined manufacture procedure were performed. Wearable computer systems may aid in recording all the steps and instructions performed and in investigating or documenting deviations taken during the manufacturing procedure. The records of the manufacturing and distribution of items with a complete manufacturing history may enable a manufacturer to retain the manufacturing information in a comprehensible and accessible format in case an investigation is needed to determine a product defect or failure. In another example, in retail operation environments, sales assistants or inventory control clerks may use wearable computer systems to identify products and/or retrieve product information associated with a product. The wearable computer system may provide guidance and aid individuals in performing tasks. However, different users may perform different tasks or the different users may perform the same task differently. Providing the same guidance and aid to the different users may be ineffective for the individuals who perform the task differently from the default guidelines. Additionally, providing the same guidance and aid to the different users may not aid the individuals in learning how to correctly perform tasks.

Implementations of the disclosure address the above-mentioned deficiencies and other deficiencies by providing methods, systems, devices, or apparatus to display an augmented reality environment to users and provide interactive user interfaces. The methods, systems, devices, or apparatus may also improve a task flow of tasks displayed in an augmented reality system.

In one embodiment, the augmented reality systems may be head-mounted systems. In another embodiment, the augmented reality systems may include sensors and/or detector units to detect, gather, and/or transmit data to a processing device. The processing device may be an integrated or onboard processing device or an external or remote processing device. In another embodiment, the augmented reality systems may include an augmented reality display to display or overlay information to a user. In another embodiment, the augmented reality systems may include providing a user access to remote help, capture images and/or sensor data to send to a remote individual, and so forth. For example, an augmented reality system may transmit a still image or a video from a perspective of the user, also referred to as a “see what I see” perspective.

In another embodiment, the augmented reality systems may overlay images from another user's device into an augmented reality environment display for a viewer to see a projected perspective of the images from the other user's device, such as a re-creation of what is being viewed by the other user. One advantage of the augmented reality systems may be to enhance task flows of users when the users need information displayed to them to perform a task. Another advantage of the augmented reality systems may be to provide a user interface for the users to interact within a variety of environments, include hands-free environments, clean and hygienic environments, collaborative environments, remote environments, and so forth.

illustrates a perspective view of a wearable devicebeing worn by a user, according to an embodiment. The wearable deviceis designed to display an image near an eye or eyes of a userand execute computer-executable instructions in response user gestures by incorporating motion and image capture features. The wearable devicemay include lenssubstantially near the eye(s) of the user. The lensmay include a transparent or partially transparent screen that is at least partially disposed within a field of view of the user. The screen may display an augmented reality where a user may be able to view augmented reality objects. The screen may have a transparency level to enable the userto see gesturesand real-world objects.

illustrates a perspective view of the wearable devicein, according to an embodiment. Some of the features inare the same or similar to some of the features inas noted by same reference numbers, unless expressly described otherwise.

The wearable devicemay include a first temple, a second temple, and a barextending between the two templesand. The first templeand second templeextend from the wearable deviceto rest on the ears of the userinto provide support for the wearable device.

The wearable devicemay include a processing device, the lens, a first image generator, a second image generator, a motion capture device, and a microphone. The wearable devicemay be configured to capture user gestures, along with other objects within a captured scene, and execute computer processes or commands in response to the captured objects. This may involve manipulating a display produced by a first image generatorand/or a second image generatoron lens. For example, the first image generatormay display a first virtual object or portion of a virtual object on a first portion of the lens. In another example, the second image generatormay display a second virtual object or portion of a virtual object on a second portion of the lens. In another example, the lensmay be a single continuous lens extending across the bar. In another example, the lensmay include multiple lens or portions of lens extending across the bar. The first virtual object and/or the second virtual object includes text, a graphical object, a video, a live data steam, and so forth.

The wearable devicemay additionally execute computer processes in response to audio captured by the microphone. By incorporating these capturing technologies. the wearable devicemay display and control computer images and processes. The processing devicemay be supported by the bar. The processing devicemay be configured to execute computer-executable instructions and control the electronic elements of the wearable device.

The processing devicemay acquire data from other elements of the wearable deviceor from external sources and execute computer executable code in response to this data. For example, the processing devicemay be configured to acquire data from the motion capture devicesuch as data that corresponds to a user gesture. The processing devicemay additionally or alternatively acquire data from the microphone. In some examples, the processing devicemay acquire data from a separate device, such as a portable music player, a personal data assistant (PDA), a smartphone, a global positioning system (GPS), or the like.

The processing devicemay be in coupled to the first image generatorand/or the second image generatorand may instruct the first image generatorand/or the second image generatorto generate and manipulate a display projected on lens. In one example, the processing devicemay generate a user interface on the lens. The processing devicemay acquire data from other elements or sensors of the wearable deviceand manipulate the user interface in response to this data.

The first image generatorand/or the second image generatormay be attached to the first templeand may be substantially aligned with the lens. The wearable devicemay include additional image generators to augment the expanse of generated images over the lenssurface area. The first image generatorand/or the second image generatormay be configured to display images on the lensfacing an eye or eyes of the userinin response to instructions executed by the processing device. These images often may overlay and/or augment a naturally viewed scene within a field of vision of the user. The first image generatorand/or the second image generatormay display opaque and/or partially transparent images on the lens.

The motion capture devicemay be connected to the bar. In one embodiment, the motion capture devicemay be oriented away from the userwhen the useris wearing the wearable deviceto capture images and motion occurring beyond the lens. In another embodiment, the motion capture devicemay include an image sensorwith a camera, a light source, and a depth sensor. The motion capture devicemay capture images that include at least a portion of the environment surrounding the wearable device(such as a head-mounted device or a head-mounted display).

The image sensormay be connected to the bar. In one embodiment, the image sensormay be oriented away from the useras the userwears the wearable device. The image sensormay be configured to capture an image from a scene and communicate data corresponding to the captured image to the processing device. The image sensormay capture light within the visual spectrum and/or light outside the visual spectrum. For example, the image sensormay capture infrared or ultraviolet light.

The light sourcemay be connected to the bar. The light sourcemay be substantially aligned with the image sensor. The light sourcemay be configured to project light in a defined direction. This light is reflected from an object and may be captured by the motion capture device. The reflected light may allow the motion capture deviceto more accurately capture images and motion of objects or the surrounding environment. The light sourcemay project visual light and/or light outside the visual spectrum.

The depth sensormay be connected to the bar. The depth sensormay be configured to capture images and motion from a scene at a different angle than image sensor. The data captured from this second viewpoint may allow the processing deviceto compare the data received from the depth sensorto the data received from the image sensorto better detect a depth of objects in the environment surrounding the wearable device.

illustrates an augmented reality systemwith a userwearing the wearable device, according to an embodiment. Some of the features inare the same or similar to some of the features inas noted by same reference numbers, unless expressly described otherwise. The user may be located along a first plane. The wearable devicemay include a lensto display a virtual object to an eye or eyes of the user. For example, the wearable devicemay include a first portion of the lensthat may display a first image or virtual object to a first eye of the userand a second portion of the lensthat may display a second image or virtual object to a second eye of the user. When combined or aggregated, the first image or virtual object and the second image or virtual object may form an augmented reality environment to project one or more virtual or augmented reality objects to the userat defined locations within a physical environment surrounding the user.

The wearable devicemay display multiple images to project one or more virtual or augmented reality objectswithin a physical environment surrounding the user. In one embodiment, the user and the wearable devicemay be located along the first planeand the augmented reality objectmay be displayed to appear at a defined distance from the userat a second plane. In one embodiment, the wearable devicemay include a position sensor, such as a GPS device or a position transceiver. For example, the wearable devicemay be a pair of glasses, smart glasses, a face shield, and so forth that may be worn by the userand include multiple sensors, including the position sensor. In another embodiment, the position sensor may separate from the wearable deviceand be attached to the body of the user.

In one example, the position sensor may be positioned proximate to a line of sight of the user. In another embodiment, an alignment sensor may measure a difference between a position of the userindicated by the position sensor and the actual location of the user's eyes and generates a view aligned to the user's line of sight. In one embodiment, the lensinmay project a virtual or augmented reality environment or a virtual or augmented reality objectat a set focal distance from the user. In one example, the focal distance may be along the second plane. The virtual or augmented reality environment or the virtual or augmented reality objectmay be a user interface that enhances or supplements the user's interactions in the physical environment and/or physical objects approximate or surrounding the user.

illustrates a systemof the wearable devicein, according to an embodiment. The systemmay include a detector, sensors, a transceiver, an augmented reality display, a local processing device, a remote processing device, and an optimizer. In one embodiment, the detectorand/or the sensorsmay take measurements and collect data from a physical environment approximate to the wearable device. In another embodiment, the local processing devicemay be a local processing device attached to or integrated into the wearable device.

The local processing devicemay be positioned proximate to the user that interacts with a remote processing device. In another embodiment, the remote processing devicemay be a remote processing device that may be external to the wearable device. The transceivermay receive instructions from the local processing deviceto project the augmented display. The transceivermay measure characteristics from the physical environment and communicate the data to the local processing deviceand an optimizer.

The optimizermay be in communication with the remote processing device, the local processing device, and the transceiver. The transceivermay measure or detect user characteristics or gestures including head position, movement, speed, etc. In one example, using gestures, the user may open applications to be displayed in the augmented environment. These applications may include task flows, attached documents, and so forth. The gestures may allow the user to access the applications and execute instructions such as select instructions, open instructions, scrolling instructions, movement instructions, and so forth. Alternatively, the gestures may be used to operate on the icon representing the application by changing the size (zooming in and out), color, sound, and so forth. The gestures may include the user performing scrolling gestures, tapping gestures, or clicking gestures with respect to a virtual object. The scrolling gestures may include body motions, head motions, hand motions, and so forth.

illustrates a functional block diagram for the optimizershown in. according to an embodiment. In one example, the databasemay include unique user profiles, industry standards, and so forth. The optimizermay include a memory device to store user profile module. In one example, the user profile information may include information about a user's age, job or responsibilities, experience level, gender, length of employment, previous tasks performed, and so forth. The optimizermay include a time threshold indicatorwith a database to store information indicating a threshold amount of time an individual should take to complete a given task when the task is performed efficiently and correctly. For example, when the user is working on changing the oil of an airplane, the database of the time threshold indicatormay include defined amounts of time for the tasks of removing an oil plug, draining the oil, removing the oil filter, inserting a new oil filter, replacing the plug, and filling the oil pan with oil. In one embodiment, the threshold amounts of time may be set by an expert or an individual experienced in performing the tasks. In another embodiment, the threshold amounts of time may be determined by averaging an amount of time taken by a set of users each performing the tasks, such as during a training period for the optimizer. The threshold amounts of time may be updated or changed over time as the average changes or, as discussed below, based on user profile information.

The optimizermay include a comparatorthat is configured to receive the user profile information from the user profile module, the threshold amounts of time for the individual to perform the tasks from the time threshold indicator, and the actual amounts of time taken by the user to perform the tasks as measured by a sensorof the optimizer. The sensorsmay be the sensors of the wearable deviceas in. The comparatormay determine whether the actual amount of time taken by the user to perform the tasks is within the threshold amounts of time.

In one embodiment, the threshold amounts of time may be fixed, where the threshold amounts of time are the same for any individual performing the tasks. For example, the threshold amount of time for removing the oil plug may be 3 minutes for any individual performing the task and the comparatormay determine whether the individual performed the task within the 3 minutes of allotted time. In another example, the comparatormay determine if the individual exceeds the 3-minute threshold amount of time and if so by how much. In another example, the comparatormay determine if the individual performed the task in less then the 3 minutes of allotted time.

In one example, the threshold amounts of time may be adjusted based on the user profile. For example, when the user profile indicates that the individual is relatively inexperienced for performing the task, the comparatormay use a first threshold amount of time instead of a second threshold amount of time associated with an individual that is relatively experienced or has a user profile with an experience level that exceeds a threshold level. In another embodiment, the comparatormay increase or decrease the threshold amount of time-based on characteristics of the user indicated in the user profile, such as experience level, age, gender, and so forth. For example, when the individual is an inexperienced elderly individual the comparatormay increase the threshold amount of time whereas when the individual is an experienced youthful individual the comparatormay decrease the threshold amount of time.

In another embodiment, the sensoror another sensor may take environmental measurements, such as light conditions measurements, weather conditions measurements, temperature conditions measurements, time of day indicators, noise levels measurements, and so forth. The comparatormay adjust the threshold amount of time-based on the environmental measurements. For example, when environmental measurements indicate that the light level is low, the noise level is high, or the weather is raining or snowing, the comparatormay increase the threshold amount of time for the individual to complete the task. In another example, when environmental measurements indicate that the light level is normal, the noise level is low, or the weather is sunny, the comparatormay decrease the threshold amount of time for the individual to complete the task.

In another example, the comparatormay store the information regarding whether the individual performed the tasks within the threshold amount of time and/or the difference between the time the individual performed the tasks and the threshold amount of time at the database. In one example, the user profile modulemay be coupled to the database. The user profile modulemay access the databaseto retrieve the information indicating the amount of time the individual took for a task or a set of tasks in comparison to the threshold amounts of time for the task(s), and the user profile modulemay update the user profile for that individual to reflect that information.

The optimizermay include a suggesting modulethat is coupled to the user profile module. In one example, the suggesting modulemay access the user profile information stored at the user profile module. A module may be an application or software executing on a processing device. In another example, the suggesting modulemay access the user profile information al the databasevia the user profile module. In another example, the suggesting modulemay access the databasedirectly. In one embodiment, when the user profile information indicates that the time the user is taking to complete a task exceeds the threshold amount of time, the suggesting modulemay provide a hint or recommendation on how to complete the task.

In another embodiment, when the user profile information indicates that the time the user is taking to complete a task exceeds the threshold amount of time, the suggesting modulemay provide the user with a tutorial video or image to aid the user in completing the task. In another embodiment, when the user profile information indicates that the time the user is taking to complete a task exceeds the threshold amount of time, the suggesting modulemay identify an expert or an individual experienced in performing the task in a database of the suggesting moduleor connected to the suggesting module. The suggesting modulemay then establish a communication link between the wearable deviceand a device associated with the expert or the experienced individual so that the expert or experienced individual may provide the user with aid and/or recommendation in completing the task. For example, the communications link may be a video chat, an audio chat, a phone call, a text message, an email, and so forth.

In one embodiment, the suggesting moduleor the wearable devicemay record the communication between the user and the expert in the database, another database, or a memory storage device. The stored communications may include instant messages, e-mails, phone calls, files, video chats, transcripts, and so forth between the users of the user devices.

In one example, when the amount of time the user or another user takes to perform the same task exceeds the threshold amount of time for the task, the suggesting modulemay playback, to the user or the other user, the recorded communication between the user and the expert. In another example, the suggesting modulemay update a task flow for the user to perform the task based on the communications between the user and the expert and/or other aids or recommendations received by the user. In another example, the suggesting modulemay monitor how the user adjust or changes how he/she performs the task based on the suggestions or recommendations and revised the task flow in view of the adjustments or changes.

As further discussed below, the suggesting modulemay send notifications to a System administrator or the expert that manages the threshold amounts of time. The notifications may include recommendations or suggestions to increase or decrease the threshold amounts of time for different tasks based on the amount of time taken by the user or multiple users to perform a task. For example, if the average time a set of users take to perform a task exceeds the threshold amount of time, the suggesting modulemay send a suggestion of the system administrator or the expert to increase the threshold amount of time.

In one embodiment, the suggesting modulemay be connected to a remote processing device. The suggesting modulemay send a record of the notifications, sensor data, and/or other information collected and/or recorded by the optimizerto the remote processing device. In one embodiment, the remote processing devicemay associated with a task flow owner so that the task flow owner may review the notifications, the sensor data, or the other information to improve the task flow. In another embodiment, the remote processing devicemay be coupled to the use profile moduleand the remote processing devicerelay the notifications, the sensor data, or the other information to the user profile module. The user profile modulemay then update the user profile associated with the user based on the notifications, the sensor data, or the other information. In another embodiment, the suggesting modulemay be coupled to the user profile moduleand may send the notifications, the sensor data, or the other information directly to the user profile module.

illustrates the wearable deviceindisplaying a message associated with a task of a task flow for a project, according to an embodiment. Some of the features inare the same or similar to some of the features inas noted by same reference numbers, unless expressly described otherwise. As discussed above, the optimizerinmay collect and analyze a task(s) of a task flow for an individual performing one or more tasks and provide recommendations to improve the task and/or task flow. For example, the method may provide steps for optimizing a task flow of the user in the augmented reality environment. The task flow may be a task-by-task process showing how a user may interact with a system in order to complete a project. As discussed below, a task flow may include a diagram or map of one or more sequences of tasks for the user to complete the project and decision points for the user to follow different sets of tasks for the user to follow to complete the project based on the activities and decisions of the user.

The tasks and decision points may represent activities performed by the user and/or a system in completing the project. The wearable devicemay display a notification or messageto a user to instruct the user regarding the next task in the task flow and/or display notifications to aid the user in completing the task within a threshold amount of time. For example, when the user is struggling to complete a task and has expended an amount of time that exceeds a threshold amount of time, the display may display the notificationto provide the user with a suggestion on how to complete a task for object. The notificationmay include text, graphical objects, videos, video chats, live data streams, and so forth. The notificationmay aid the user to complete the task and/or provide feedback to the user after the user completes the task.

illustrates a flowchartof a method for revising a task flow for a set of tasks, according to an embodiment. In one embodiment, the method in the flowchartmay be performed by the optimizerinexecuting on a processing device. The method may include identifying a task in a task flow to collect data associated with a user performing the task (block).

In one embodiment, to identify the task to collect data for, a processing device may identify the task based on one or more parameters or criteria. In one example, the parameters may include an experience level or skill level of the user, industry-specific restrictions, and so forth. For example, when the individual is a trainee with an insufficient amount of skills or experience to perform the task, the processing device may determine not to collect data associated with the user performing the task. In another example, when the individual is relatively experienced, the processing device may identity the task to collect data on. In another example, when the task includes information associated with privacy rules, such as patient privacy, user privacy, or trade secrets, the processing device may not collect data associated with the task or may only collect data for certain tasks.

In another embodiment, the collected data may include performance information, speed measurements, a number of movements by the individual, a size of motion or movements by the individual (measured by distance or angle measurements), environment information, and so forth. In another embodiment, the type of data collected for the task may be adjusted based on the individual performing the task. For example, the wearable deviceinmay include one or more sensors (such as an optical camera or infrared scanner) to measure the physical features of an individual, such as a height of the individual, visual capabilities of the individual, body type of the individual, gender of the individual, and so forth. When the wearable devicemeasures the physical features, the processing device may select an initial user profile based on the physical feature of the individual. In another example, based on the physical features of the user, the processing device may suggest physical aids for the user to utilize via a display, such as using stepstool or repositioning the machinery. In another embodiment, the task may be identified by a system administrator or user for optimization of the task or for further review by the system administrator. In another embodiment, the system administrator or user may identify tasks that were previously optimized or reviewed that do not need further review or optimization. In another embodiment, the selected tasks may be chosen by a system administrator or user based on a priority of the task being performed by the user.

The method may include displaying a message associated with the task (block). For example, the task flow may be a task-by-task process of activities a user is to perform to complete a project. In one example, a task flow may include a diagram or map of the one or more sequences of tasks for the user to complete the project and decision points for the user to follow different sets of tasks for the user to follow to complete the project based on the activities and decisions of the user. The tasks and decision points may represent activities performed by the user and/or a system in completing the project. In one embodiment, the wearable deviceofmay include a display with a user interface to indicate one or more of the tasks the user is to perform and/or activities associated with the one or more tasks to complete the tasks in the task flow for the project.

The method may include identify a data model based on a user profile of the user (block). The data model may be a model of an execution time of the task by an expert or an average of all task performers. In one embodiment, an initial data model may be defined by an expert in performing the task. In one example, when the initial data model is defined, the processing device may update the model based on users performing the task. The processing device may adjust the data model based on an average of time taken for users to perform the task. The data model may be iteratively updated as the number of users that perform the task increases and/or the number of times the users performs the task increases.

The method may include comparing an amount of time the user expends to perform the task to a threshold amount of time associated with the data model (block). For example, an expert may define an optimal amount of time for a task as being 5 min to execute. In one embodiment, when the user finishes the task in two minutes, the processing device may tag the data as indicating a user completed the task in less than the threshold amount of time. In another embodiment, when a threshold number of users complete the task in an amount of time less than the threshold amount of time, the processing device may decrease the threshold amount of time, such as to an average of the amount of time taken by the users. For example, when a first user completes the task in 2 minutes and a second user completes the task in 3 minutes, the processing device may set the threshold amount of time as 2.5 minutes. In another example, the processing device may select the highest amount of time taken by a user that is less than 5 minutes and may set the threshold amount of time as that amount of time.

In another embodiment, when the amount of time taken by the user is approaching the first threshold amount of time (i.e. within a second threshold amount of time) the processing device may provide a notification to the user suggesting the user contact an expert. When the first amount of time is reached, the processing device may require the user to contact the expert or provide a second recommendation to contact the expert. For example, for a first threshold amount of time of 5 minutes, when the user has taken 4 minutes to work on the task, the processing device may suggest the user contact the expert. If the user ignores the 4-minute recommendation and the user reaches the 5-minute threshold, the processing device may provide a second recommendation or may require the user contact the expert. The expert may be an individual have experienced with the task or the overall project. For example, the individual that originally generated the task or the task flow for the project may be defined as an expert for the task.

When the user completing the task within the threshold amount of time, the processing device may identify another task of the task flow to monitor. For example, the processing device may iteratively monitor different tasks in the task flow to increase the user's efficiency and decrease the amount of time the user expends when performing the different tasks in the task flow.

The method may include, when the user does not complete task within the threshold amount of time, providing a first notification to the user while the user is performing the task (block). For example, when the amount of time the user takes to complete the task exceeds the threshold amount of time, the processing device via a display may display a notification providing a suggestion for how the user may reduce the time taken to complete the task or how to otherwise improve the user's efficiency in performing the task. As discussed above, the threshold amounts of time may be based on a profile of the user and/or environmental factors in the area where the user is performing the task. For example, when the user is performing the task on an oil rig and the ambient temperature increase beyond a threshold degree level, the processing device may increase the amount of time the user may take to perform the task. In another example, the environmental factors may include light parameters, such as a hue level, an intensity level, an illumination level, and so forth.

The method may include providing a second notification to the user when the task or the project is completed (block). In one embodiment, when the processing device via a display displayed a notification to the user as the user performed the task, the processing device via the display may display a second notification when the task or the project is complete, where the second notification requests feedback from the user as to whether the first notification was helpful to the user in completing the task. For example, to determine that the first notification was helpful to the user, the processing device via the display may determine when the user proceeds to the next step of the project and display a notification requesting input as to whether the first notification was helpful. The user may then use an input device (such as a gesture detection device for an augmented reality environment) to input whether the first notification was helpful. In one embodiment, the user response to the second notification may be sent to a device associated with the system administrator or another device for the system administrator to use in adjusting the task flow for a project. In another embodiment, the processing device may automatically update the task flow for the project, the task, and/or the first notification based on the user response. The update may aid subsequent users performing the same task.