Disclosed are an image processing method and an apparatus for performing the same. An image processing method according to one embodiment comprises the steps of: acquiring information about sensing a user; and displaying a back-projection image obtained by back-projecting at least one object that is visible through a transparent flat plate onto the transparent flat plate on the basis of the information.
Legal claims defining the scope of protection, as filed with the USPTO.
. An image processing method comprising:
. The method of, wherein the providing comprises displaying the image comprising a stereoscopic image of the at least one object in the first operation mode.
. The method of, wherein the providing comprises displaying the image comprising a monoscopic image of the at least one object in the second operation mode.
. The method of, wherein the image is an image obtained by back-projecting the at least one object while maintaining an apparent size of the at least one object.
. The method of, wherein the information on sensing comprises a distance between the user's hand and the transparent flat plate and whether the user's hand holds the transparent flat plate.
. The method of, wherein the providing comprises providing the image for gradually flattening the at least one object as the distance decreases during switching from the first operation mode to the second operation mode.
. The method of, wherein the providing comprises providing the image for gradually unflattening the at least one object as the distance increases during switching from the first operation mode to the second operation mode.
. The method of, wherein the providing comprises displaying the image comprising a monoscopic image of the at least one object on the transparent flat plate when the distance is less than or equal to a threshold.
. The method of, further comprising:
. A computer program stored in a non-transitory computer-readable recording medium in combination with hardware to execute the method of.
. An image processing apparatus comprising:
. The image processing apparatus of, wherein the processor is configured to display the image comprising a stereoscopic image of the at least one object in the first operation mode.
. The image processing apparatus of, wherein the processor is configured to display the image comprising a monoscopic image of the at least one object in the second operation mode.
. The image processing apparatus of, wherein the image is an image obtained by back-projecting the at least one object while maintaining an apparent size of the at least one object.
. The image processing apparatus of, wherein the information on sensing comprises a distance between the user's hand and the transparent flat plate and whether the user's hand holds the transparent flat plate.
. The image processing apparatus of, wherein the processor is configured to display provide the image for gradually flattening the at least one object as the distance decreases during switching from the first operation mode to the second operation mode.
. The image processing apparatus of, wherein the processor is configured to provide the image for gradually unflattening the at least one object as the distance increases during switching from the first operation mode to the second operation mode.
. The image processing apparatus of, wherein the processor is configured to display the image comprising a monoscopic image of the at least one object on the transparent flat plate when the distance is less than or equal to a threshold.
. The image processing apparatus of, wherein the processor is configured to control at least one of the at least one object and a vicinity of the at least one object by using the image.
. The image processing apparatus of, wherein a sensor configured to sense the user is located in at least one of the transparent flat plate and the image processing apparatus.
Complete technical specification and implementation details from the patent document.
The following description relates to a technology for processing an image using a transparent flat plate.
We envision a future where wearable devices obtain information from the Internet and local nodes. For example, a user may wear smart glasses on their eyes and imagine a rich computing future that brings information from the Internet of things (IoT) into the space around the user as interactive virtual elements.
Whenever a particular computing form factor becomes common, such as when desktop computers appeared in the 1980s, or smart phones appeared in the 2000s, a user interface (UI) that facilitates these computing form factors has been developed.
When a developer designs a new UI, the developer may design by optionally borrowing concepts from previous paradigms such as windows that work adaptively for their intended purpose, icons, menus, the pointer (WIMP) and touch user interface (TUI).
A small head mounted display (HMD) device worn on the user's eyes will be able to provide the user with augmented reality (AR) more realistically by placing a virtual object in a physical space. This virtual object may be used for IoT, robot engineering, three-dimensional (3D) sketching and the like, since the virtual object can be simple annotations that allow the user to gain more information about physical spaces and the object, or quickly view more complex spatial UI elements that the user need to interact for rich space work.
The virtual object may appear in various environments within the immersive AR situation, and the user may easily control nearby virtual object through gestures. However, the method of interacting with a remote virtual object is relatively difficult and requires complex technology. A method such as ray casting is inevitably less direct than a method of simply touching by hand, and inevitably lowers intuition.
An aspect provides an image processing technology capable of displaying an object on a transparent flat plate. Another aspect provides a technique for performing an interaction with an object using the image processing technology.
According to an aspect, there is provided an image processing method including obtaining information on sensing a user and displaying an image obtained by back-projecting at least one object viewed through a transparent flat plate onto the transparent flat plate based on the information on the transparent flat plate.
The displaying may include displaying the image including a stereoscopic image of the at least one object.
The displaying may include displaying the image including a monoscopic image of the at least one object.
The image may be an image obtained by back-projecting while maintaining an apparent size of the at least one object.
The information on sensing may include a distance between the user's hand and the transparent flat plate.
The displaying may include displaying the image for gradually flattening the at least one object as the distance decreases.
The displaying may include displaying the image for gradually unflattening the at least one object as the distance increases.
The displaying may include displaying the image including the monoscopic image of the at least one object on the transparent flat plate when the distance is less than or equal to a threshold.
The image processing method may further include controlling at least one of the at least one object and a vicinity of the at least one object using the image.
According to another aspect, there is provided an image processing apparatus including a memory including instructions and a processor configured to execute the instructions, wherein, when the instructions are executed by the processor, the processor is configured to display an image obtained by back-projecting at least one object viewed through a transparent flat plate onto the transparent flat plate based on information on sensing a user on the transparent flat plate.
The processor may be configured to display the image including a stereoscopic image of the at least one object.
The processor may be configured to display the image including a monoscopic image of the at least one object.
The image may be an image obtained by back-projecting while maintaining an apparent size of the at least one object.
The sensed information may include a distance between the user's hand and the transparent flat plate.
The processor may be configured to display the image for gradually flattening the at least one object as the distance decreases.
The processor may be configured to display the image for gradually unflattening the at least one object as the distance increases.
The processor may be configured to display the image including the monoscopic image of the at least one object on the transparent flat plate when the distance is less than or equal to the threshold.
The processor may be configured to control at least one of the at least one object and the vicinity of the at least one object by using the image.
A sensor for sensing the information of the user may be located in at least one of the transparent flat plate and the image processing apparatus.
Hereinafter, example embodiments will be described in detail with reference to the accompanying drawings. However, since various modifications may be made to the example embodiments, the scope of the patent application is not limited or restricted by these example embodiments. It should be understood that all changes, equivalents and replacements for the example embodiments are included in the scope of the application.
The terminology used in the example embodiments is for the purpose of description only, and should not be construed as limiting. The singular expression includes the plural expression unless the context clearly indicates otherwise. In the present specification, it will be understood that the terms “comprises” and/or “comprising,” specify the presence of features, integers, steps, operations, elements, components or a combination thereof which are stated in the specification, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
Although terms of “first,” “second,” and the like are used to explain various components, the components are not limited to such terms. Terms are used only to distinguish one component from another component. For example, a first component may be referred to as a second component, or similarly, the second component may be referred to as the first component within the scope according to the concept of an example embodiment.
Unless otherwise defined herein, all terms used herein including technical or scientific terms have the same meanings as those generally understood by one of ordinary skill in the art. Terms defined in dictionaries generally used should be construed to have meanings matching contextual meanings in the related art and are not to be construed as an ideal or excessively formal meaning unless otherwise defined in the present application.
In addition, when describing with reference to the accompanying drawings, the same components are denoted by the same reference numerals regardless of reference numerals and a repeated description related thereto will be omitted. In describing example embodiment, when it is determined that a specific description of a related known technology may unnecessarily obscure the gist of the example embodiments, a detailed description thereof will be omitted.
Hereinafter, example embodiments will be described in detail with reference to the accompanying drawings. However, the scope of the patent application is not limited or restricted by these example embodiments.
is a schematic diagram illustrating an image processing system for processing an image using a transparent flat plate according to an example embodiment, andis a schematic block diagram illustrating an image processing apparatus shown in.
Referring toand, an image processing systemincludes an image processing apparatus, a transparent flat plate, and a sensor. The image processing apparatusmay implement a world using virtual and/or reality. For example, the image processing apparatusmay implement AR, virtual reality (VR), mixed reality (MR), or extended reality (XR). A user may experience the AR, the VR, the MR, or the XR implemented through the image processing apparatus.
The image processing apparatusmay be implemented as a portable electronic device. For example, the image processing apparatusmay be implemented as a smart device. The smart device may include a head mounted display (HMD), smart glasses, a smart watch, and a smart band. In other words, the image processing apparatusmay be a wearable device that the user can wear or is suitable for wearing.
The image processing apparatusmay process at least one objectin the AR, the VR, the MR, or the XR using the transparent flat plateand the sensor.
The objectmay be located away from the transparent flat plate. For example, the objectmay include one or more objects that a user views through transparent flat plate. The objectmay include one or more objects that may be viewed by the user through the transparent flat plateeven if the user does not view the objectthrough the transparent flat plate. The objectmay be a real object, a virtual object, or a virtual image.
The user may view the at least one objectin a flat or three-dimensional manner through the transparent flat plateby the image processing apparatus. Also, the user may interact with at least one of the at least one objectand the vicinity of the objectusing the transparent flat plate. The vicinity of the objectmay mean a vicinity area surrounding the objectviewed through the transparent flat plate. The size of the vicinity area may vary depending on the transparent flat plate.
The image processing apparatusincludes a memoryand a processor. The processormay control the overall operation of the image processing apparatus. In the AR, the VR, the MR, or the XR, the processormay process an image of at least one objectviewed through the transparent flat plate. The user may use the image processed by the image processing apparatusto interact with the objectand the vicinity of the objectin the AR, the VR, the MR, or the XR.
The processormay display a back-projection image obtained by back-projecting at least one objectviewed through the transparent flat plateonto the transparent flat platebased on information obtained by the sensorsensing the user on the transparent flat plate. The back-projection image may be an image obtained by back-projecting only the objector an image obtained by back-projecting the objectand the vicinity of the objecttogether. For example, the back-projection image may be a back-projection image including a stereoscopic image of the objector a back-projection image including a monoscopic image of the object.
The processormay operate (or call) the operation mode of the image processing apparatusas a first operation mode or a second operation mode based on information that the sensorsenses the user. The first operation mode may mean a stereoscopic viewing mode, and the second operation mode may mean a monoscopic touching mode.
In the stereoscopic viewing mode, the user may view the objectstereoscopically through the transparent flat plate. In the monoscopic touching mode, the user may view the back-projection image including the monoscopic image obtained by back-projecting the objectonto the transparent flat plate. In the monoscopic touching mode, the user may control the objectusing the monoscopic image displayed on the transparent flat plate.
The processormay provide a method of interacting with the objectin the AR, the VR, the MR, or the XR in a method such as by touching the transparent flat plateby the user and the like by displaying the back-projection image obtained by back-projecting the objectthat is far away from the transparent flat plate(e.g., distant) on the transparent flat plate.
The processormay process data stored in the memory. The processormay execute computer-readable codes (e.g., software) stored in the memoryand instructions induced by the processor.
The processormay be a data processing device implemented as hardware having a circuit having a physical structure for executing desired operations. For example, desired operations may include code or instructions included in a program.
For example, the data processing device implemented as hardware includes a microprocessor, a central processing unit, a processor core, a multi-core processor, and a multiprocessor, an application-specific integrated circuit (ASIC), and a field programmable gate array (FPGA).
The transparent flat platemay have a transparent flat plate shape. The transparent flat platemay communicate with the image processing apparatus.
The sensormay sense the user. For example, the sensormay sense the distance between the user's hand and the transparent flat plate. The sensormay transmit information that the sensorsenses the user, for example, information on a distance between the user's hand and the transparent flat plateto the processorof the image processing apparatus.
The sensormay be implemented in the image processing apparatusor the transparent flat plate. In addition, the sensormay be separately implemented outside the image processing apparatusor the transparent flat plate.
As described above, so that the user may better interact with the objectand the vicinity of the object, the image processing apparatusmay provide a method that the user may directly touch at least one of the distant objectand the vicinity area of the object using the transparent flat plate.
Unknown
March 10, 2026
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.