The present invention is to provide a terminal for performing a secretarial function for assisting a person's memory at home or in a company, and a system comprising the same. The terminal comprises: a camera module configured to capture an image of a predetermined space when a person's motion in the predetermined space is detected; and a memory for storing the image captured by the camera module, wherein when a user request is received, the position of an object corresponding to the user request is detected using the image stored in the memory, and guide information can be outputted in at least one manner of a visual manner, an auditory manner and a tactile manner so that the detected position is guided.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A terminal comprising: a display unit; a camera module configured to capture an image of a predetermined space when a motion of a person is sensed in the predetermined space; a memory configured to store the image taken from the camera module; and a controller configured to: in response to receiving a user request for determining a position of an object, detect the position of the object using the image stored in the memory, and output guide information for guiding the user to the position of the object in at least one of visual, auditory and tactile manners, wherein the user request is performed by the user applying a touch input to a virtual keyboard displayed on the display unit or by a natural language voice input by the user, and wherein the guide information includes: a plane view of the predetermined space with a graphic object at a point corresponding to the determined position of the object, wherein the graphic object is an image corresponding to the object, or an image of the predetermined space captured by the camera module and highlighting the object in the displayed image.
2. The terminal according to claim 1 , wherein the controller controls the memory to store the captured image only if the image fulfills a preset condition.
3. The terminal according to claim 2 , wherein the camera module captures a first image of the predetermined space only if the motion of the person is sensed in the predetermined space and captures a second image of the predetermined space only if the motion of the person disappears, and the controller stores the first and second images in the memory if the first and second images fulfills the preset condition.
4. The terminal according to claim 3 , wherein the controller detects the object of which position is changed, in the predetermined space by comparing the first and second images with each other, and stores the first and second images in the memory in accordance with the detected result.
5. The terminal according to claim 1 , wherein in response to receiving the user request, the controller sets the point where the terminal is located, to a departure place and sets the position of the object corresponding to the user request to an arrival place, and the guide information is path guide information for guiding a path from the departure place to the arrival place.
6. The terminal according to claim 1 , wherein the user request is performed by the natural language voice input by the user together with a gesture corresponding to at least one of a size and shape of the object, wherein the gesture is captured by the camera module, and wherein the controller is configured to calculate a size or shape of the object based on the gesture.
7. The terminal according to claim 1 , wherein the determining the position of the object includes determining a plurality of possible candidate positions of the object, and wherein the guide information includes displaying the plurality of possible candidate positions of the object in the form of separate graphic objects in the plane view of the predetermined space.
8. A control method of a terminal including a display unit and a camera module, the method comprising: setting an object for tracing a position based on of a user input; capturing an image, using the camera module, of a predetermined space when a motion of a person is sensed from the predetermined space; in response to receiving a user request from a user for determining a position of the object, detecting the position of the object using the image; updating position information of the object when the position of the object is changed; and outputting guide information on the display unit for guiding the user to the position of the object in at least one of visual, auditory and tactile manners, wherein the user request is performed by the user applying a touch input to a virtual keyboard displayed on the display unit or by a natural language voice input by the user to the terminal, and wherein the guide information includes: a plane view of the predetermined space with a graphic object at a point corresponding to the determined position of the object, wherein the graphic object is an image corresponding to the object, or an image of the predetermined space captured by the camera module and highlighting the object in the displayed image.
9. The control method according to claim 8 , wherein the user request is performed by the natural language voice input by the user together with a gesture corresponding to at least one of a size and shape of the object, wherein the gesture is captured by the camera module, and wherein the controller is configured to calculate a size or shape of the object based on the gesture.
10. The control method according to claim 8 , wherein the determining the position of the object includes determining a plurality of possible candidate positions of the object, and wherein the guide information includes displaying the plurality of possible candidate positions of the object in the form of separate graphic objects in the plane view of the predetermined space.
11. A system comprising: a terminal, the terminal comprising: a display unit; and a camera module configured to capture an image of a predetermined space when a motion of a person is sensed from the predetermined space; and a server configured to generate position information on one or more objects arranged in the predetermined space by using the captured image and storing the generated position information, the terminal being configured to: send a request to the server for the position of the one or more objects in response to a user request, and output guide information for guiding the user to the position of the one or more objects in at least one of visual, auditory and tactile manners, wherein the user request is performed by the user applying a touch input to a virtual keyboard displayed on the display unit or by a natural language voice input by the user to the terminal, and wherein the guide information includes: a plane view of the predetermined space with a graphic object at a point corresponding to the position of the one or more objects, wherein each graphic object is an image corresponding to a respective one of the one or more objects, or an image of the predetermined space captured by the camera module and highlighting the one or more objects in the displayed image.
12. The system according to claim 11 , wherein the camera module captures a first image of the predetermined space only if the motion of the person is sensed from the predetermined space and captures a second image of the predetermined space only if the motion of the person disappears, and the server generates the position information on the one or more objects by using the first and second images which are taken at their respective timing points.
13. The system according to claim 12 , wherein the server detects an object among the one or more objects of which position is changed, from the predetermined space by comparing the first and second images with each other, and generates the position information for only the detected object.
14. The system according to claim 12 , wherein the one or more objects are objects preset by a user input, and the server generates the position information for only the preset object.
15. The system according to claim 11 , wherein the terminal sets the point where the terminal is located, to a departure place and sets the position of the one or more objects corresponding to the user request to an arrival place, and the guide information further comprises path guide information for guiding a path from the departure place to the arrival place.
16. The system according to claim 11 , wherein the user request is performed by the natural language voice input by the user together with a gesture corresponding to at least one of a size and shape of the one or more objects, wherein the gesture is captured by the camera module, and wherein the terminal is configured to calculate a size or shape of the one or more objects based on the gesture.
17. The system according to claim 11 , wherein the determining the position of the one or more objects includes determining a plurality of possible candidate positions of each object, and wherein the guide information includes displaying the plurality of possible candidate positions of each object in the form of separate graphic objects in the plane view of the predetermined space.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
August 18, 2015
April 14, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.