The present technology relates to an imaging device and an electronic device that enable construction of an imaging device that outputs information required by a user with a small size. A single-chip imaging device includes: an imaging unit in which a plurality of pixels is arranged two-dimensionally and that captures an image; a signal processing unit that performs signal processing using a captured image output from the imaging unit; an output I/F that outputs a signal processing result of the signal processing and the captured image to an outside; and an output control unit that performs output control of selectively outputting the signal processing result of the signal processing and the captured image from the output I/F to the outside. The present technology can be applied to, for example, an imaging device that captures an image.
Legal claims defining the scope of protection, as filed with the USPTO.
1. An imaging device, comprising: an imaging unit in which a plurality of pixels is arranged two-dimensionally and that captures an image; a recognition processor that performs at least part of a recognition process for a first captured image output from the imaging unit; an output I/F that outputs both a recognition result by the recognition processor or intermediate data acquired midway of the recognition process and a second captured image to an outside of the imaging device; and an output control processor that causes at least one of the recognition result or the intermediate data and the captured image to be selectively outputted from the output I/F to the outside of the image device, wherein the imaging unit and the recognition processor are arranged on a single chip.
2. The imaging device according to claim 1 , wherein the imaging device has a stacked structure in which a plurality of dies is stacked.
3. The imaging device according to claim 1 , further comprising an image compressing unit that compresses the captured image and generates a compressed image having a smaller amount of data than that of the captured image.
4. The imaging device according to claim 1 , further comprising: an imaging control processor that includes a register storing imaging information associated with capturing of the captured image and controls capturing of the captured image according to the imaging information; and an imaging information calculating processor that calculates the imaging information using the recognition result, wherein the imaging control processor and the imaging information calculating processor are connected to each other via a predetermined connection line, and the imaging information calculating processor feeds back the imaging information to the register of the imaging control processor via the predetermined connection line.
5. The imaging device according to claim 4 , further comprising a first communication I/F that exchanges information to be read from and written to the register with the outside of the imaging device.
6. The imaging device according to claim 1 , wherein the recognition processor executes a program, and the imaging device further comprises a second communication I/F that downloads the program which is executed by the recognition processor from the outside of the imaging device.
7. The imaging device according to claim 1 , further comprising an input I/F that receives an external sensor output from an external sensor, wherein the recognition processor performs recognition processing using the captured image and the external sensor output.
8. The imaging device according to claim 7 , wherein the external sensor output is an output of a range sensor that senses information associated with a distance or an output of an image sensor that senses light and outputs an image corresponding to the light.
9. The imaging device according to claim 8 , wherein the recognition processor performs a fusion process of calculating a distance using the captured image and the output of the range sensor or a self-localization process using an image as the output of the image sensor and the captured image as the recognition processing.
10. An electronic device comprising: an optical system that collects light; and a single-chip imaging device that receives light and outputs an image corresponding to a received light intensity of the light, wherein the imaging device includes: an imaging unit in which a plurality of pixels is arranged two-dimensionally and that captures an image; a recognition processors that performs at least part of a recognition process for a first captured image output from the imaging unit; an output I/F that outputs both a recognition result by the recognition processor or intermediate data acquired midway of the recognition process and a second captured image to an outside of the imaging device; and an output control processor that cause at least one of the recognition result or the intermediate data and the captured image to be selectively outputted from the output I/F to the outside of the image device, wherein the imaging unit and the recognition processor are arranged on a single chip.
11. The imaging device according to claim 1 , wherein the single chip includes a first substrate and a second substrate joint together, wherein the first substrate includes the imaging unit, and wherein the second substrate includes the recognition processor.
12. The imaging device according to claim 1 , wherein the captured image is an image generated by compressing the output of the imaging unit, and wherein the recognition processor performs at least part of the recognition process for the captured image generated by the compression.
13. The imaging device according to claim 1 , wherein the captured image is an image generated by performing a gradation process for the output of the imaging unit.
14. The imaging device according to claim 1 , further comprising: an imaging processor that processes the captured image outputted from the imaging unit; and a control processor that controls processing by the imaging processor or an imaging operation to be executed by the imaging unit, according to the result of the recognition process.
15. The imaging device according to claim 14 , wherein the control processor causes the imaging processor to execute a process of cutting out a part associated with an area specified by the recognition process from the captured image.
16. The imaging device according to claim 14 , wherein the control processor controls the imaging operation by the imaging unit such that a part associated with an area specified by the recognition process is read out.
17. The imaging device according to claim 14 , wherein the control processor controls at least one of an exposure time, a focus, a frame rate, and an imaging mode in the imaging operation by the imaging unit.
18. The imaging device according to claim 1 , wherein the recognition processors performs the recognition process using a technique of deep learning.
19. The imaging device according to claim 18 , wherein the technique of deep learning is a convolutional neural network.
20. The imaging device according to claim 1 , wherein the recognition processor executes the recognition process after acquisition of the captured image based on the output of the imaging unit being completed in each frame.
21. The imaging device according to claim 1 , wherein the recognition processor starts the recognition process before acquisition of the captured image based on the output of the imaging unit being completed in each frame.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
September 1, 2017
October 6, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.