For causing a near-infrared light image obtained through imaging in a second imaging mode to be easily viewed while suppressing an increase in size and number of components, in a case where an imaging unit is operating in the second imaging mode, a change region where there is a change is detected through comparison with the second image data that the imaging unit has acquired before the second image data, from within the second image data acquired by the imaging unit. Combination image data obtained by combining the change region image data corresponding to the change region within the second image data with background image data of a color image generated on the basis of first image data acquired in a first imaging mode immediately before the second imaging mode by the imaging unit and corresponding to a background region different from the change region within the second image data is generated on the basis of the detection result of the change region.
Legal claims defining the scope of protection, as filed with the USPTO.
1. An imaging device installed at a fixed position, comprising: an optical system in which an infrared ray cut filter is insertable into an imaging optical path or retractable from the imaging optical path; an imaging unit including signal processing circuitry and having a first imaging mode in which the imaging unit images a subject through the optical system in which the infrared ray cut filter has been inserted into the imaging optical path to acquire first image data indicating a visible light image that is a visible light image of the subject, the visible light image being a color image, and a second imaging mode in which the imaging unit images the subject through the optical system in which the infrared ray cut filter has been retracted from the imaging optical path to acquire second image data including a near-infrared light image that is a near-infrared light image of the subject, the near-infrared light image being a monochrome image; and processing circuitry configured to: alternately switch between the first imaging mode and the second imaging mode; detect a change region having a change within the second image data acquired by the imaging unit when compared with the second image data that the imaging unit has acquired prior to the second image data in a case where the imaging unit is operating in the second imaging mode; generate combination image data obtained by combining the change region image data corresponding to the change region within the second image data with background image data of the color image generated on the basis of the first image data acquired in the first imaging mode immediately before the second imaging mode by the imaging unit and corresponding to a background region different from the change region within the second image data on the basis of the detection result; and perform a point image restoration process based on first optical characteristics of the optical system with respect to near-infrared light on at least the change region image data, wherein the processing circuitry is further configured to generate the background image data on the basis of color information extracted from a region corresponding to the background region within the first image data, and the luminance information in the background region of the second image data, wherein the first optical characteristics are a first point spread function of the optical system with respect to the near-infrared light; and wherein the point image restoration process that the processing circuitry performs on the change region image data includes restoring an amplitude component of the change region image data, and a phase component due to a shape of the non-point-symmetric, first point spread function.
2. The imaging device according to claim 1 , wherein the processing circuitry is further configured to generate the background image data using the image data itself included in the region corresponding to the background region within the first image data.
3. The imaging device according to claim 1 , wherein the processing circuitry is further configured to detect a non-change region in the plurality of pieces of first image data that the imaging unit has acquired at different timings in the first imaging mode, and generate the background image data from the image data corresponding to the non-change region within the first image data on the basis of a detection result.
4. The imaging device according to claim 2 , wherein the processing circuitry is further configured to detect a non-change region in the plurality of pieces of first image data that the imaging unit has acquired at different timings in the first imaging mode, and generate the background image data from the image data corresponding to the non-change region within the first image data on the basis of a detection result.
5. The imaging device according to claim 1 , wherein the processing circuitry is further configured to perform a sharpening process on at least the change region image data.
6. The imaging device according to claim 1 , wherein the processing circuitry is further configured to perform a lateral chromatic aberration correction process on at least the change region image data.
7. The imaging device according to claim 1 , wherein the processing circuitry is further configured to perform a point image restoration process based on second optical characteristics of the optical system with respect to visible light on the first image data.
8. The imaging device according to claim 7 , wherein: the second optical characteristics are a second point spread function of each color of the optical system with respect to red, green, and blue visible light; and the point image restoration process that the processing circuitry performs on the first image data includes a process of restoring a phase component of the first image data using a restoration filter based on the second point spread function for each color.
9. An image processing method for an imaging device comprising an optical system in which an infrared ray cut filter is insertable into an imaging optical path or retractable from the imaging optical path, an imaging unit including signal processing circuitry and having a first imaging mode in which the imaging unit images a subject through the optical system in which the infrared ray cut filter has been inserted into the imaging optical path to acquire first image data indicating a visible light image that is a visible light image of the subject, the visible light image being a color image, and a second imaging mode in which the imaging unit images the subject through the optical system in which the infrared ray cut filter has been retracted from the imaging optical path to acquire second image data including a near-infrared light image that is a near-infrared light image of the subject, the near-infrared light image being a monochrome image, and processing circuitry that alternately switches between the first imaging mode and the second imaging mode, the imaging device being installed at a fixed position, the method comprising: a change region detection step of detecting a change region having a change within the second image data acquired by the imaging unit when compared with the second image data that the imaging unit has acquired prior to the second image data in a case where the imaging unit is operating in the second imaging mode; a combination image data generation step of generating combination image data obtained by combining the change region image data corresponding to the change region within the second image data with background image data of the color image generated on the basis of the first image data acquired in the first imaging mode immediately before the second imaging mode by the imaging unit and corresponding to a background region different from the change region within the second image data on the basis of the detection result in the change region detection step; and a first point image restoration processing step of performing a point image restoration process based on first optical characteristics of the optical system with respect to near-infrared light on at least the change region image data, wherein the first optical characteristics are a first point spread function of the optical system with respect to the near-infrared light; and wherein the point image restoration process that the first point image restoration processing step performs on the change region image data includes restoring an amplitude component of the change region image data, and a phase component due to a shape of the non-point-symmetric, first point spread function.
10. A non-transitory computer-readable tangible medium containing a program, executable by processing circuitry, for causing a computer of an imaging device comprising an optical system in which an infrared ray cut filter is insertable into an imaging optical path or retractable from the imaging optical path, an imaging unit including signal processing circuitry and having a first imaging mode in which the imaging unit images a subject through the optical system in which the infrared ray cut filter has been inserted into the imaging optical path to acquire first image data indicating a visible light image that is a visible light image of the subject, the visible light image being a color image, and a second imaging mode in which the imaging unit images the subject through the optical system in which the infrared ray cut filter has been retracted from the imaging optical path to acquire second image data including a near-infrared light image that is a near-infrared light image of the subject, the near-infrared light image being a monochrome image, and the processing circuitry that alternately switches between the first imaging mode and the second imaging mode, the imaging device being installed at a fixed position, to perform a method comprising: detecting a change region having a change within the second image data acquired by the imaging unit when compared with the second image data that the imaging unit has acquired prior to the second image data in a case where the imaging unit is operating in the second imaging mode; and generating combination image data obtained by combining the change region image data corresponding to the change region within the second image data with background image data of the color image generated on the basis of the first image data acquired in the first imaging mode immediately before the second imaging mode by the imaging unit and corresponding to a background region different from the change region within the second image data on the basis of the detection result; and performing a point image restoration process based on first optical characteristics of the optical system with respect to near-infrared light on at least the change region image data, wherein the first optical characteristics are a first point spread function of the optical system with respect to the near-infrared light; and wherein the point image restoration process that is performed on the change region image data includes restoring an amplitude component of the change region image data, and a phase component due to a shape of the non-point-symmetric, first point spread function.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
October 20, 2017
April 28, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.