Image Processing Apparatus, Image Processing Method, and Program

The present application is a Continuation of PCT International Application No. PCT/JP2024/030215 filed on Aug. 26, 2024 claiming priority under 35 U.S.C § 119(a) to Japanese Patent Application No. 2023-148541 filed on Sep. 13, 2023. Each of the above applications is hereby expressly incorporated by reference, in its entirety, into the present application.

The present invention relates to an image processing apparatus, an image processing method, and a program, and particularly relates to a technology of visualizing an imaging status of an area.

2. Description of the Related Art

In a case of a disaster, a local government flies a plurality of drones at the same time to capture an aerial image of a local government area, and ascertains a disaster status of a house from the aerial image.

WO2023-047799A discloses a technology of determining a damage level of a house by aligning a captured image with map data and cutting out a region of each house shown in the captured image by collating the region with the map data.

In the aerial image, in a case where there is an uncaptured region in a target area, the disaster status of the region cannot be ascertained. Therefore, it is preferable that the number of uncaptured regions is small. In addition, in a case where a ground resolution of the aerial image is low, it is difficult to ascertain the disaster status of the target area. Therefore, it is preferable that the ground resolution is high. Further, in order to ascertain the disaster status of the house, it is preferable that the aerial image is captured at an appropriate camera angle.

However, it is difficult for a user such as an imaging person to ascertain the imaging status of the aerial image.

The present invention has been made in view of such circumstances, and an object thereof is to provide an image processing apparatus, an image processing method, and a program with which a user such as an imaging person can easily ascertain an imaging status of a set area.

In order to achieve the above object, an image processing apparatus according to a first aspect of the present disclosure is an image processing apparatus comprising: one or more processors; and one or more memories that store a program to be executed by the one or more processors, in which the processor is configured to execute an instruction of the program to: acquire a set area; acquire a group of images in which the set area is captured by using a camera; acquire a map including the set area; calculate meta-information of each image of the group of images; calculate an imaging status of the set area based on the meta-information of each of the images; and output visualization information in which a result of the calculation is superimposed on the set area of the map.

According to the first aspect, the user such as the imaging person can easily ascertain the imaging status of the set area.

According to a second aspect of the present disclosure, in the image processing apparatus according to the first aspect, it is preferable that the processor is configured to: acquire a plurality of set areas; acquire a group of images in which each set area of the plurality of set areas is captured; acquire a map including the plurality of set areas; calculate an imaging status of each of the set areas based on the meta-information of each of the images; and superimpose the result of the calculation on each of the set areas of the map.

According to a third aspect of the present disclosure, in the image processing apparatus according to the first or second aspect, it is preferable that the processor is configured to superimpose at least one of a color, a shade, a hatching, or a contour line corresponding to the result of the calculation on the set area of the map.

According to a fourth aspect of the present disclosure, in the image processing apparatus according to any one of the first to third aspects, it is preferable that the meta-information includes latitude information and longitude information of four corners of each of the images, and the imaging status is the number of the images.

According to a fifth aspect of the present disclosure, in the image processing apparatus according to any one of the first to fourth aspects, it is preferable that the meta-information includes position information and orientation information of the camera during imaging, and the imaging status is an angle distribution of the camera.

According to a sixth aspect of the present disclosure, in the image processing apparatus according to any one of the first to fifth aspects, it is preferable that the map includes information on a house, the meta-information includes position information and orientation information of the camera during imaging, the processor is configured to calculate the imaging status of the set area based on a focal length and a sensor size of the camera, and the imaging status is a distribution of a ground resolution for each house included in the image.

According to a seventh aspect of the present disclosure, in the image processing apparatus according to any one of the first to sixth aspects, it is preferable that the meta-information includes latitude information and longitude information of four corners of each of the images, and the imaging status is an area coverage rate that is a ratio of a captured area to an area of the set area.

According to an eighth aspect of the present disclosure, in the image processing apparatus according to any one of the first to seventh aspects, it is preferable that the map includes information on a house, the meta-information includes latitude information and longitude information of four corners of each of the images, and the imaging status is a house coverage rate that is a ratio of the number of captured houses to the number of houses in the set area.

According to a ninth aspect of the present disclosure, in the image processing apparatus according to any one of the first to eighth aspects, it is preferable that text information of the result of the calculation for the set area of the map is output.

According to a tenth aspect of the present disclosure, in the image processing apparatus according to any one of the first to ninth aspects, it is preferable that the group of images is a group of images captured by the camera mounted on a flying object.

In order to achieve the above object, an image processing method according to an eleventh aspect of the present disclosure is an image processing method executed by one or more processors, the method comprising: via the processor, acquiring a set area; acquiring a group of images in which the set area is captured by using a camera; acquiring a map including the set area; calculating meta-information of each image of the group of images; calculating an imaging status of the set area based on the meta-information of each of the images; and outputting visualization information in which a result of the calculation is superimposed on the set area of the map.

According to the eleventh aspect, the user such as the imaging person can easily ascertain the imaging status of the set area.

In order to achieve the above object, a program according to a twelfth aspect of the present disclosure is a program for causing a computer to execute the image processing method according to the eleventh aspect. The present disclosure also includes a non-transitory computer-readable recording medium, such as a compact disk-read only memory (CD-ROM), on which the program according to the twelfth aspect is recorded.

According to the present invention, the user such as the imaging person can easily ascertain the imaging status of the set area.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the present specification, identical reference numerals are denoted by identical components, and duplicate descriptions will be omitted as appropriate.

1 FIG. 10 10 12 14 12 16 20 12 16 12 is a schematic diagram showing a configuration example of a captured image processing system. The captured image processing systemincludes a dronefor aerial imaging, a cameramounted on the drone, a remote controller, and an image processing apparatus. The droneis an unmanned aerial vehicle that is remotely operated by an imaging person using the remote controllerto perform aerial imaging, and is an example of a flying object. The dronemay have an auto-pilot function of flying in accordance with a program.

14 12 13 14 The camerais mounted on the dronevia a gimbal head. The cameraincludes an optical system (not shown), an image sensor, and a signal processing circuit. The optical system includes one or more lenses, such as a focus lens. The image sensor may be, for example, a charge coupled device (CCD) image sensor or a complementary metal-oxide semiconductor (CMOS) image sensor.

14 14 14 12 12 14 16 20 The cameragenerates digital image data of a captured target by processing a signal obtained from the image sensor by the signal processing circuit. The digital image data generated by the cameracan be an “image”. The image captured by using the camerais stored in a storage device such as an internal storage built in the droneand/or a memory card that is attachably and detachably mounted on the drone. In addition, the image captured by using the cameramay be transmitted to the remote controllerby using wireless communication, or may be transmitted to the image processing apparatus.

16 14 12 12 14 The remote controlleris a transmitter that controls operations of the cameraand the dronevia wireless communication. A form of the wireless communication may be a form of a wireless local area network (LAN). The form of the wireless communication may be a communication form using radio waves in a 2.4 GHz band or a 5.7 GHz band. The form of the wireless communication may be a form using a mobile communication network. A communication form of a control signal for controlling the droneand a communication form for transmitting the image captured by using the cameraor the like may be different from each other or may be common to each other.

16 16 12 13 14 The remote controllercomprises a displayA, left and right sticks (not shown) for operating a flight operation of the drone, a lever (not shown) for operating the gimbal head, an imaging button (not shown) for instructing the imaging by the camera, and an imaging mode button (not shown) for switching between video imaging and still image imaging.

16 12 13 14 The displayA may be a touch panel display. Various operations on the drone, the gimbal head, and the cameramay be performed by a touch operation on the touch panel display. The touch operation includes a tap operation, a double tap operation, a flick operation, a swipe operation, a drag operation, a pinch-in operation, and a pinch-out operation.

14 16 16 16 12 16 A live video captured by using the camerais displayed on the displayA of the remote controlleror the like. In addition, the remote controllerascertains a situation of an aircraft, such as a flight position and a flight speed, in real time based on data of various sensors provided in the drone. Flight information indicating the situation of the aircraft may be displayed on the displayA.

10 14 20 The captured image processing systemcaptures a plurality of still images (captured images) from the air by using the camera, and processes the captured images in the image processing apparatus.

20 20 The image processing apparatusis for automatically setting a ground control point (GCP) required for generating a high-accuracy three-dimensional model and an ortho image from a group of aerial images in which images adjacent to each other have an overlap region. The GCP is a point on the ground of which a latitude, a longitude, and an elevation are known, and is a point of a characteristic terrain that is visible on the image. The image processing apparatusmay generate a high-accuracy three-dimensional model and an ortho image from the group of aerial images.

20 20 The image processing apparatusis configured by a computer. The computer applied to the image processing apparatusmay be a server, a personal computer, or a workstation.

20 16 22 22 20 12 14 20 22 20 14 14 The image processing apparatusperforms data communication with the remote controllervia a network. The networkmay be a local area network or a wide area network. The image processing apparatusacquires various types of information from the droneand the camera. The image processing apparatusacquires map data of an imaging target range from a geographical information system (not shown) via the network. The image processing apparatusmay acquire the map data in advance before the imaging by the camera, or may acquire the map data after the imaging by the camera.

2 FIG. 12 14 12 30 32 34 36 38 40 42 44 is a block diagram schematically showing an example of an electrical configuration of the droneon which the camerais mounted. The droneincludes a global positioning system (GPS) receiver, an atmospheric pressure sensor, an azimuth sensor, a gyro sensor, a motor, a processor, a storage device, a communication interface, a battery (not shown), and a charging terminal of the battery.

30 12 32 12 12 12 32 12 14 The GPS receiveracquires a latitude and a longitude of a position of the drone. The atmospheric pressure sensordetects an atmospheric pressure of the position of the drone. The droneacquires an altitude of the position of the dronebased on the atmospheric pressure detected by using the atmospheric pressure sensor. The term “acquisition” includes the concept of generating information through data processing, such as calculation. The latitude, the longitude, and the altitude of the droneconstitute position information including the latitude information, the longitude information, and the altitude information of the camera.

34 12 14 34 The azimuth sensormay be, for example, a geomagnetic sensor. The dronedetects an azimuth angle in which a lens of the camerafaces by the azimuth sensor.

36 12 14 36 30 32 34 36 14 The gyro sensordetects a roll angle indicating a rotation angle with respect to a roll axis, a pitch angle indicating a rotation angle with respect to a pitch axis, and a yaw angle indicating a rotation angle with respect to a yaw axis. The droneacquires orientation information of the camerabased on the rotation angle acquired by using the gyro sensor. It should be noted that a part or all of sensors, such as the GPS receiver, the atmospheric pressure sensor, the azimuth sensor, and the gyro sensor, may be disposed on the cameraside.

38 12 38 The motoris a power source that rotates a rotary wing (rotor) (not shown). The droneincludes a plurality of motorsthat drive a plurality of rotary wings.

42 40 12 The storage devicemay be a memory, an internal storage, an external storage device, or a combination thereof. The processoracts as a flight controller, and performs various operations necessary for flight control of the dronebased on sensor data obtained from various sensors.

44 16 44 The communication interfaceis a communication unit that performs the wireless communication with the remote controllerand the like. The communication interfacemay comprise a communication terminal corresponding to wired communication.

3 FIG. 20 20 202 204 206 208 210 is a block diagram showing a hardware configuration example of the image processing apparatus. The image processing apparatusincludes one or more processors, one or more computer-readable media, a communication interface, an input/output interface, and a bus.

202 A hardware structure of the processoris various processors as described below. The various types of processors include a central processing unit (CPU) that is a general-purpose processor which acts as various types of functional units by executing software (program), a graphics processing unit (GPU) that is a processor specialized in image processing, a programmable logic device (PLD) that is a processor of which a circuit configuration is changeable after manufacture, such as a field programmable gate array (FPGA), and a dedicated electric circuit that is a processor which has a circuit configuration specifically designed in order to execute specific processing, such as an application specific integrated circuit (ASIC).

One processing unit may be configured by one of the various types of processors or may be configured by the same type or different types of two or more processors (for example, a plurality of FPGAs, a combination of a CPU and an FPGA, or a combination of a CPU and a GPU). In addition, one processor may configure a plurality of functional units. As an example of configuring a plurality of functional units by one processor, first, there is a form in which one processor is configured by a combination of one or more CPUs and software and the processor acts as the plurality of functional units, as represented by a computer such as a client and a server. Second, there is a form in which a processor that realizes functions of the entire system including a plurality of functional units with one integrated circuit (IC) chip is used, as represented by a system on chip (SoC) or the like. As described above, the various types of functional units are configured by one or more of the various types of processors used as a hardware structure.

Further, the hardware structure of the various types of processors is, more specifically, an electric circuit (circuitry) in which circuit elements such as semiconductor elements are combined.

202 204 206 208 210 The processoris connected to the computer-readable medium, the communication interface, and the input/output interfacethrough the bus.

204 202 204 204 204 The computer-readable mediumstores an instruction to be executed by the processor. The computer-readable mediumincludes a memory that is a main memory, and a storage that is an auxiliary memory. For example, the computer-readable mediummay be a semiconductor memory, a hard disk drive (HDD) device, a solid state drive (SSD) device, or a combination thereof. The computer-readable mediumstores various programs, data, and the like including an image processing program.

206 22 The communication interfacecontrols communication via the network.

208 214 216 20 The input/output interfaceis connected to an input deviceand a display device, and controls input and output to the image processing apparatus.

214 The input deviceis configured by, for example, a keyboard, a mouse, a multi-touch panel, another pointing device, a voice input device, or an appropriate combination thereof.

216 The display deviceis configured, for example, by using a liquid crystal display, an organic electro-luminescence (OEL) display, a projector, or an appropriate combination thereof.

20 214 216 The image processing apparatusmay have a configuration including the input deviceand the display device.

4 FIG. 20 20 100 102 104 106 108 110 20 202 204 is a block diagram showing a functional configuration example of the image processing apparatus. The image processing apparatusincludes a set area acquisition unit, a captured image acquisition unit, a map information acquisition unit, a meta-information calculation unit, an imaging status calculation unit, and a visualization information output unit. Each function of the image processing apparatusis materialized by the processorexecuting the program stored in the computer-readable medium.

100 100 The set area acquisition unitacquires a set area for visualizing the imaging status. For example, the user divides an aerial imaging region that is a certain imaging region into a plurality of regions for which the respective imaging status is to be visualized. It is preferable that the plurality of regions are divided such that there is no gap between the regions adjacent to each other and there is no overlap between the regions adjacent to each other. Each of the plurality of regions may be a different administrative division. The set area acquisition unitacquires each of the divided plurality of regions as a plurality of set areas.

102 14 14 102 The captured image acquisition unitacquires a plurality of images (an example of a “group of images”) in which the aerial imaging region is captured by using the camerafrom the camera. The captured image acquisition unitacquires a plurality of images in which a part of the aerial imaging region is respectively captured, and the plurality of images are captured with an overlap region with images adjacent to each other.

104 102 104 204 214 22 The map information acquisition unitacquires map information including an imaging region of the plurality of images acquired by the captured image acquisition unit, that is, map information including the aerial imaging region. The map information acquisition unitmay acquire the map information from the computer-readable medium, may acquire the map information from the input device, or may acquire the map information via the network.

106 102 The meta-information calculation unitcalculates the meta-information of each image of the plurality of images acquired by the captured image acquisition unit. The meta-information of each image includes, for example, latitude and longitude information including the latitude and the longitude of four corners of the imaging region of each image, and a ground resolution of a house included in each image.

106 14 14 The meta-information calculation unitmay acquire sensor data and camera information recorded as an exchange image file format (exif) data from each image. The sensor data includes the latitude, the longitude, the altitude, the roll angle, the pitch angle, and the yaw angle of the cameraat the time of imaging. The camera information includes a sensor size and a focal length of the camera.

106 104 14 106 The meta-information calculation unitmay match the sensor data and the camera information of each image with the map information acquired by the map information acquisition unitto obtain accurate position information and accurate orientation information of the cameraat the time of imaging. The meta-information calculation unitmay apply a georeferencing technique to the captured image and project a road line segment of the map to the captured image with an accuracy of about within an error of 3 to 5 meters.

106 104 The meta-information calculation unitmay acquire house information from the map information acquired by the map information acquisition unit.

108 106 The imaging status calculation unitcalculates the imaging status of the set area based on the meta-information of each image calculated by the meta-information calculation unit. The imaging status of the set area includes the number of sheets of captured images, the camera angle distribution, the ground resolution distribution, the imaging coverage rate, and the imaging overlap rate.

The number of sheets of captured images is the number of images in which the set area is captured. The number of sheets of captured images may be a total number of images in which the set area is captured, or may be the number of images per unit area. The unit area may be 1 square kilometer.

14 14 The camera angle is an angle of a downward inclination of an optical axis of the lens of the camerawith respect to a horizontal direction. The camera angle is obtained from the orientation information of the cameraor the pitch angle of the sensor data. The camera angle distribution is a distribution of an appearance frequency of the camera angle for each image. The camera angle distribution is represented by, for example, a histogram showing the number of images belonging to each of intervals of a predetermined constant interval of the value of the camera angle. In addition, the camera angle distribution may be the number of images in which the nadir is captured and the number of images in which the oblique is captured, or may be a ratio of these.

14 The ground resolution is a size of a house included in the image, that is, a house captured in the image in a real space per one pixel of the image. The ground resolution is obtained from the sensor size of the camera, an actual distance of four corners of the imaging range, and information on a position of the house obtained from the map information. The ground resolution distribution is a distribution of an appearance frequency of the ground resolution for each house. The ground resolution distribution is represented by, for example, a histogram showing the number of houses belonging to each of intervals of a predetermined constant interval of the value of the ground resolution.

The imaging coverage rate includes an area coverage rate and a house coverage rate. The area coverage rate is a ratio of an area of a region captured in the set area to an area of the set area. The area of the set area is obtained from the map information of the set area. In addition, the area of the captured region is obtained from the latitude and longitude information of the four corners of each image. The house coverage rate is a ratio of the number of captured houses to the number of houses in the set area. The number of houses in the set area is obtained from the house information of the map information. In addition, the number of captured houses is obtained from the house information of the map information and the latitude and longitude information of the four corners of each image.

The imaging overlap rate is a ratio of an overlap region that is captured to overlap with an image other than the image to the imaging region shown in the image. The imaging overlap rate is obtained from an average of ratios of an area of the imaging region of each image to an area of the overlap region of the image.

110 110 The visualization information output unitgenerates and outputs the visualization information in which the result of the calculated imaging status is superimposed on the set area of the map. The visualization information output unitmay generate and output the visualization information in which the result of the calculated imaging status is superimposed on the set area of the map according to a predetermined legend. The predetermined legend may fill the set area with at least one of a color, a shade, or a hatching corresponding to the result of the calculated imaging status. The filling includes making the filled area transparent in order to recognize information on the map such as a road and a house even after the filling. The predetermined legend may superimpose a contour line corresponding to the result of the calculated imaging status. The contour line may outline the set area with at least one of a color, a shade, or a line type corresponding to the result of the calculated imaging status.

110 216 204 22 206 The visualization information output from the visualization information output unitis displayed on, for example, the display device. The visualization information may be stored in the computer-readable medium, or may be transmitted to a device connected to the networkvia the communication interface.

110 The user such as the imaging person can easily ascertain the imaging status of the set area by the visualization information output from the visualization information output unit.

5 FIG. 12 202 204 22 is a flowchart showing each step of the image processing method. The image processing method enables the user such as the imaging person to easily ascertain the imaging status of the set area for a plurality of aerial images captured by using the drone. The image processing method is implemented by the processorexecuting an image processing program stored in the computer-readable medium. The image processing program may be provided by a non-transitory computer-readable storage medium, or may be provided via the network.

1 20 In step S, the user sets an area for ascertaining the imaging status. The image processing apparatusacquires the area set by the user as the set area.

100 216 214 For example, the user of the local government that performs aerial imaging may divide the local government area that is the aerial imaging region into a plurality of areas in units of districts, towns, villages, blocks, or the like. In a case where the user divides the local government area into a plurality of areas, the set area acquisition unitacquires the plurality of areas as each set area. The user may set the area by inputting a selection instruction to the map displayed on the display deviceby using the input device.

6 FIG. 6 FIG. 1 2 3 is a diagram showing an example of the set area. Here, an example is shown in which the user divides the local government area into a plurality of areas of “1-chome, A Town”, “2-chome, A Town”, and “3-chome, A Town”. As shown in, “1-chome, A Town” is set as a set area AS, “2-chome, A Town” is set as a set area AS, and “3-chome, A Town” is set as a set area AS.

2 20 In step S, the image processing apparatusacquires the plurality of images.

102 12 Here, the captured image acquisition unitacquires a plurality of still images in which the local government area is captured with overlap in one flight of the drone, and a part of the local government area is respectively captured.

3 20 2 In step S, the image processing apparatuscalculates the meta-information of each image of the plurality of images acquired in step S.

104 106 104 Here, the map information acquisition unitacquires map information including the local government area that is the aerial imaging region. The meta-information calculation unitacquires house information from the map information acquired by the map information acquisition unit.

106 2 106 In addition, the meta-information calculation unitacquires the exif data from each image of the plurality of images acquired in step S. Further, the meta-information calculation unitacquires the sensor data and the camera information from the exif data.

106 14 106 14 106 14 Next, the meta-information calculation unitmatches the sensor data and the camera information of each image with the map information to obtain the accurate position information and the accurate orientation information of the cameraat the time of imaging. Subsequently, the meta-information calculation unitcalculates the latitude and longitude information of four corners of the imaging region of each image based on the obtained position information and orientation information of the camera. Further, the meta-information calculation unitcalculates the ground resolution of each house in the image based on the house information and the position information and the orientation information of the camera.

7 FIG. 7 FIG. 106 7 7 is a diagram showing an example of a calculation result of the meta-information calculation unit. FA shown inis a table showing the position information of the camera at the time of imaging, the orientation information of the camera, and the latitude and longitude information of four corners of the imaging region for each image. In FA, for each of images “0001.jpg”, “0002.jpg”, and the like, the latitude and longitude information as the camera position information and the roll angle, the pitch angle, and the yaw angle are calculated as the camera orientation information.

7 7 1 1 2 2 3 3 4 4 7 FIG. FB shown inis a diagram showing an image of four corners of the imaging region of the image “0001.jpg”. In FB, (lat, lon), (lat, lon), (lat, lon), and (lat, lon), which are the latitude and longitude information of four corners of the imaging region of the image “0001.jpg”, are plotted at corresponding positions on the map, and the imaging region is indicated by surrounding the plotted points with a straight line.

7 7 FIG. FC shown inis a table showing the ground resolution for each house for each image. Here, the image in which each house of house IDs “Bld001”, “Bld002”, . . . is shown and the ground resolution are associated with each other. For example, it is shown that the house of the house ID “Bld001” is shown with a ground resolution of “5 cm” in the image “0001.jpg” and is shown with a ground resolution of “15 cm” in the image “0002.jpg”.

5 FIG. 4 20 3 1 Returning to the description of, in step S, the image processing apparatusaggregates the meta-information calculated in step Sin units of the set area acquired in step S.

108 1 Here, the imaging status calculation unitcalculates the number of sheets of captured images, the camera angle distribution, the ground resolution distribution, and the imaging coverage rate for each set area of the plurality of set areas acquired in step S, respectively.

5 20 4 216 216 20 Finally, in step S, the image processing apparatusgenerates the visualization information in which the meta-information aggregated in step Sin units of the set area is superimposed on a region of the set area on the map, and outputs the visualization information to the display device. The display devicedisplays the visualization information acquired from the image processing apparatus.

5 The user such as the imaging person can easily ascertain the imaging status of the set area by the visualization information output in step S.

8 FIG. 8 FIG. 216 5 1 1 2 3 is a diagram showing an example of the visualization information displayed on the display devicein step S. In visualization information IVshown in, each of the set area AS, the set area AS, and the set area ASis hatched with hatching corresponding to the meta-information, and the hatching of the predetermined legend is superimposed on the region of each set area on the map. The meta-information includes at least one of the number of sheets of captured images, the camera angle distribution, the ground resolution distribution, the imaging coverage rate, or the imaging overlap rate.

1 The user can ascertain a difference in the imaging status for each set area by the visualization information IV.

1 2 3 214 1 216 8 FIG. In a case where the user performs a selection operation on any of the set area AS, the set area AS, or the set area ASby using the input devicein a state in which the visualization information IVshown inis displayed on the display device, the imaging status of the selected set area is displayed in a pop-up manner. The selection operation is, for example, a click operation after moving a mouse cursor to a desired set area.

9 FIG. 9 FIG. 216 9 2 2 1 2 1 2 1 1 is a diagram showing an example of the visualization information displayed on the display deviceafter the selection operation of the set area. FA shown inshows the visualization information IVin a case where the set area ASof the visualization information IVis subjected to the selection operation. In the visualization information IV, a pop-up display PUis displayed as the text information for the set area AS. The pop-up display PUhas a balloon shape and includes a rectangular portion for indicating information on the set area and a sharp portion for indicating which set area it is. The pop-up display PUincludes an “area name”, the “number of sheets of captured images”, the “camera angle distribution”, the “ground resolution distribution”, and the “imaging coverage rate”. Here, the “area name” is “2-chome, A Town”, the “number of sheets of captured images” is “A sheets”, and the “imaging coverage rate” is “B %”.

2 1 2 1 Here, the example has been described in which the visualization information IVis displayed in a pop-up manner on the visualization information IV, but the display screen of the visualization information IVmay be transitioned and displayed from the display screen of the visualization information IV.

9 3 3 1 3 2 3 2 9 FIG. FB shown inshows the visualization information IVin a case where the set area ASof the visualization information IVis subjected to the selection operation. In the visualization information IV, a pop-up display PUis displayed as the text information for the set area AS. The pop-up display PUincludes an “area name”, the “number of sheets of captured images”, the “camera angle distribution”, the “ground resolution distribution”, and the “imaging coverage rate”. Here, the “area name” is “3-chome, A Town ”, the “number of sheets of captured images” is “C sheets”, and the “imaging coverage rate” is “D %”.

2 3 In the visualization information IVand the visualization information IV, the values are not displayed for the “camera angle distribution” and the “ground resolution distribution”, but may be displayed. For example, for the “camera angle distribution”, the “number of images in which the nadir is captured” and the “number of images in which the oblique is captured” may be displayed. In addition, for the “ground resolution distribution”, an “average value of the ground resolution of all houses” may be displayed.

2 3 The user can ascertain the imaging status of the set area of which the details are desired to be known by the visualization information IVand the visualization information IV.

214 2 3 216 216 9 FIG. In a case where the user performs a selection operation on any of the items of “camera angle distribution” or “ground resolution distribution” by using the input devicein a state in which the visualization information IVor the visualization information IVshown inis displayed on the display device, the display devicetransitions to the display of the visualization information of the selected item. The visualization information of the selected item is, for example, a graph display of the selected item.

10 FIG. 10 FIG. 216 10 4 1 4 2 is a diagram showing an example of the visualization information displayed on the display deviceafter the selection operation of the item. FA shown inshows the visualization information IVin a case where the “camera angle distribution” of the pop-up display PUis subjected to the selection operation. The visualization information IVincludes a histogram showing a distribution of the camera angles of the images in which the set area ASis captured. In the histogram, the horizontal axis indicates “camera angle (unit: degree)”, and the vertical axis indicates “number of images (unit: sheets)”. Here, the image in which the camera angle at the time of imaging is 25 to 34 degrees is classified as “30 degrees”, and the image in which the camera angle at the time of imaging is 35 to 44 degrees is classified as “40 degrees”. Similarly, the image in which the camera angle at the time of imaging is 75 to 84 degrees is classified as “80 degrees”, and the image in which the camera angle at the time of imaging is 85 to 94 degrees is classified as “90 degrees”. In addition, here, the “camera angle” is classified as “90 degrees” and “80 degrees” as “nadir”, and “40 degrees” and “30 degrees” as “oblique”.

4 The user can ascertain the camera angle distribution of the set area of which the details are desired to be known by the visualization information IV.

214 4 10 216 In a case where the user performs a selection operation on any of the camera angles of the “camera angle” of the histogram by using the input devicein a state in which the visualization information IVshown in FA is displayed on the display device, the visualization information of the selected camera angle is displayed. The visualization information of the selected camera angle is, for example, a region captured at the selected camera angle.

10 5 4 5 1 2 1 10 FIG. FB shown inshows the visualization information IVin a case where “40 degrees” of the camera angles of the visualization information IVis subjected to the selection operation. The visualization information IVincludes a map MPof the set area AS, and 12 regions AP each captured by 12 images in which the camera angle at the time of imaging is 35 to 44 degrees are superimposed on the map MP.

5 The user can ascertain the region captured at the selected camera angle in the set area by the visualization information IV.

11 FIG. 11 FIG. 216 11 6 1 6 2 is a diagram showing another example of the visualization information displayed on the display deviceafter the selection operation of the item. FA shown inshows the visualization information IVin a case where the “ground resolution distribution” of the pop-up display PUis subjected to the selection operation. The visualization information IVincludes a histogram showing a distribution of the ground resolution of the house shown in the image in which the set area ASis captured. In the histogram, the horizontal axis indicates “ground resolution (unit: centimeter/pixel)”, and the vertical axis indicates the “number of houses (unit: houses)”. Here, each house is classified into a ground resolution of 4 to 30 centimeters/pixel in increments of 1 centimeter/pixel. For example, the house having a ground resolution of 2.5 to 3.4 centimeters/pixel is classified as “3 centimeters/pixel”.

6 The user can ascertain the ground resolution distribution of the set area of which the details are desired to be known by the visualization information IV.

214 6 11 216 In a case where the user performs a selection operation on any of the ground resolutions of the “ground resolution” of the histogram by using the input devicein a state in which the visualization information IVshown in FA is displayed on the display device, the visualization information of the selected ground resolution is displayed. The visualization information of the ground resolution is, for example, a position of the house captured at the selected ground resolution.

11 7 6 7 1 2 1 11 FIG. FB shown inshows the visualization information IVin a case where “3 centimeters/pixel” of the “ground resolution” of the visualization information IVis subjected to the selection operation. The visualization information IVincludes a map MPof the set area AS, and a position of a house BL having a ground resolution of 3 centimeters/pixel is superimposed on the map MP.

7 The user can ascertain the house captured at the selected ground resolution in the house of the set area by the visualization information IV.

12 FIG. 12 FIG. 216 12 8 8 2 2 is a diagram showing another example of the visualization information displayed on the display device. FA shown inshows the visualization information IV. The visualization information IVincludes a map MPincluding the aerial imaging region, and a region AF captured on the map MPis superimposed. The region AF is a region captured by the plurality of images. The region AF is calculated from the latitude and longitude information of four corners of the imaging region of each image. The number of sheets of images in which each region AF is captured may be superimposed on the region AF.

8 The user can ascertain a range of the captured region by the visualization information IV.

214 8 12 216 216 In a case where the user performs a selection operation on any of the regions AF by using the input devicein a state in which the visualization information IVshown in FA is displayed on the display device, the display devicetransitions to the display of the imaging status of the selected region AF.

12 9 8 9 9 12 FIG. FB shown inshows the visualization information IVin a case where any of the regions AF of the visualization information IVis subjected to the selection operation. The visualization information IVis the imaging status for the selected region AF. The visualization information IVincludes the “number of sheets of captured images”, the “camera angle”, the “ground resolution”, and the “imaging overlap rate”. Here, the “number of sheets of captured images” is “E sheets (F sheets per square kilometer)”, the “camera angle” is “nadir G sheets/oblique H sheets”, the “ground resolution” is “average I centimeters”, and the “imaging overlap rate” is J %.

214 9 12 216 216 In a case where the user performs a selection operation on any of the items of “the number of sheets of captured images”, “camera angle”, “ground resolution”, and “imaging overlap rate” by using the input devicein a state in which the visualization information IVshown in FB is displayed on the display device, the display devicemay transition to the display of the visualization information of the selected item. The visualization information of the selected item is, for example, a graph display of the selected item.

The technical scope of the present invention is not limited to the scope described in the above-described embodiments. The configuration and the like in each embodiment can be combined between the embodiments as appropriate without departing from the gist of the present invention.

10 : captured image processing system 12 : drone 13 : gimbal head 14 : camera 16 : remote controller 16 A: display 20 : image processing apparatus 22 : network 30 : GPS receiver 32 : atmospheric pressure sensor 34 : azimuth sensor 36 : gyro sensor 38 : motor 40 : processor 42 : storage device 44 : communication interface 100 : set area acquisition unit 102 : captured image acquisition unit 104 : map information acquisition unit 106 : meta-information calculation unit 108 : imaging status calculation unit 110 : visualization information output unit 202 : processor 204 : computer-readable medium 206 : communication interface 208 : input/output interface 210 : bus 214 : input device 216 : display device AF: region AP: region 1 AS: set area 2 AS: set area 3 AS: set area BL: house ID: house 1 IV: visualization information 2 IV: visualization information 3 IV: visualization information 4 IV: visualization information 5 IV: visualization information 6 IV: visualization information 7 IV: visualization information 8 IV: visualization information 9 IV: visualization information 1 MP: map 2 MP: map 1 PU: pop-up display 2 PU: pop-up display 1 5 Sto S: steps of image processing method