The present invention enables a high-quality filtered image to be generated even from multimodal and multispectral images containing positional deviations. An image perturbation part generates a perturbed guide image group comprising first to K-th perturbed guide images obtained by deforming a guide image. A filtering part generates a filtered image group comprising first to K-th filtered images by applying first to K-th filtering processing to a target image by using the perturbed guide image group. A reliability calculation part calculates a reliability group comprising first to K-th reliabilities for the first to K-th filtered images of the filtered image group on the basis of first to K-th correlation values between the first to K-th perturbed guide images and the target image. A weight optimization part generates, on the basis of the first to K-th reliabilities, a weight group comprising first to K-th weights to be respectively used when compositing the first to K-th filtered images. An output image compositing part composites an output image from the weight group and the filtered image group.
Legal claims defining the scope of protection, as filed with the USPTO.
1. An image compositing system comprising: an image perturbation part configured to generate a perturbed guide image group comprising first to K-th perturbed guide images obtained by deforming a guide image, where K represents an integer which is not less than two; a filtering part configured to apply first to K-th filtering processing to a target image by using the perturbed image guide group to generate a filtered image group comprising first to K-th filtered images; a reliability calculation part configured to calculate a reliability group comprising first to K-th reliabilities for the first to K-th filtered images of the filtered image group on the basis of a correlation value group comprising first to K-th correlation values between the first to K-th perturbed guide images and the target image; a weight optimization part configured to calculate, on the basis of the first to K-th reliabilities, a weight group comprising first to K-th weights to be respectively used when compositing the first to K-th filtered images; and an output image compositing part configured to composite an output image from the weight group and the filtered image group.
2. The image compositing system as claimed in claim 1 , wherein the image perturbation part is configured to generate the perturbed guide image group by deforming the guide image with respect to an amount of positional deviations.
3. The image compositing system as claimed in claim 1 , wherein the reliability calculation part is configured to calculate the first to K-th correlation values between the first to K-th perturbed guide images and the target image on the basis of an objective function group comprising first to K-th objective functions which are used on the first to K-th filtering processing of the corresponding filtering part.
4. The image compositing system as claimed in claim 1 , wherein the weight optimization part is configured to calculate the weight group by minimizing an energy function based on a fidelity term related to the reliability group calculated by the reliability calculation part and on a spatial smoothness of the reliability group.
5. The image compositing system as claimed in claim 1 , wherein the weight optimization part is configured to calculate the weight group by minimizing an energy function based on a fidelity term related to the reliability group calculated by the reliability calculation part, on a spatial smoothness of the reliability group, and on a sparseness of the reliability group.
6. The image compositing system as claimed in claim 1 , wherein the weight optimization part is configured to use, as the first to K-th weights, 1 for one among the first to K-th reliabilities calculated by the reliability calculation part that has a maximum value per each pixel and 0 for the other reliabilities.
7. An image compositing method comprising: generating a perturbed guide image group comprising first to K-th perturbed guide images obtained by deforming a guide image, where K represents an integer not less than two; applying, first to K-th filtering processing to a target image by using the perturbed guide image group to generate a filtered image group comprising first to K-th filtered images; calculating a reliability group comprising first to K-th reliabilities for the first to K-th filtered images of the filtered image group on the basis of a correlation value group comprising first to K-th correlation values between the first to K-th perturbed images and the target image; calculating on the basis of the first to K-th reliabilities, a weight group comprising first to K-th weights to be respectively used when compositing the first to K-th filtered images; and compositing an output image from the weight group and the filtered image group.
8. The image compositing method as claimed in claim 7 , wherein the generating the perturbed guide image group generates the perturbed guide image group by deforming the guide image with respect to an amount of positional deviations.
9. The image compositing method as claimed in claim 7 , wherein the calculating the reliability group calculates the first to K-th correlation values between the first to K-th perturbed guide images and the target image on the basis of an objective function group comprising first to K-th objective functions which are used on the first to K-th filtering processing.
10. The image compositing method as claimed in claim 7 , wherein the calculating the weight group calculates the weight group by minimizing an energy function based on a fidelity term related to the reliability group calculated in the reliability calculation step and on a spatial smoothness of the reliability group.
11. The image compositing method as claimed in claim 7 , wherein the calculating the weight group calculates the weight group by minimizing an energy function based on a fidelity term related to the reliability group calculated in the reliability calculation step, on a spatial smoothness of the reliability group, and on a sparseness of the reliability group.
12. The image compositing method as claimed in claim 7 , wherein the calculating the weight group uses, as the first to K-th weights, 1 for one among the first to K-th reliabilities calculated in the reliability calculation step that has a maximum value per each pixel and 0 for the other reliabilities.
13. A non-transitory computer readable image compositing program recording medium recording an image compositing program for causing a computer to execute: an image perturbation procedure for generating a perturbed guide image group comprising first to K-th perturbed guide images obtained by deforming a guide image, where K represents an integer which is not less than two; a filtering procedure for applying first to K-th filtering processing to a target image by using the perturbed image guide group to generate a filtered image group comprising first to K-th filtered images; a reliability calculation procedure for calculating a reliability group comprising first to K-th reliabilities for the first to K-th filtered images of the filtered image group on the basis of a correlation value group comprising first to K-th correlation values between the first to K-th perturbed guide images and the target image; a weight optimization procedure for calculating, on the basis of the first to K-th reliabilities, a weight group comprising first to K-th weights to be respectively used when compositing the first to K-th filtered images; and an output image compositing procedure for compositing an output image from the weight group and the filtered image group.
14. The non-transitory computer readable image compositing program recording medium as claimed in claim 13 , wherein the image perturbation procedure causes the computer to generate the perturbed guide image group by deforming the guide image with respect to an amount of positional deviations.
15. The non-transitory computer readable image compositing program recording medium as claimed in claim 13 , wherein the reliability calculation procedure causes the computer to calculate the first to K-th correlation values between the first to K-th perturbed guide images and the target image on the basis of an objective function group comprising first to K-th objective functions which are used on the first to K-th filtering processing of the filtering procedure.
16. The non-transitory computer readable image compositing program recording medium as claimed in claim 13 , wherein the weight optimization procedure causes the computer to calculate the weight group by minimizing an energy function based on a fidelity term related to the reliability group calculated by the reliability calculation procedure and on a spatial smoothness of the reliability group.
17. The non-transitory computer readable image compositing program recording medium as claimed in claim 13 , wherein the weight optimization procedure causes the computer to calculate the weight group by minimizing an energy function based on a fidelity term related to the reliability group calculated by the reliability calculation procedure, on a spatial smoothness of the reliability group, and on a sparseness of the reliability group.
18. The non-transitory computer readable image compositing program recording medium as claimed in claim 13 , wherein the weight optimization procedure causes the computer to use, as the first to K-th weights, 1 for one among the first to K-th reliabilities calculated in the reliability calculation step that has a maximum value per each pixel and 0 for the other reliabilities.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
October 26, 2017
March 9, 2021
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.