Robust Image Registration for Multi-Spectral/Multi-Modality Imagery

1. A method of registering images comprising: obtain a master image from a first image sensor sensitive in a first band; obtain a first image from a second image sensor sensitive in a second band different from the first band: identifying a coarse warp for estimated registration of the first image from the second image sensor and the second band to the master image from the first image sensor and the first band; applying the coarse warp to the first image from the second image sensor and the second band to produce a dewarped image; identifying a block size of pixels for relative warp estimation on a block by block basis to improve registration of the dewarped image to the master image from the first image sensor and the first band; performing the relative warp estimation on a block by block basis to produce relative warp data, wherein the relative warp data includes a warp vector for each block of the master image from the first image sensor and the first band and its associated uncertainty, wherein uncertainty is covariance of an optimal translation of the relative warp estimation; fitting a warp model to the relative warp data to produce reduced noise relative warp data; combining the reduced noise relative warp data with the coarse warp to produce a refined warp; applying the refined warp to the first image from the second image sensor and second band for improved registration of the first image from the second image sensor and the second band to the master image from the first image sensor and the first band; and outputting a composite image in a non-transitory tangible medium of the master image from the first image sensor and the first band and a final dewarped image of the first image from the second image sensor and the second band.

2. The method as recited in claim 1 , further comprising: iteratively repeating a process of identifying a coarse warp, applying the coarse warp, identifying a block size, performing the relative warp estimation, fitting a warp model, combining the reduced noise relative warp data with the coarse warp, and applying the refined warp to the first image as recited in claim 1 , wherein on a first iteration of the process, the coarse warp is an identity map, and wherein for each successive iteration of the process, the refined warp of an immediately preceding iteration is used as the coarse warp.

3. The method as recited in claim 2 , further comprising: terminating the iterative repeating of the process upon the relative warp meeting a predetermined condition indicative of a predetermined degree of registration between the dewarped image and the master image, wherein on a final iteration of the process, applying the refined warp to the first image produces a final dewarped image registered to the master image.

4. The method as recited in claim 2 , wherein identifying a block size includes identifying a successively more refined block size with each successive iteration of the process.

5. The method as recited in claim 4 , wherein each of the coarse, refined, and relative warps includes a warp vector for each block, and wherein applying the respective warp to the first image to produce a dewarped image includes producing a pixel by pixel warp based on the respective warp vectors for the blocks.

6. The method as recited in claim 5 , wherein producing the pixel by pixel warp includes interpolation of the warp vectors of the blocks to provide a respective warp vector for each pixel.

7. The method as recited in claim 1 , wherein fitting a warp model to the relative warp data to produce reduced noise relative warp data includes smoothing the warp vectors relative to one another to reduce occurrences of outliers among the warp vectors and estimation noise, wherein more smoothing is done where the warp vectors have greater associated uncertainty, and less smoothing is done where the warp vectors have less associated uncertainty.

8. The method as recited in claim 1 , wherein the warp model includes a Markov Random Field (MRF) to model smoothness for the relative warp and a Gaussian noise model for modeling uncertainty for the relative warp.

9. The method as recited in claim 1 , wherein performing the relative warp estimation includes: computing gradients for each of the dewarped and master images; computing noise from the gradients of each of the dewarped and master images; normalizing the gradients and reducing their magnitudes in areas of high noise to produce normalized gradients for each of the dewarped and master images; dividing the dewarped and master images into blocks of identified block size; correlating normalized gradient vectors of the dewarped image and master image for each block to produce correlation for all possible translations between the dewarped image and master image blocks; finding a translation between the dewarped and master image block that yields maximum correlation to compute an estimated warp vector for each block; computing uncertainty of the estimated warp vector for each block from the correlation; and collecting the estimated warp vector and associated uncertainty for each block to produce the relative warp data.

13. The method as recited in claim 9 , wherein computing uncertainty includes computing uncertainty Ĉ wherein C ^ = ∑ d x , d y ⁢ P ⁡ ( d x , d y ) ⁡ [ ( d x - d ^ x ) 2 ( d x - d ^ x ) ⁢ ( d y - d ^ y ) ( d x - d ^ x ) ⁢ ( d y - d ^ y ) ( d y - d ^ y ) 2 ] , where {{circumflex over (d)} x ,{circumflex over (d)} y } denotes the translation where the correlation S is maximized and the summation is done over all possible translations {d x ,d y } between the blocks of the dewarped and the master image; the probability is computed as P ⁡ ( d x , d y ) = L ⁡ ( d x , d y ) ∑ d x , d y ⁢ L ⁡ ( d x , d y ) , where the likelihood is computed as L ⁡ ( d x , d y ) = exp ⁡ ( - ( max ⁢ ⁢ S S ⁡ ( d x , d y ) ) 2 ) , from the correlation S and max S denotes the maximum value of the correlation S over all possible translations.

14. The method as recited in claim 1 , wherein identifying a coarse warp for estimated registration of a first image to a master image includes identifying a respective coarse warp for each of a plurality of first images; wherein applying the coarse warp includes applying the respective coarse warp to each of the first images to produce a respective dewarped image for each of the first images; wherein identifying a block size includes identifying a respective block size for each of the first images for a warp estimation on a block by block basis to improve registration of the respective dewarped images to the master image; performing the warp estimation includes: forming pairs of the first images and master image to one another wherein at least one of the pairs of images includes one of the first images and the master image; performing a pair-wise relative warp estimation for each pair on a block by block basis to produce pair-wise relative warp data, wherein the pair-wise relative warp data includes a warp vector for each block and its associated uncertainty; and computing respective set of relative warp data for each of the dewarped images with respect to the master image from the pair-wise relative warp data; performing a warp model fit on each set of relative warp data to produce a respective set of reduced noise relative warp data for each dewarped image; combining the respective set of reduced noise relative warp data with the respective coarse warp to produce a respective refined warp for each respective dewarped image; and applying the respective refined warp to the each respective first image for improved registration of the first images to the master image.

15. The method as recited in claim 14 , wherein computing a set of respective relative warp data for each of the dewarped images with respect to the master image from the pair-wise relative warp data includes: forming a tree of the pair-wise groupings with the master image as a root and each first image as a node with a connection between each pair of paired images; grouping dewarped images by the depth in the tree, wherein depth in the tree is a measure of how far removed a given node is from the root; wherein for group 1 images, the pair-wise relative warp data is for a relative warp with respect to the master image; wherein for group i+1 images, the relative warp data with respect to the master is obtained by summing the pair-wise relative warp of the respective image with corresponding relative warp data of the group i image with respect to the master image; and repeating computation for each group in the order of increasing group number until relative warp with respect to master image is computed for all first images.

16. A system for registering images including: a module configured to implement machine readable instructions to perform the method recited in claim 1 .

17. A system as recited in claim 16 , wherein the module is configured to implement machine readable instructions to compute gradients for each of the dewarped and master images; compute noise from the gradients of each of the dewarped and master images; normalize the gradients and reducing their magnitudes in areas of high noise to produce normalized gradients for each of the dewarped and master images; divide the dewarped and master images into blocks of identified block size; correlate normalized gradient vectors of the dewarped image and master image for each block to produce correlation for all possible translations between the dewarped image and master image blocks; find a translation between the dewarped and master image block that yields maximum correlation to compute an estimated warp vector for each block; compute uncertainty of the estimated warp vector for each block from the correlation; and collect the estimated warp vector and associated uncertainty for each block to produce the relative warp data.

18. A system as recited in claim 16 , wherein identifying a coarse warp for estimated registration of a first image to a master image includes identifying a respective coarse warp for each of a plurality of first images; wherein applying the coarse warp includes applying the respective coarse warp to each of the first images to produce a respective dewarped image for each of the first images; wherein identifying a block size includes identifying a respective block size for each of the first images for a warp estimation on a block by block basis to improve registration of the respective dewarped images to the master image; performing the warp estimation includes: forming pairs of the first images and master image to one another wherein at least one of the pairs of images includes one of the first images and the master image; performing a pair-wise relative warp estimation for each pair on a block by block basis to produce pair-wise relative warp data, wherein the pair-wise relative warp data includes a warp vector for each block and its associated uncertainty; and computing respective set of relative warp data for each of the dewarped images with respect to the master image from the pair-wise relative warp data; performing a warp model fit on each set of relative warp data to produce a respective set of reduced noise relative warp data for each dewarped image; combining the respective set of reduced noise relative warp data with the respective coarse warp to produce a respective refined warp for each respective dewarped image; and applying the respective refined warp to the each respective first image for improved registration of the first images to the master image.

19. A method of registering images comprising: obtain a master image from a first image sensor sensitive in a first band; obtain a first image from a second image sensor sensitive in a second band different from the first band; identifying a block size of pixels for a warp estimation on a block by block basis to improve registration of the first image from the second image sensor and the second band to the master image from the first image sensor and the first band; performing the relative warp estimation on a block by block basis to produce relative warp data, wherein the relative warp data includes a warp vector for each block of the master image from the first image sensor and the first band and its associated uncertainty, wherein uncertainty is covariance of an optimal translation of the relative warp estimation; performing a warp model fit on the warp data to produce reduced noise warp data; applying the reduced noise warp data to the first image from the second image sensor and the second band for improved registration of the first image from the second image sensor and the second band to the master image from the first image sensor and the first band; outputting a composite image in a non-transitory tangible medium of the master image from the first image sensor and the first band and a final dewarped image of the first image from the second image sensor and the second band.