Shadow Mask Generation Using Elevation Data

1. A non-transitory machine-readable medium including instructions that, when executed by a machine, cause the machine to perform operations comprising: determining, based on elevation data of a geographic region corresponding to a location at which an image was captured and a solar elevation angle at a time the image was captured, whether each pixel of the image is a shadow or a non-shadow to create a shadow mask of the image; generating an eroded shadow mask that includes the shadow mask within a specified number of pixel values from a perimeter of each shadow in the shadow mask changed to respective values corresponding to non-shadows; generating a dilated shadow mask that includes the specified number of pixel values in the shadow mask changed to values corresponding to shadows; and refining the shadow mask using the image, the eroded shadow mask, and the dilated shadow mask including identifying a pixel in the dilated shadow mask that includes a value different from a corresponding pixel in the eroded shadow mask, determine, for each identified pixel, a mean shadow value based on pixels in a window of pixels centered on the identified pixel that include a shadow mask value less than a shadow threshold, determine, for each identified pixel, a mean non-shadow value based on pixels in the window that include a shadow mask value greater than a non-shadow threshold, and update the value of the shadow mask pixel based on the intensity of that pixel and the determined mean shadow value and mean non-shadow value.

2. The non-transitory machine-readable medium of claim 1 , wherein the operations comprise: generating a shadow histogram of non-shadow values in the complement of the dilated shadow mask; and generating a non-shadow histogram of shadow values in the eroded shadow mask.

3. The non-transitory machine-readable medium of claim 2 , wherein the operations further comprise: smoothing the shadow histogram and the non-shadow histogram; and estimating a probability that a pixel corresponds to a shadow based on the smoothed shadow histogram and the smoothed non-shadow histogram.

4. The non-transitory machine-readable medium of claim 3 , wherein the operations further comprise: weighting, based on a likelihood determined based on a number of shadow values in each bin of the shadow histogram and a sum of entries in a corresponding bin of the non-shadow histogram and the shadow histogram, the shadow value; and determining, based on the weighted shadow value, a weighted non-shadow value.

5. The non-transitory machine-readable medium of claim 4 , wherein weighting the values of the total shadow value include identifying how many pixels in the non-shadow histogram are in a bin corresponding to the value of that pixel and dividing by a total number of pixels in the bin of the non-shadow histogram and a corresponding bin the shadow histogram.

6. The non-transitory machine-readable medium of claim 4 , wherein weighting the values of the total non-shadow value include identifying how many pixels in the shadow histogram are in a bin corresponding to the value of that pixel and dividing by a total number of pixels in the bin of the non-shadow histogram and a corresponding bin the shadow histogram.

7. The non-transitory machine-readable medium of claim 1 , wherein the operations further include: projecting the image data to a solar coordinate system in which a direction of solar rays from the sun are perpendicular to columns of pixels of the image data; and projecting the elevation data to the solar coordinate system.

8. The non-transitory machine-readable medium of claim 7 , wherein determining whether each pixel of the image is a shadow or a non-shadow to create a shadow mask of the image occurs in the solar coordinate system and the operations further include projecting the shadow mask to an image coordinate system accounting for an orientation of a camera relative to the geographic location.

9. A method for generating a refined shadow mask, the method comprising: determining, based on elevation data of a geographic region corresponding to a location at which an image was captured and a solar elevation angle at a time the image was captured, whether each pixel of the image is a shadow or a non-shadow to create a shadow mask of the image; generating an eroded shadow mask that includes the shadow mask with a specified number of pixels from a perimeter of each shadow in the shadow mask changed to respective values corresponding to non-shadows; generating a dilated shadow mask that includes the specified number of pixels in the shadow mask changed to values corresponding to shadows; and refining the shadow mask using the eroded shadow mask and the dilated shadow mask to create a refined shadow mask including identifying a pixel in the dilated shadow mask that includes a value different from a corresponding pixel in the eroded shadow mask, detei mine, for each identified pixel, a mean shadow value based on pixels in a window of pixels centered on the identified pixel that include a shadow mask value less than a shadow threshold, determine, for each identified pixel, a mean non-shadow value based on pixels in the window that include a shadow mask value greater than a non-shadow threshold and update the value of the shadow mask pixel based on the intensity of that pixel and the determined mean shadow value and mean non-shadow value.

10. The method of claim 9 , further comprising: generating a shadow histogram of non-shadow values in the dilated shadow mask; and generating a non-shadow histogram of shadow values in the eroded shadow mask.

11. The method of claim 10 , further comprising: weighting, based on a likelihood determined based on the first and non-shadow histogram, shadow values; and weighting, based on a likelihood determined based on the first and non-shadow histogram, non-shadow values.

12. The method of claim 11 , wherein weighting the values of the total shadow value include identifying how many pixels in the non-shadow histogram are in a bin corresponding to the value of that pixel and dividing by a total number of pixels in the bin of the non-shadow histogram and a corresponding bin the shadow histogram.

13. The method of claim 11 , wherein weighting the values of the total non-shadow value include identifying how many pixels in the shadow histogram are in a bin corresponding to the value of that pixel and dividing by a total number of pixels in the bin of the non-shadow histogram and a corresponding bin of the shadow histogram.

14. The method of claim 9 , further comprising: projecting the image data to a solar coordinate system in which a direction of solar rays from the sun are perpendicular to columns of pixels of the image data; and projecting the elevation data to the solar coordinate system.

15. The method of claim 14 , wherein determining whether each pixel of the image is a shadow or a non-shadow to create a shadow mask of the image occurs in the solar coordinate system and the operations further include projecting the shadow mask to an image coordinate system accounting for an orientation of a camera relative to the geographic location.

16. A system comprising: processing circuitry; a memory including program instructions that, when executed the processing circuitry, configure the processing circuitry to: determine, based on elevation data of a geographic region corresponding to a location at which an image was captured and a solar elevation angle at a time the image was captured, whether each pixel of the image is a shadow or a non-shadow to create a shadow mask of the image; generate an eroded shadow mask that includes the shadow mask with a specified number of pixels from a perimeter of each shadow in the shadow mask changed to respective values corresponding to non-shadows; generate a dilated shadow mask that includes the specified number of pixels in the shadow mask changed to values corresponding to shadows; generate a shadow histogram of non-shadow values in the dilated shadow mask; generate a non-shadow histogram of shadow values in the eroded shadow mask; and refine the shadow mask using the eroded shadow mask, the dilated shadow mask, the shadow histogram, and the non-shadow histogram including identifying a pixel in the dilated shadow mask that includes a value different from a corresponding pixel in the eroded shadow mask, determine, for each identified pixel, a mean shadow value based on pixels in a window of pixels centered on the identified pixel that include a shadow mask value less than a shadow threshold, determine, for each identified pixel, a mean non-shadow value based on pixels in the window that include a shadow mask value greater than a non-shadow threshold, and update the value of the shadow mask pixel based on the intensity of that pixel and the determined mean shadow value and mean non-shadow value.

17. The system of claim 16 , wherein the program instructions further configure the processing circuitry to: project the image data to a solar coordinate system in which a direction of solar rays from the sun are perpendicular to columns of pixels of the image data; and project the elevation data to the solar coordinate system.