Generating a High Dynamic Range Image Using a Temporal Filter

1. An apparatus, comprising: a camera configured to capture a plurality of image frames suitable for generation of a high dynamic range (HDR) image frame; an image signal processor comprising a temporal filter module, wherein the image signal processor is configured to: receive a current image frame for filtering at the temporal filter module, wherein the current image frame is one of the plurality of image frames captured by the camera; receive a reference image frame for the temporal filter module that comprises image data of one or more different image frames of the plurality of image frames captured by the camera, the reference image frame having a longer exposure than the current image frame; generate a normalized current image frame by applying digital gain to the current image frame in response to the longer exposure of the reference image frame relevant to the exposure of the current image frame; generate a filtered version of the current image frame according to an HDR blending scheme for the plurality of images that blends the reference image frame with the normalized current image frame; determine that the current image frame is last of the plurality of image frames to be filtered; and in response to the determination that the current image frame is the last of the plurality of image frames to be filtered, provide the filtered version of the current image frame as the HDR image frame.

2. The apparatus of claim 1 , wherein to generate the filtered version of the current image frame, the image signal processor is configured to: determine a motion estimate for the reference image frame with regard to the current image frame; and align the reference image frame to the current image frame according to the determined motion estimate.

3. The apparatus of claim 1 , wherein to generate the filtered version of the current image frame, the image signal processor is configured to: determine a respective filter weight for each pixel in the current image frame, wherein the determination comprises application of one or more spatial filtering techniques to determine a difference between a given pixel of the current image frame and a corresponding pixel of the reference image frame; and blend each pixel in the current image frame with the corresponding pixel in the reference image frame according to the respective filter weight for the pixel.

4. The apparatus of claim 1 , wherein the apparatus comprises a mobile computing device.

5. A method, comprising: receiving, by an image signal processor, a current image frame for filtering at a temporal filter module, wherein the current image frame is one of plurality of image frames captured to generate a high dynamic range (HDR) image frame; obtaining, by the image signal processor, a reference image frame for the temporal filter module that comprises image data of one or more different image frames of the plurality of image frames captured to generate the HDR image frame, the reference image frame having a longer exposure relative to the exposure of the current image frame; generating a normalized current image frame by applying digital gain to the current image frame in response to the longer exposure of the reference image frame relevant to the exposure of the current image frame; generating, by the image signal processor, a filtered version of the current image frame according to an HDR blending scheme for the plurality of images that blends the reference image frame with the normalized current image frame; determining, by the image signal processor, that the current image frame is last of the plurality of image frames to be filtered; and in response to determining that the current image frame is the last of the plurality of image frames to be filtered, providing, by the image signal processor, the filtered version of the current image frame as the HDR image frame.

6. The method of claim 5 , wherein the generating the filtered version of the current image frame comprises applying one or more resealing techniques to least one of the current image frame and the reference image frame.

7. The method of claim 5 , wherein generating the filtered version of the current image frame comprises: determining a motion estimate for the reference image frame to register the reference image frame with regard to the current image frame; and aligning the reference image frame to the current image frame according to the determined motion estimate.

8. The method of claim 5 , wherein generating the filtered version of the current image frame comprises: determining a respective filter weight for each pixel in the current image frame, wherein the determining comprises applying one or more spatial filtering techniques to determine a difference between a given pixel of the current image frame and a corresponding pixel of the reference image frame; and blending each pixel in the current image frame with the corresponding pixel in the reference image frame according to the respective filter weight for the pixel.

9. The method of claim 5 , wherein generating the filtered version of the current image frame further comprises: modifying the respective filter weight determined for each pixel in the current image frame according to a respective noise history maintained for the corresponding pixel of the reference image frame, wherein each pixel of the current image frame is blended according to the modified respective filter weight.

10. The method of claim 5 , wherein the one or more different image frames includes at least one previously filtered image frame of the plurality of image frames captured to generate the HDR image frame, and wherein the receiving, the obtaining, and the generating were performed with respect to the at least one previously filtered image frame.

11. The method of claim 5 , wherein the plurality of image frames that are captured to generate the HDR image frame are part of a stream of image frames for generating different respective HDR image frames such that a video stream of HDR image frames is provided by the image signal processor.

12. The method of claim 5 , wherein the image signal processor is implemented as part of a mobile computing device.

13. A system, comprising: a device configured to perform image processing, the device comprising: a sensor interface configured to receive image data from an image sensor; and an image processing pipeline comprising a temporal filter module, the image processing pipeline configured to: receive a current image frame for filtering at the temporal filter module, wherein the current image frame is one of plurality of image frames captured to generate a high dynamic range (HDR) image frame; obtain a reference image frame for filtering at the temporal filter module, wherein the reference image frame comprises image data of one or more different image frames of the plurality of image frames captured to generate the HDR image frame, the reference image frame having a longer exposure relative to the exposure of the current image frame; generate a normalized current image frame by applying digital gain to the current image frame in response to the longer exposure of the reference image frame relevant to the exposure of the current image frame; generate a filtered version of the current image frame according to an HDR blending scheme for the plurality of images that blends the reference image frame with the normalized current image frame; determine that the current image frame is last of the plurality of image frames to be filtered; and in response to the determination that the current image frame is the last of the plurality of image frames to be filtered, provide the filtered version of the current image frame as the HDR image frame.

14. The system of claim 13 , wherein to generate the filtered version of the current image frame, the image processing pipeline is configured to apply one or more scaling algorithms to at least one of current image frame and the reference image frame.

15. The system of claim 13 , wherein to generate the filtered version of the current image frame, the image processing pipeline is configured to: determine a motion estimate for the reference image frame to register the reference image frame with regard to the current image frame; and align the reference image frame to the current image frame according to the determined motion estimate.

16. The system of claim 13 , wherein to generate the filtered version of the current image frame, the image processing pipeline is configured to: determine a respective filter weight for each pixel in the current image frame, wherein the determination comprises application of one or more spatial filtering techniques to determine a difference between a given pixel of the current image frame and a corresponding pixel of the reference image frame; and blend each pixel in the current image frame with the corresponding pixel in the reference image frame according to the respective filter weight for the pixel.

17. The system of claim 16 , wherein the current image frame and the reference image frame are formatted according to a Bayer image filter.

18. The system of claim 13 , wherein the one or more different image frames includes at least one image frame of the plurality of image frames captured to generate the HDR image frame that was not filtered at the temporal filter module.

19. The system of claim 13 , wherein the system is a mobile computing device, and wherein the device is an image signal processor.

20. The apparatus of claim 1 , wherein generating a filtered version of the current image frame according to an HDR blending scheme comprises: for one or more particular pixel locations of the normalized current image frame: if a brightness value of the particular pixel location in the reference image frame is greater than or equal to a clipping threshold, setting the brightness of the particular pixel location in the filtered version of the current image frame to be equal to the brightness of the particular pixel location in the normalized current image frame; if a brightness value of the particular pixel location in the normalized current image frame is lower than or equal to a noise threshold, setting the brightness of the particular pixel location in the filtered version of the current image frame to be equal to the brightness of the particular pixel location in the reference image frame; if the brightness value of the particular pixel location in the reference image frame is less than the clipping threshold and the brightness value of the particular pixel location in the normalized current image frame is greater than the noise threshold: calculating an average pixel brightness of the particular pixel location by averaging the brightness of the particular pixel location in the normalized current image frame with the brightness of the particular pixel location in the reference image frame; and setting the brightness of the particular pixel location in the filtered version of the current image frame using a weighted blend of the brightness value of the particular pixel location in the normalized current image frame and the brightness value of the particular pixel location in the reference image frame, wherein the weighting is determined at least in part in response to the calculated average pixel brightness of the particular pixel location.