An enhanced fluorescence imaging system includes a light source for emitting non-visible and visible light and an image sensor for capturing non-visible and visible light image data. Data processing hardware performs operations that include determining a non-visible value associated with an amount of non-visible light captured by the image sensor and applying a color map to each non-visible value to generate non-visible light selected color values. The operations also include weighting a visible light chroma value with a non-visible chroma value to generate weighted chroma values and combining luma values of each pixel of the visible light image data to the weighted chroma values. The operations also include generating RGB values based on the luma values of the visible light image data and the weighted chroma values and transmitting the RGB values to the display.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A fluorescence imaging system comprising a light source configured to emit non-visible light and visible light and an image sensor including a plurality of pixels configured to capture non-visible light image data and visible light image data, the fluorescence imaging system configured to generate a video image onto a display, the fluorescence imaging system comprising: data processing hardware in communication with the image sensor; and memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising: determining, for each pixel of the plurality of pixels, a non-visible light value, the non-visible light value associated with an amount of non-visible light captured by the image sensor; applying a color map to each non-visible light value to associate the non-visible light value with a select color to generate non-visible light selected color values; converting, for each pixel, the associated visible light image data into a visible light luma value, a visible light blue-difference chroma value, and a visible light red-difference chroma value; converting the non-visible light selected color values into a non-visible light luma value, a non-visible light blue-difference chroma value, and a non-visible light red-difference chroma value; weighting the visible light blue-difference chroma value and the non-visible light blue-difference chroma value based on a weighting factor to generate a weighted blue-difference chroma value; weighting the visible light red-difference chroma value and the non-visible light red-difference chroma value based on the weighting factor to generate a weighted red-difference chroma value; and combining luma values of each pixel of the visible light image data to the weighted blue-difference and red-difference chroma values so as to enhance the video image.
2. The fluorescence imaging system of claim 1 , wherein the color map is configured to transform each non-visible light value to a select color, the magnitude of the non-visible light value associated with an intensity of the select color.
3. The fluorescence imaging system of claim 1 , wherein the select color is green.
4. The fluorescence imaging system of claim 1 , wherein the weighting factor is based on the non-visible light value for the associated pixel.
5. The fluorescence imaging system of claim 4 , wherein when the non-visible light value is below a first threshold value, the weighted blue-difference chroma value is equal to the visible light blue-difference chroma value and the weighted red-difference chroma value is equal to the visible light red-difference chroma value.
6. The fluorescence imaging system of claim 5 , wherein when the non-visible light value is above a second threshold value, the weighted blue-difference chroma value is equal to the non-visible light blue-difference chroma value and the weighted red-difference chroma value is equal to the non-visible light red-difference chroma value.
7. The fluorescence imaging system of claim 6 , wherein when the non-visible light value is between the first threshold value and the second threshold value, the weighted blue-difference chroma value is between the visible light blue-difference chroma value and the non-visible light blue-difference chroma value and the weighted red-difference chroma value is in between the visible light red-difference chroma value and the non-visible light red-difference chroma value.
8. The fluorescence imaging system of claim 1 , wherein each non-visible light value is between a non-visible light minimum value and a non-visible light maximum value based on a pixel bit depth.
9. A fluorescence imaging system including a light source configured to emit non-visible light and visible light and an image sensor comprising a plurality of pixels configured to capture non-visible light image data and visible light image data, the fluorescence imaging system configured to generate a video image onto a display, the fluorescence imaging system comprising: data processing hardware in communication with the image sensor; and memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising: determining, for each pixel of the plurality of pixels, a non-visible light value, the non-visible light value associated with an amount of non-visible light captured by the image sensor; adding the non-visible light value of each pixel to a select color of the visible light image data to generate added color light values; applying a color map to each non-visible light value to associate the non-visible light value with a select color to generate a non-visible light selected color value; weighting a visible light chroma value of the visible light image data with a non-visible light chroma value of the non-visible light selected color values to generate weighted chroma values; combining luma values of the visible light image data to the weighted chroma values to generate combined luma-chroma light values; converting the combined luma-chroma light values into replaced color light values; and weighting the added color light values with replaced color light values to generate weighted color light values so as to enhance the video image.
10. The fluorescence imaging system of claim 9 , wherein adding the non-visible light value to a select color of pixels of the visible light image data includes: determining, for each pixel, a visible RGB value; determining, for each pixel, a non-visible light RGB value based on the non-visible light image data; and adding, for each pixel, the non-visible light RGB value the visible RGB value.
11. The fluorescence imaging system of claim 10 , wherein weighting the added color light values with the replaced color light values comprises weighting based on a weighting factor, the weighting factor based on the non-visible light value of each associated pixel.
12. The fluorescence imaging system of claim 11 , wherein the weighting factor is based on the non-visible light value.
13. The fluorescence imaging system of claim 11 , wherein the weighting factor is based on the added color light values.
14. The fluorescence imaging system of claim 11 , wherein the weighting factor is based on a bit depth of the image sensor.
15. The fluorescence imaging system of claim 10 , wherein the weight of each added color light value is inversely correlated with the associated non-visible light value.
16. A method for weighting image values of a fluorescence imaging system comprising a light source configured to emit non-visible light and visible light and an image sensor comprising a plurality of pixels configured to capture non-visible light image data and visible light image data, the fluorescence imaging system configured to generate a video image onto a display, the method comprising: determining, for each pixel of the plurality of pixels, a non-visible light value, the non-visible light value associated with an amount of non-visible light captured by the image sensor; adding the non-visible light value of each pixel to a select color of the visible light image data to generate added color light values; applying a color map to each non-visible light value to associate the non-visible light value with a select color to generate a non-visible light selected color value; weighting a visible light chroma value of the visible light image data with a non-visible light chroma value of the non-visible light selected color values to generate a weighted chroma value; combining luma values of the visible light image data to the weighted chroma values to generate combined luma-chroma light values; converting the combined luma-chroma light values into replaced color light values; and weighting the added color light values with replaced color light values to generate weighted color light values.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
December 11, 2019
March 16, 2021
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.