Processing Fundus Images Using Machine Learning Models to Generate Blood-Related Predictions

1. A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations comprising: obtaining a fundus image, the fundus image being an image of a fundus of an eye of a patient; and processing a model input comprising the fundus image using a fundus image processing neural network, wherein the fundus image processing neural network is configured to process the model input comprising the fundus image to generate a model output that predicts a property of the blood of the patient; and generating, from at least the model output, a final prediction of the property of the blood of the patient.

2. The system of claim 1 , wherein the model input further comprises other patient data comprising patient demographic data.

3. The system of claim 2 , wherein the fundus image processing neural network comprises a convolutional neural network and one or more fully-connected layers, and wherein the fundus image processing neural network is configured to: process the fundus image in the model input through the convolutional neural network to generate a convolutional output; and process the convolutional output and (i) the other patient data or (ii) data derived from the other patient data through the fully-connected layers to generate the model output.

4. The system of claim 1 , wherein the fundus image processing neural network comprises a convolutional neural network and one or more fully-connected layers, and wherein the fundus image processing neural network is configured to: process the fundus image in the model input through the convolutional neural network to generate a convolutional output; and process the convolutional output through the fully-connected layers to generate the model output.

5. The system of claim 1 , wherein the model output generated by the fundus image processing neural network by processing the model input comprising the fundus image comprises a predicted level of hemoglobin in the blood of the patient.

6. The system of claim 1 , wherein the model output generated by the fundus image processing neural network by processing the model input comprising the fundus image comprises a predicted hematocrit of the blood of the patient.

7. The system of claim 1 , wherein the model output generated by the fundus image processing neural network by processing the model input comprising the fundus image comprises a predicted red blood cell count of the blood of the patient.

8. The system of claim 1 , wherein generating, from at least the model output, a final prediction of the property of the blood of the patient comprises: obtaining another fundus image, the other fundus image being an image of a fundus of the other eye of the patient; and processing another model input comprising the other fundus image using the fundus image processing neural network to generate another model output that predicts a property of the blood of the patient; and generating the final prediction from at least the model output and the other model output.

9. The system of claim 8 , wherein the fundus image processing neural network is one of a plurality of neural networks in an ensemble of neural networks, and wherein generating, from at least the model output, a final prediction of the property of the blood of the patient comprises: processing the model input and the other model input using each other neural network in the ensemble to generate a respective model output and other model output for each of the other neural networks in the ensemble; and generating the final prediction by combining the model outputs and the other model outputs generated by all of the neural networks in the ensemble.

10. The system of claim 1 , wherein the fundus image processing neural network is one of a plurality of neural networks in an ensemble of neural networks, and wherein generating, from at least the model output, a final prediction of the property of the blood of the patient comprises: processing the model input using each other neural network in the ensemble to generate a respective model output for each of the other neural networks in the ensemble; and generating the final prediction by combining the model outputs generated by all of the neural networks in the ensemble.

11. A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations comprising: obtaining a fundus image, the fundus image being an image of a fundus of an eye of a patient; and processing a model input comprising the fundus image using a fundus image processing neural network, wherein the fundus image processing neural network is configured to process the model input comprising the fundus image to generate a model output that predicts a current condition of the patient with respect to anemia; and generating, from at least the model output, a final prediction of the current condition with respect to anemia.

12. The system of claim 11 , wherein the model input further comprises other patient data comprising patient demographic data.

13. The system of claim 12 , wherein the fundus image processing neural network comprises a convolutional neural network and one or more fully-connected layers, and wherein the fundus image processing neural network is configured to: process the fundus image in the model input through the convolutional neural network to generate a convolutional output; and process the convolutional output and (i) the other patient data or (ii) data derived from the other patient data through the fully-connected layers to generate the model output.

14. The system of claim 11 , wherein the fundus image processing neural network comprises a convolutional neural network and one or more fully-connected layers, and wherein the fundus image processing neural network is configured to: process the fundus image in the model input through the convolutional neural network to generate a convolutional output; and process the convolutional output through the fully-connected layers to generate the model output.

15. The system of claim 11 , wherein the model output comprises a probability that the patient has anemia.

16. The system of claim 11 , wherein the model output comprises a respective probability for each of multiple anemia classifications, and wherein the probability for a given disease state represents the predicted likelihood that the patient has the corresponding anemia classification.

17. The system of claim 16 , wherein the anemia classifications comprise one or more of: anemia, moderate anemia, or approximate anemia.

18. The system of claim 11 , wherein generating, from at least the model output, a final prediction comprises: obtaining another fundus image, the other fundus image being an image of a fundus of the other eye of the patient; and processing another model input comprising the other fundus image using the fundus image processing neural network to generate another model output; and generating the final prediction from at least the model output and the other model output.

19. The system of claim 18 , wherein the fundus image processing neural network is one of a plurality of neural networks in an ensemble of neural networks, and wherein generating, from at least the model output, a final prediction comprises: processing the model input and the other model input using each other neural network in the ensemble to generate a respective model output and other model output for each of the other neural networks in the ensemble; and generating the final prediction by combining the model outputs and the other model outputs generated by all of the neural networks in the ensemble.

20. The system of claim 11 , wherein the fundus image processing neural network is one of a plurality of neural networks in an ensemble of neural networks, and wherein generating, from at least the model output, a final prediction of the property of the blood of the patient comprises: processing the model input using each other neural network in the ensemble to generate a respective model output for each of the other neural networks in the ensemble; and generating the final prediction by combining the model outputs generated by all of the neural networks in the ensemble.

21. A method comprising: obtaining a fundus image, the fundus image being an image of a fundus of an eye of a patient; and processing a model input comprising the fundus image using a fundus image processing neural network, wherein the fundus image processing neural network is configured to process the model input comprising the fundus image to generate a model output that predicts a current condition of the patient with respect to anemia; and generating, from at least the model output, a final prediction of the current condition with respect to anemia.