Attention-Based Multiple Instance Learning

1. A computer-implemented method for using machine learning to process digital pathology images to predict disease progression, the method comprising: accessing a digital pathology image that depicts a specimen stained with one or more stains, the specimen having been collected from a subject; defining a set of patches for the digital pathology image, wherein each patch of the set of patches depicts a portion of the digital pathology image; generating, for each patch of the set of patches and using an attention-score neural network, an attention score, wherein the attention-score neural network is trained using a loss function, the loss function penalizes attention-score variability across patches in training digital pathology images, the training digital pathology images labeled to indicate subsequent disease progression has occurred; generating, using a result-prediction neural network and the attention scores, a result representing a prediction of whether or an extent to which a disease of the subject will progress; and outputting the result.

2. The computer-implemented method of claim 1, further comprising: generating, for each patch of the set of patches and using a feature-vector neural network, a feature vector for the patch, wherein the result further depends on the feature vectors for the set of patches.

3. The computer-implemented method of claim 2, wherein the result is an image-based output generated based on the feature vectors and the attention scores for the set of patches.

4. The computer-implemented method of claim 2, wherein the generating the result includes: generating a cross-patch feature vector using the feature vectors and the attention scores for the set of patches; and generating the result by processing the cross-patch feature vector using the result-prediction neural network.

5. The computer-implemented method of claim 2, wherein the feature vectors for the set of patches represent cell nuclei regions, wherein the generating the result further comprises: performing nuclei detection and segmentation to segment the set of patches into cell nuclei and non-cell nuclei regions; performing nuclei classification to identify individual cell nucleus from a nuclei segmentation mask; calculating cellular features from the set of patches and the nuclei segmentation mask; and calculating one or more patch-level metrics to form a patch-level representation, wherein the one or more patch-level metrics represent feature distribution of the cell nuclei regions.

6. The computer-implemented method of claim 2, wherein the feature-vector neural network includes a convolutional neural network.

7. The computer-implemented method of claim 1, wherein the loss function is configured to depend on multiple terms, wherein at least one of the multiple terms depends on an accuracy of the prediction and a second term defined.

8. The computer-implemented method of claim 1, wherein the loss function is defined using a K-L divergence technique.

9. The computer-implemented method of claim 1, wherein the loss function is configured such that a penalty depends on a degree of non-uniformity across the attention scores generated for the patches in the training digital pathology images labeled to indicate subsequent disease progression has occurred.

10. The computer-implemented method of claim 1, wherein the attention-score neural network includes a perceptron neural network.

11. The computer-implemented method of claim 1, wherein the attention-score neural network is trained using a training data set in which at least 90% of the training digital pathology images were labeled to indicate subsequent disease progression has occurred.

12. The computer-implemented method of claim 1, wherein the result represents a high likelihood of the disease of the subject progressing by at least a predefined threshold amount within a predefined period of time.

13. The computer-implemented method of claim 1, wherein the result further includes an identification of a subset of the set of patches that were more influential than other patches in the set of patches in the generation of result.

14. The computer-implemented method of claim 1, wherein the loss function further penalizes a lack of cross-portion variation in the attention scores in the training digital pathology images associated with no disease progression.

15. The computer-implemented method of claim 1, wherein the disease is at least one of: diffuse B-cell lymphoma, follicular lymphoma, chronic lymphocytic leukemia, small lymphocytic lymphoma, acute myeloid leukemia, or breast cancer.

16. A system for using machine learning to process digital pathology images to predict disease progression, the system comprising one or more data processors, memory, and one or more programs stored in the memory for execution by the one or more processors and including instructions for: accessing a digital pathology image that depicts a specimen stained with one or more stains, the specimen having been collected from a subject; defining a set of patches for the digital pathology image, wherein each patch of the set of patches depicts a portion of the digital pathology image; generating, for each patch of the set of patches and using an attention-score neural network, an attention score, wherein the attention-score neural network is trained using a loss function, the loss function penalizes attention-score variability across patches in training digital pathology images, the training digital pathology images labeled to indicate subsequent disease progression has occurred; generating, using a result-prediction neural network and the attention scores, a result representing a prediction of whether or an extent to which a disease of the subject will progress; and outputting the result.

17. The system of claim 16, wherein the one or more programs include further instructions for generating, for each patch of the set of patches and using a feature-vector neural network, a feature vector for the patch, wherein the result further depends on the feature vectors for the set of patches.

18. The system of claim 17, wherein the result is an image-based output generated based on the feature vectors and the attention scores for the set of patches.

19. The system of claim 17, wherein the generating the result includes: generating a cross-patch feature vector using the feature vectors and the attention scores for the set of patches; and generating the result by processing the cross-patch feature vector using the result-prediction neural network.

20. The system of claim 17, wherein the feature vectors for the set of patches represent cell nuclei regions, wherein the generating the result further comprises: performing nuclei detection and segmentation to segment the set of patches into cell nuclei and non-cell nuclei regions; performing nuclei classification to identify individual cell nucleus from a nuclei segmentation mask; calculating cellular features from the set of patches and the nuclei segmentation mask; and calculating one or more patch-level metrics to form a patch-level representation, wherein the one or more patch-level metrics represent feature distribution of the cell nuclei regions.

21. The system of claim 16, wherein the feature-vector neural network includes a convolutional neural network.

22. The system of claim 16, wherein the loss function is configured to depend on multiple terms, wherein at least one of the multiple terms depends on an accuracy of the prediction and a second term defined.

23. The system of claim 16, wherein the loss function is defined using a K-L divergence technique.

24. The system of claim 16, wherein the loss function is configured such that a penalty depends on a degree of non-uniformity across the attention scores generated for the patches in the training digital pathology images labeled to indicate subsequent disease progression has occurred.

25. The system of claim 16, wherein the attention-score neural network is trained using a training data set in which at least 90% of the training digital pathology images were labeled to indicate subsequent disease progression has occurred.

26. The system of claim 16, wherein the result represents a high likelihood of the disease of the subject progressing by at least a predefined threshold amount within a predefined period of time.

27. The system of claim 16, wherein the result further includes an identification of a subset of the set of patches that were more influential than other patches in the set of patches in the generation of result.

28. The system of claim 16, wherein the loss function further penalizes a lack of cross-portion variation in the attention scores in the training digital pathology images associated with no disease progression.

29. The system of claim 16, wherein the disease is at least one of: diffuse B-cell lymphoma, follicular lymphoma, chronic lymphocytic leukemia, small lymphocytic lymphoma, acute myeloid leukemia, or breast cancer.

30. A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause one or more data processors to perform: accessing a digital pathology image that depicts a specimen stained with one or more stains, the specimen having been collected from a subject; defining a set of patches for the digital pathology image, wherein each patch of the set of patches depicts a portion of the digital pathology image; generating, for each patch of the set of patches and using an attention-score neural network, an attention score, wherein the attention-score neural network is trained using a loss function, the loss function penalizes attention-score variability across patches in training digital pathology images, the training digital pathology images labeled to indicate subsequent disease progression has occurred; generating, using a result-prediction neural network and the attention scores, a result representing a prediction of whether or an extent to which a disease of the subject will progress; and outputting the result.