System and Method for Adverse Event Detection or Severity Estimation from Surgical Data

2. The system of claim 1, wherein the set of audio or video data includes a set of video frames that have been stabilized to reduce camera motion through the use of bundled-camera path stabilization that reduces jitter and smooths camera paths so that the latent features are accumulated across a plurality of frames.

3. The system of claim 2, wherein stabilization includes warping images to align each frame's camera view based at least on homography.

4. The system of claim 1, wherein the feature extractor neural network is a three dimensional (3D) or two-dimensional (2D) convolutional network.

5. The system of claim 1, wherein the classification tasks include at least bleeding and thermal injury detection, and wherein the classification tasks are causally distinct and include distinguishing active injury events from prior injury artifacts.

7. The system of claim 1, wherein the processor is configured to receive a set of audio data, and the feature extractor neural network extracts the vector of latent features from a combination of the set of audio data and the set of video data.

8. The system of claim 7, wherein the training data set includes both training video data and training audio data.

10. The method of claim 9, wherein the set of audio or video data includes a set of video frames that have been stabilized to reduce camera motion through the use of bundled-camera path stabilization that reduces jitter and smooths camera paths so that the latent features are accumulated across a plurality of frames.

11. The method of claim 10, wherein stabilization includes warping images to align each frame's camera view based at least on homography.

12. The method of claim 9, wherein the feature extractor neural network is a three dimensional (3D) or two-dimensional (2D) convolutional network.

13. The method of claim 9, wherein the classification tasks include at least bleeding and thermal injury detection, and wherein the classification tasks are causally distinct and include distinguishing active injury events from prior injury artifacts.

15. The method of claim 9, the method comprising receiving a set of audio data, and extracting, by the feature extractor neural network, the vector of latent features from a combination of the set of audio data and the set of video data.