Control and Monitoring of Physical System Based on Trained Bayesian Neural Network

1. A training system for training a neural network for use as a trained model in controlling or monitoring a robot or vehicle operating in an environment, wherein the model is trained to infer a state of the environment or the robot or vehicle based on one or more video or image sensor measurements and to provide a quantification of its inference uncertainty during use, the training system comprising: an input interface configured to access model data defining a Bayesian neural network, and training data including video or image sensor measurements and associated states of the environment or the robot or vehicle; a processor subsystem configured to train the Bayesian neural network based on the training data by: recursively integrating out weights of the Bayesian neural network from an input layer to an output layer of the Bayesian neural network to obtain a marginal likelihood function of the Bayesian neural network; and maximizing the marginal likelihood function to tune hyperparameters of priors of the integrated-out weights of the Bayesian neural network so as to obtain a trained Bayesian neural network, wherein the marginal likelihood function is maximized by minimizing a loss function which includes: i) a data fit term expressing a fit to the training data, and ii) a regularization term; and an output interface configured to output trained model data representing the trained Bayesian neural network.

2. The training system according to claim 1 , wherein the processor subsystem is configured to recursively integrate out the weights of the Bayesian neural network using progressive moment matching based on a first moment and a second moment of the respective probability distribution of each of the weights.

3. The training system according to claim 1 , wherein: the input interface is configured to access prior knowledge data representing a prior knowledge-based model of a relation between the video or image sensor measurements and the associated states; and the processor subsystem is configured to train the model to incorporate prior knowledge of the robot or the vehicle by incorporating the prior knowledge-based model in the regularization term of the loss function.

4. The training system according to claim 3 , wherein the prior knowledge-based model includes a modelling of a physical law which at least in part defines a relation between the video or image sensor measurements and the associated states.

5. The training system according to claim 3 , wherein the prior knowledge-based model is a probabilistic model.

6. The training system according to claim 1 , wherein the regularization term is a probably approximately correct (PAC) bound.

7. The training system according to claim 1 , wherein the regularization term penalizes complexity of the Bayesian neural network.

8. An inference system for, using a trained model, controlling or monitoring a robot or vehicle operating in an environment, wherein the model is trained to infer a state of the environment or the robot or vehicle based on one or more video or image sensor measurements and to provide a quantification of an inference uncertainty of the inference, the inference system comprising: an input interface configured to access input data representing at least video or image sensor measurement, and model data defining a trained Bayesian neural network as obtained by the training system; an output interface to an output device which is used in the control or the monitoring of the robot or vehicle; a processor subsystem configured to: use the input data as input to the trained Bayesian neural network to obtain an inference of a state of the environment or the robot or vehicle and a quantification of an inference uncertainty of the inference, and based on the inference and the quantification of the inference uncertainty of the inference, generate output data for the output device for controlling or monitoring the robot or vehicle; wherein the inference system is further configured to, using the output interface, provide the output data to the output device.

9. The interference system as recited in claim 8 , wherein the Bayesian neural network is trained by: recursively integrating out weights of the Bayesian neural network from an input layer to an output layer of the Bayesian neural network to obtain a marginal likelihood function of the Bayesian neural network; and maximizing the marginal likelihood function to tune hyperparameters of priors of the integrated-out weights of the Bayesian neural network so as to obtain a trained Bayesian neural network, wherein the marginal likelihood function is maximized by minimizing a loss function which includes: i) a data fit term expressing a fit to the training data, and ii) a regularization term.

10. The inference system according to claim 8 , wherein the output device is an actuator associated with the robot or vehicle, and wherein the inference system is an inference-based control system configured to control the robot or vehicle by providing control data as the output data to the actuator.

11. The inference system according to claim 8 , wherein the output device is a rendering device, wherein the inference system is an inference-based monitoring system configured to monitor an operation of the robot or vehicle and to generate and provide the output data to the rendering device to cause the rendering device to generate a sensory perceptible output signal.

12. A computer-implemented training method for training a neural network for use as a trained model in controlling or monitoring a robot or vehicle operating in an environment, wherein the model is trained to infer a state of the environment or the robot or vehicle based on one or more video or image sensor measurements and to provide a quantification of its inference uncertainty during use, the training method comprising the following steps: accessing, via an input interface, model data defining a Bayesian neural network, and training data including video or image sensor measurements and associated states of the environment or the robot or vehicle; training the Bayesian neural network based on the training data by: recursively integrating out weights of the Bayesian neural network from an input layer to an output layer of the Bayesian neural network to obtain a marginal likelihood function of the Bayesian neural network, and maximizing the marginal likelihood function to tune hyperparameters of priors of the integrated-out weights of the Bayesian neural network so as to obtain a trained Bayesian neural network, wherein the marginal likelihood function is maximized by minimizing a loss function which includes: i) a data fit term expressing a fit to the training data, and ii) a regularization term; and outputting, via an output interface, trained model data representing the trained Bayesian neural network.

13. A computer-implemented inference method for, using a trained model, controlling or monitoring a robot or vehicle operating in an environment, wherein the model is trained to infer a state of the environment or the robot or vehicle based on one or more video or image sensor measurements and to provide a quantification of an inference uncertainty of the inference, the inference method comprising the following steps: accessing, via an input interface, input data representing at least one video or image sensor measurement, and model data defining a trained Bayesian neural network; using the input data as input to the trained Bayesian neural network to obtain an inference of a state of the environment or the robot or vehicle and a quantification of an inference uncertainty of the inference; based on the inference and the quantification of the inference uncertainty of the inference, generating output data for an output device which is used in the control or the monitoring of the robot or vehicle; and via an output interface, providing the output data to the output device.

14. The computer-implemented inference method as recited in claim 13 , wherein the Bayesian neural network is trained by: recursively integrating out weights of the Bayesian neural network from an input layer to an output layer of the Bayesian neural network to obtain a marginal likelihood function of the Bayesian neural network; and maximizing the marginal likelihood function to tune hyperparameters of priors of the integrated-out weights of the Bayesian neural network so as to obtain a trained Bayesian neural network, wherein the marginal likelihood function is maximized by minimizing a loss function which includes: i) a data fit term expressing a fit to the training data, and ii) a regularization term.

15. A non-transitory computer-readable medium on which is stored instructions for training a neural network for use as a trained model in controlling or monitoring a robot or vehicle operating in an environment, wherein the model is trained to infer a state of the environment or the robot or vehicle based on one or more video or image sensor measurements and to provide a quantification of its inference uncertainty during use, the instructions, when executed by a processor system, causing the processor system to perform the following steps: accessing, via an input interface, model data defining a Bayesian neural network, and training data including video or image sensor measurements and associated states of the environment or the robot or vehicle; training the Bayesian neural network based on the training data by: recursively integrating out weights of the Bayesian neural network from an input layer to an output layer of the Bayesian neural network to obtain a marginal likelihood function of the Bayesian neural network, and maximizing the marginal likelihood function to tune hyperparameters of priors of the integrated-out weights of the Bayesian neural network so as to obtain a trained Bayesian neural network, wherein the marginal likelihood function is maximized by minimizing a loss function which includes: i) a data fit term expressing a fit to the training data, and ii) a regularization term; and outputting, via an output interface, trained model data representing the trained Bayesian neural network.