Closed-Loop Real Time Ssvep-Based Heads-Up Display to Control in Vehicle Features Using Deep Learning

1. A vehicle system, comprising a controller programmed to: display a plurality of icons on a heads-up-display (HUD) of a vehicle; receive electroencephalography (EEG) data from a driver of the vehicle; perform a Fast Fourier Transform (FFT) of the EEG data to obtain an EEG spectrum; input the EEG spectrum into a trained machine learning model that outputs a prediction of an icon that the driver is looking at based on the EEG spectrum; determine which of the plurality of icons the driver is viewing based on the output of the trained machine learning model; and perform one or more vehicle operations based on the output of the trained machine learning model.

2. The vehicle system of claim 1, wherein each of the plurality of icons has a different color.

3. The vehicle system of claim 1, wherein each of the plurality of icons has a different shape.

4. The vehicle system of claim 1, wherein the controller is further programmed to: apply a band pass filter to the EEG data to obtain filtered EEG data; and perform the FFT of the filtered EEG data.

5. The vehicle system of claim 1, wherein the controller is further programmed to: perform a data segmentation of the EEG data to obtain segmented EEG data; and perform the FFT of the segmented EEG data.

6. The vehicle system of claim 1, wherein the controller is further programmed to: receive training data comprising EEG data collected from a plurality of individual subjects while each subject is viewing specific icons; and train a machine learning model to predict which icon the individual subjects are viewing based on the training data to achieve the trained machine learning model.

7. The vehicle system of claim 1, wherein the trained machine learning model comprises a convolutional neural network.

8. The vehicle system of claim 7, wherein the convolutional neural network comprises a residual neural network architecture.

9. The vehicle system of claim 1, wherein the trained machine learning model comprises one or more squeeze and excite (SE) blocks.

10. The vehicle system of claim 9, wherein at least one of the SE blocks comprises a global max pooling layer, a first fully connected layer having a rectified linear unit activation function, and a second fully connected layer having a sigmoid activation function.

11. The vehicle system of claim 1, wherein the trained machine learning model comprises two SE-Res blocks, wherein each SE-Res block comprises: a two-dimensional convolutional layer; a batch normalization layer; an activation layer; and an SE block.

12. The vehicle system of claim 11, wherein an input to each SE-Res block is summed with an output of the SE-Res block.

13. The vehicle system of claim 11, wherein the trained machine learning model further comprises: a dropout layer; and a Softmax classification layer.

14. A method, comprising: displaying a plurality of icons on a heads-up-display (HUD) of a vehicle; receiving electroencephalography (EEG) data from a driver of the vehicle; performing a Fast Fourier Transform (FFT) of the EEG data to obtain an EEG spectrum; inputting the EEG spectrum into a trained machine learning model that outputs a prediction of an icon that the driver is looking at based on the EEG spectrum; determining which of the plurality of icons the driver is viewing based on the output of the trained machine learning model; and performing one or more vehicle operations based on the output of the trained machine learning model.

15. The method of claim 14, further comprising: apply a band pass filter to the EEG data to obtain filtered EEG data; performing a data segmentation of the filtered EEG data to obtain segmented EEG data; and performing the FFT of the segmented EEG data.

16. The method of claim 14, wherein the trained machine learning model comprises a convolutional neural network comprising: two SE-Res blocks, wherein each SE-Res block comprises: a two-dimensional convolutional layer; a batch normalization layer; an activation layer; and an SE block.

17. The method of claim 16, wherein the SE block comprises a global max pooling layer, a first fully connected layer having a rectified linear unit activation function, and a second fully connected layer having a sigmoid activation function.

18. The method of claim 16, wherein the trained machine learning model further comprises: a dropout layer; and a Softmax classification layer.

19. A method, comprising: receiving training data comprising EEG data collected from a plurality of individual subjects while each subject is viewing specific icons; performing an FFT of the training data to obtain EEG spectrum data; and training a machine learning model to predict which icon the individual subjects are viewing based on the EEG spectrum data.

20. The method of claim 19, wherein the machine learning model comprises: two SE-Res blocks, wherein each SE-Res block comprises: a two-dimensional convolutional layer; a batch normalization layer; an activation layer; an SE block; and wherein the SE block comprises a global max pooling layer, a first fully connected layer having a rectified linear unit activation function, and a second fully connected layer having a sigmoid activation function.