A resolving method and a system, the method includes: creating a sample library by utilization of an original high-resolution (HR) image set; training a convolutional structural network by utilization of the sample library; and obtaining an HR output signal by processing a low-resolution (LR) input signal by utilization of the trained convolutional structural network. In the resolving method and system according to this disclosure, data after expansion may be processed by simple expansion hardware, without a large change algorithm; and complex algorithms are allocated into parallelizing design, and different servers operate mutually independent; and also, due to the modular design, the design proposals of functional modules may be modified by latter optimization.
Legal claims defining the scope of protection, as filed with the USPTO.
1. A resolving method, comprising: creating a sample library by utilization of an original high-resolution (HR) image set; training a convolutional structural network by utilization of the sample library; and obtaining an HR output signal by processing a low-resolution (LR) input signal by utilization of the trained convolutional structural network, wherein the convolutional structural network is formed by an alternate connection of a plurality of convolutional layers and excitation layers, and each convolutional layer includes a plurality of filter units with adjustable filtering parameters, wherein the sample library includes a face feature sample library, and the creating the sample library by utilization of the original HR image set further includes: obtaining an LR image set by a downsampling of the original HR image set; extracting face feature information of LR images by a face feature extraction method; obtaining face feature information of HR images by marking face feature points on the HR images; and creating the face feature sample library, including pairs of the face feature information of the LR images and relevant face feature information of the HR images, by utilization of the face feature information of the LR images and the face feature information of the HR images, wherein the training the convolutional structural network by utilization of the sample library further includes: obtaining a high-pass filtering face result and a low-pass filtering face result by respectively performing high-pass filtering and low-pass filtering on the face feature information of the HR images; and obtaining a detail template of the face feature information of the HR images as a feedback signal of the convolutional structural network by a superimposition and a feature classification of the high-pass filtering face result and the low-pass filtering face result, wherein the high-pass filtering face result including structure and contour information of the face, the low-pass filtering face result including skin texture and roughness of the face.
2. The resolving method according to claim 1 , wherein the training the convolutional structural network by utilization of the sample library further includes: obtaining first filtering parameters for the convolutional structural network by analyzing a correlation between the pairs of the face feature information of the LR images and the relevant face feature information of the HR images in the face feature sample library; and adopting the face feature information of the LR images as an input signal of the convolutional structural network, adjusting the first filtering parameters in the convolutional structural network, and obtaining a forecast result signal, being the same with the feedback signal, by processing the input signal by utilization of the convolutional structural network according to the adjusted first filtering parameters.
3. The resolving method according to claim 2 , wherein the obtaining the HR output signal by processing the LR input signal by utilization of the trained convolutional structural network further includes: inputting the face feature information of the LR images; processing the inputted face feature information of the LR images by utilization of the convolutional structural network according to the adjusted first filtering parameters; and outputting face feature information of the HR images, processed by the convolutional structural network.
4. The resolving method according to claim 2 , wherein the sample library includes an image sample library, and the creating the sample library by utilization of the original HR image set further includes: creating the image sample library, including pairs of the LR images and relevant HR images, by utilization of the LR image set and the HR image set.
5. The resolving method according to claim 4 , wherein the training the convolutional structural network by utilization of the sample library further includes: obtaining second filtering parameters of the convolutional structural network by analyzing a correlation between the pairs of the LR images and the relevant HR images; obtaining a high-pass filtering result and a low-pass filtering result by respectively performing high-pass filtering and low-pass filtering on the HR images; obtaining a detail template of the HR images as a feedback signal of the convolutional structural network by a superimposition and a feature classification of the high-pass filtering result and the low-pass filtering result; and adopting the LR images as an input signal of the convolutional structural network, adjusting the second filtering parameters in the convolutional structural network, and obtaining a forecast result signal, being the same with the feedback signal, by processing the input signal by utilization of the convolutional structural network according to the adjusted second filtering parameters.
6. The resolving method according to claim 5 , wherein the obtaining the HR output signal by processing the LR input signal by utilization of the trained convolutional structural network further includes: inputting the LR images; processing the inputted LR images by utilization of the convolutional structural network according to the adjusted second filtering parameters; and outputting HR images processed by the convolutional structural network.
7. The resolving method according to claim 6 , wherein each filter unit adopts a formula F(x)=Wx+b to execute a convolution operation, where W and b refer to the filtering parameters; x refers to input; and F(x) refers to output.
8. The resolving method according to claim 7 , wherein the forecast result signal is determined to be the same with the feedback signal when J(W,b) obtained according to the following formula is less than a first threshold: J ( W , b ) = 1 m ∑ i = 1 m ( 1 2 h W , b ( I HR i - I HR _ i ) 2 ) where J(W,b) refers to a mean square error; m refers to a number of image sets in the face feature sample library; I HR i refers to the feedback signal; I HR i refers to the forecast result signal; and h W,b refers to a weight coefficient.
9. The resolving method according to claim 8 , wherein when the forecast result signal is different from the feedback signal, a partial derivative of J(W,b) is calculated for each filtering parameter, and the first filtering parameters or the second filtering parameters are adjusted according to the partial derivatives.
10. The resolving method according to claim 9 , wherein the first filtering parameters are classifier filtering parameters for the convolutional structural network.
11. A resolving system, comprising: a sample library creating device configured to create a sample library by utilization of an original high-resolution (HR) image set; a training device configured to train a convolutional structural network by utilization of the sample library; and an output device configured to obtain an HR output signal by processing a low-resolution (LR) input signal by utilization of the trained convolutional structural network, wherein the convolutional structural network is formed by an alternate connection of a plurality of convolutional layers and excitation layers, and each convolutional layer includes a plurality of filter units with adjustable filtering parameters, wherein the sample library includes a face feature sample library, and the sample library creating device further includes: a downsampling unit configured to obtain an LR image set by a downsampling of the original HR image set; a face analysis unit configured to extract face feature information of LR images from the LR images by a face feature extraction method; a feature point marking unit configured to obtain face feature information of HR images by marking face feature points on the HR images; and a face feature sample library creating unit configured to create a face feature sample library, including pairs of the face feature information of the LR images and relevant face feature information of the HR images, by utilization of the face feature information of the LR images and the face feature information of the HR images, wherein the training device further includes a first training unit configured to obtain a high-pass filtering face result and a low-pass filtering face result by respectively performing high-pass filtering and low-pass filtering on the face feature information of the HR images; and obtain a detail template of the face feature information of the HR images as a feedback signal of the convolutional structural network by a superimposition and a feature classification of the high-pass filtering face result and the low-pass filtering face result, wherein the high-pass filtering face result including structure and contour information of the face, the low-pass filtering face result including skin texture and roughness of the face.
12. The resolving system according to claim 11 , wherein the training device further includes: a first training unit configured to: obtain first filtering parameters of the convolutional structural network by analyzing a correlation between the pairs of the face feature information of the LR images and the relevant face feature information of the HR images in the face feature sample library; and adopt the face feature information of the LR images as an input signal of the convolutional structural network, adjust the first filtering parameters in the convolutional structural network, and obtain a forecast result signal, being the same with the feedback signal, by processing the input signal by utilization of the convolutional structural network according to the adjusted first filtering parameters.
13. The resolving system according to claim 12 , wherein the sample library includes an image sample library, and the sample library creating device further includes: an image sample library creating unit configured to create the image sample library, including pairs of the LR images and relevant HR images, by utilization of the LR images and the HR images.
14. The resolving system according to claim 13 , wherein the training device further includes: a second training unit configured to obtain second filtering parameters of the convolutional structural network by analyzing a correlation between the pairs of the LR images and the relevant HR images; obtain a high-pass filtering result and a low-pass filtering result by respectively performing high-pass filtering and low-pass filtering on the HR images; obtain a detail template of the HR images as a feedback signal of the convolutional structural network by a superimposition and a feature classification of the high-pass filtering result and the low-pass filtering result; and adopt the LR images as an input signal of the convolutional structural network, adjust the second filtering parameters in the convolutional structural network, and obtain a forecast result signal, being the same with the feedback signal, by processing the input signal by utilization of the convolutional structural network according to the adjusted second filtering parameters.
15. The resolving system according to claim 14 , wherein the output device further includes: an input unit further configured to input the face feature information and/or images with low resolution; a convolutional structural network further configured to process the inputted face feature information of the LR images and/or the images by utilization of the convolutional structural network according to the adjusted first and/or second filtering parameters; and an output unit further configured to output face feature information and/or images with high resolution, processed by the convolutional structural network.
16. The resolving system according to claim 15 , wherein each filter unit adopts a formula F(x)=Wx+b to execute a convolution operation, where W and b refer to the filtering parameters; x refers to input; and F(x) refers to output.
17. The resolving system according to claim 16 , wherein the forecast result signal is determined to be the same with the feedback signal when J(W,b) obtained according to the following formula is less than a first threshold: J ( W , b ) = 1 m ∑ i = 1 m ( 1 2 h W , b ( I HR i - I HR _ i ) 2 ) where J(W,b) refers to a mean square error; m refers to a number of image sets in the face feature sample library; I HR i refers to the feedback signal; I HR i refers to the forecast result signal; and h W,b refers to a weight coefficient.
18. The resolving system according to claim 17 , wherein, when the forecast result signal is different from the feedback signal, a partial derivative of J(W,b) is calculated for each filtering parameter, and the first filtering parameters or the second filtering parameters are adjusted according to the partial derivatives.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
June 21, 2016
September 8, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.