Systems, methods and computer-accessible mediums for modifying an image(s) can be provided. For example, first image information for the image(s) can be received. Second image information can be generated by separating the first image information into at least two overlapping images. The image(s) can be modified using a prediction procedure based on the second image information.
Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A non-transitory computer-accessible medium having stored thereon computer-executable instructions for modifying at least one image, wherein, when a computer hardware arrangement executes the instructions, the computer arrangement is configured to perform procedures comprising: receiving first image information for the at least one image; generating second image information by separating the first image information into at least two overlapping images; and modifying the at least one image using a prediction procedure based on the second image information wherein the prediction procedure is associated with at least one convolutional neural network that receives each of the overlapping images as an input.
A computer program stored on a non-transitory medium modifies images by receiving the image data, splitting it into at least two overlapping images, and then using a prediction procedure, specifically a convolutional neural network, to enhance the image. The convolutional neural network takes the overlapping images as input to improve the image quality. This is intended to remove artifacts from the original image.
2. The non-transitory computer-accessible medium of claim 1 , wherein the prediction procedure includes an average prediction determination for each overlapping pixels of the at least two overlapping images.
Building on the image modification process using a convolutional neural network, as described above, the prediction procedure calculates an average prediction for each overlapping pixel present in the at least two overlapping images. This averaging technique is applied to reduce noise and refine the predicted clean image.
3. The non-transitory computer-accessible medium of claim 1 , wherein the prediction procedure further includes predicting a clean image for at least one of the overlapping images.
In addition to using a convolutional neural network to modify images, as described above, the prediction procedure also involves predicting a clean version of at least one of the overlapping images. The convolutional neural network aims to generate a cleaner, clearer image from the overlapping segments.
4. The non-transitory computer-accessible medium of claim 1 , wherein the at least one convolutional neural network includes at least two convolutional neural networks, and wherein each of the at least two convolutional neural networks is configured to remove at least one of dirt, debris or water from the at least one image.
Expanding on the image modification using a convolutional neural network, as described above, the convolutional neural network actually consists of at least two convolutional neural networks. Each of these networks is designed to remove specific types of obstructions, such as dirt, debris, or water, from the original image. This allows for specialized cleaning of images captured through windows.
5. The non-transitory computer-accessible medium of claim 1 , wherein the at least one convolutional neural network includes a series of layers.
Expanding on the image modification using a convolutional neural network, as described above, the convolutional neural network is constructed as a series of interconnected layers. This layered architecture enables the network to learn complex patterns and relationships within the image data, facilitating more effective image enhancement.
6. The non-transitory computer-accessible medium of claim 5 , wherein each of the layers applies a linear map to at least one of the overlapping images.
Building on the layered convolutional neural network used for image modification, as described above, each layer within the network applies a linear transformation (linear map) to the at least one of the overlapping images. This linear transformation allows the network to extract relevant features from the image data in each layer.
7. The non-transitory computer-accessible medium of claim 6 , wherein each of the layers further applies an element-wise sigmoid to at least one of the overlapping images.
Further enhancing the layered convolutional neural network for image modification, as described above with the application of a linear map in each layer, each layer additionally applies an element-wise sigmoid function to the at least one of the overlapping images. The sigmoid function introduces non-linearity, enabling the network to model more complex relationships.
8. The non-transitory computer-accessible medium of claim 4 , wherein the prediction procedure includes the at least one convolutional neural network.
Expanding on the image modification involving multiple convolutional neural networks to remove dirt, debris or water from the at least one image, as described above, the prediction procedure directly leverages these convolutional neural networks to achieve image enhancement. The convolutional neural networks are integral to predicting and generating a cleaner image.
9. The non-transitory computer-accessible medium of claim 1 , wherein the modification includes removing components of the at least one image associated with at least one of dirt, debris, or water from the at least one image.
This invention relates to image processing techniques for enhancing digital images by removing unwanted elements such as dirt, debris, or water. The system processes at least one image to identify and eliminate these unwanted components, improving image clarity and quality. The method involves analyzing the image to detect regions affected by dirt, debris, or water, then applying modifications to remove or reduce these elements. The processing may include techniques such as inpainting, filtering, or other restoration methods to reconstruct the affected areas while preserving the original image content. The invention ensures that the modified image retains its essential features while eliminating visual distortions caused by external contaminants. This approach is particularly useful in applications where image quality is critical, such as medical imaging, surveillance, or photography, where unwanted artifacts can obscure important details. The system may be implemented as part of a broader image enhancement pipeline, integrating seamlessly with other processing steps to deliver a refined final output. The invention addresses the challenge of maintaining image integrity while removing unwanted elements, providing a solution that enhances visual clarity without introducing additional artifacts.
10. The non-transitory computer-accessible medium of claim 9 , wherein the at least one convolutional neural network includes at least three convolutional neural networks, a first convolutional neural network for the removal of the dirt, a second convolutional neural network for the removal of the debris and a third convolutional neural network for the removal of the water.
Further specifying the image modification process of removing dirt, debris, or water from the image using a convolutional neural network, as described above, the system employs at least three distinct convolutional neural networks. The first network is dedicated to removing dirt, the second to removing debris, and the third to removing water, allowing targeted removal of specific types of obstructions.
11. The non-transitory computer-accessible medium of claim 1 , wherein the first image information includes information from a picture taken through a window having at least one of dirt, debris or water thereon.
Expanding on the image modification using a convolutional neural network, as described above, the initial image information comes from a picture taken through a window that contains dirt, debris, or water. The system specifically addresses the common issue of image quality degradation caused by these window contaminants.
12. The non-transitory computer-accessible medium of claim 1 , wherein the computer arrangement is further configured to generate at least one weight for use by the prediction procedure.
Expanding on the image modification using a convolutional neural network, as described above, the system also generates at least one weight that the prediction procedure (convolutional neural network) uses. These weights control the influence of different features during image processing, helping to fine-tune the image modification process.
13. The non-transitory computer-accessible medium of claim 12 , wherein the computer arrangement is further configured to generate the at least one weight by minimizing a mean squared error over a dataset of corresponding noisy and clean image pairs.
Further enhancing the system that modifies images using convolutional neural networks and generating weights, as described above, the system creates the at least one weight by minimizing the mean squared error over a dataset of "noisy" (contaminated) and clean image pairs. The system learns the optimal weight values that best map noisy images to their clean counterparts.
14. The non-transitory computer-accessible medium of claim 13 , wherein the computer arrangement is further configured to reduce a loss of the mean squared error.
Continuing the description of the image modification system that minimizes mean squared error to create weights, as described above, the system further reduces the loss of the mean squared error. This indicates an iterative process is used to optimize the weight values for more accurate image cleaning.
15. The non-transitory computer-accessible medium of claim 14 , wherein the computer arrangement is further configured to reduce the loss using a Stochastic Gradient Descent.
Further refining the image modification process by reducing the loss of the mean squared error to create weights, as described above, the loss reduction is achieved using Stochastic Gradient Descent (SGD). SGD is an optimization algorithm that iteratively updates the weights based on the gradient of the error function.
16. The non-transitory computer-accessible medium of claim 15 , wherein the computer arrangement is further configured to determine a gradient of the error by backpropagating a depatchifying procedure.
In the image modification system that uses Stochastic Gradient Descent (SGD) to reduce error and create weights, as described above, the gradient of the error is determined by backpropagating a "depatchifying" procedure. This likely means that after processing the overlapping image patches through the convolutional neural network, the gradient information is propagated back through the network and the reassembly process to calculate the overall error gradient.
17. The non-transitory computer-accessible medium of claim 12 , wherein the computer arrangement is further configured to initialize the at least one weight by randomly drawing from a normal distribution with a mean of about 0 and a standard deviation of about 0.001.
In the image modification system that generates weights for a convolutional neural network, as described above, the at least one weight is initialized by randomly drawing values from a normal distribution. The normal distribution has a mean of approximately 0 and a small standard deviation of approximately 0.001, setting the initial state of the convolutional neural network weights.
18. The non-transitory computer-accessible medium of claim 12 , wherein the computer arrangement is further configured to decorrelate the at least one weight.
Further enhancing the image modification system using convolutional neural networks and weight generation, as described above, the system is also configured to decorrelate the at least one weight. Decorrelation techniques can improve the training and generalization performance of the neural network by reducing redundancy between the weights.
19. A method for modifying at least one image, comprising: receiving first image information for the at least one image; generating second image information by separating the first image information into at least two overlapping images; and using a computer hardware arrangement, modifying the at least one image using a prediction procedure based on the second image information wherein the prediction procedure is associated with at least one convolutional neural network that receives each of the overlapping images as an input.
A method for modifying images comprises the steps of: receiving image data, splitting the image data into at least two overlapping images, and then using a computer to modify the image with a prediction procedure, specifically a convolutional neural network. This convolutional neural network takes the overlapping images as input to improve the image quality. This is intended to remove artifacts from the original image.
20. A system for modifying at least one image, comprising a computer hardware arrangement configured to: receive first image information for the at least one image; generate second image information by separating the first image information into at least two overlapping images; and modify the at least one image using a prediction procedure based on the second image information wherein the prediction procedure is associated with at least one convolutional neural network that receives each of the overlapping images as an input.
A system for modifying images is comprised of a computer that: receives image data, splits the image data into at least two overlapping images, and then modifies the image with a prediction procedure, specifically a convolutional neural network. This convolutional neural network takes the overlapping images as input to improve the image quality. This is intended to remove artifacts from the original image.
21. The non-transitory computer-accessible medium of claim 4 , wherein the at least one of the dirt, debris or water is from a picture taken through a window.
Expanding on the image modification involving multiple convolutional neural networks to remove dirt, debris or water from the at least one image, as described above, the dirt, debris, or water originates from a picture taken through a window. The system is designed to specifically address the problem of cleaning up images captured through windows.
22. The non-transitory computer-accessible medium of claim 9 , wherein the at least one of the dirt, debris or water is from a picture taken through a window.
Expanding on the image modification system that removes dirt, debris, or water from the image using a convolutional neural network, as described above, the source of the dirt, debris, or water is from a picture taken through a window. The invention specifically addresses image degradation issues caused by taking pictures through dirty or wet windows.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
June 20, 2016
June 6, 2017
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.