An image processing apparatus has processing circuitry configured to perform learning of a first model for detecting a length measurement target region included in a learning image, perform learning of a second model for detecting an attention region included in the learning image, input a detection target image to the learned first model and detect a length measurement target region inferred by the first model, input an image of the detected length measurement target region to the learned second model and detect an attention region inferred by the second model, and perform length measurement of the length measurement target region based on the detected attention region.
Legal claims defining the scope of protection, as filed with the USPTO.
perform learning of a first model for detecting a length measurement target region included in a learning image; perform learning of a second model for detecting an attention region included in the learning image; input a detection target image to the learned first model and detect a length measurement target region inferred by the first model; input an image of the detected length measurement target region to the learned second model and detect an attention region inferred by the second model; and perform length measurement of the length measurement target region based on the detected attention region. . An image processing apparatus comprising processing circuitry configured to:
claim 1 the attention region includes at least one of a contour of the length measurement target region, a shape of the length measurement target region, or a part included in the length measurement target region. . The image processing apparatus according to, wherein
claim 1 generate a length measurement target region learning image to be used for learning the first model based on the learning image; and generate an attention region learning image to be used for learning the second model based on the learning image, wherein the processing circuitry is configured to: perform learning of the first model for detecting a length measurement target region included in the length measurement target region learning image, and perform learning of the second model for detecting an attention region included in the attention region learning image. . The image processing apparatus according to, wherein the processing circuitry is further configured to:
claim 3 the processing circuitry is configured to: extract the length measurement target region from the learning image, and generate the length measurement target region learning image and a first label image for identifying the length measurement target region based on the extracted length measurement target region, and extract the length measurement target region from the learning image, and after removing noise in the extracted length measurement target region, add an image of an attention region and an image of a non-attention region, and generate the attention region learning image and a second label image indicating the attention region and the non-attention region. . The image processing apparatus according to, wherein
claim 3 . The image processing apparatus according to, wherein the processing circuitry is further configured to generate a plurality of the learning images to be used.
claim 5 the processing circuitry is configured to select, as the learning image, from a learning image data set including a plurality of learning image candidates, a learning image candidate having similarity of a predetermined value or more to a plurality of the detection target images. . The image processing apparatus according to, wherein
claim 6 the processing circuitry is configured to select, as the learning image, the learning image candidate in which cosine similarity of an average feature amount of the plurality of detection target images with respect to a feature amount of each of the plurality of learning image candidates, or an average value of the cosine similarity of the feature amount of each of the plurality of detection target images with respect to the feature amount of each of the plurality of learning image candidates is equal to or more than the predetermined value. . The image processing apparatus according to, wherein
claim 5 the processing circuitry is configured to generate the learning image, from a learning image data set including a plurality of learning image candidates, by randomly adding at least one of a plurality of structural parts and a plurality of noise parts associated with a representative image to the representative image, with the learning image candidate having similarity of a predetermined value or more to the plurality of detection target images as the representative image. . The image processing apparatus according to, wherein
claim 5 the processing circuitry is configured to input a learning image candidate and user instruction information to a learned multimodal model, and set an output image inferred by the multimodal model as the learning image. . The image processing apparatus according to, wherein
claim 9 the instruction information includes information for designating at least one of the attention region and noise. . The image processing apparatus according to, wherein
claim 1 input the detection target image to the learned first model and determine whether a position of the length measurement target region inferred by the first model is in a normal range; and add the detection target image including the length measurement target region determined to be in the normal range to a learning image data set as a new learning image. . The image processing apparatus according to, wherein the processing circuitry is further configured to:
claim 11 the processing circuitry is configured to add the detection target image including the length measurement target region after correcting the length measurement target position to the learning image data set as the new learning image. . The image processing apparatus according to, wherein the processing circuitry is further configured to correct a length measurement target position when determined that the position is not in the normal range, wherein
claim 3 the processing circuitry is configured to generate the attention region learning image by using the detected noise. . The image processing apparatus according to, wherein the processing circuitry is further configured to input the image of the length measurement target region to the learned second model and detect noise included in the attention region inferred by the second model, wherein
claim 1 determine whether a length measurement value of the measured length measurement target region falls within an allowable range; and add the detection target image including the length measurement target region determined to fall within the allowable range to a new learning image. . The image processing apparatus according to, wherein the processing circuitry is further configured to:
claim 3 determine whether a length measurement value of the measured length measurement target region falls within an allowable range; and add the detection target image including the length measurement target region determined to fall within the allowable range to a new learning image, wherein when determined that the length measurement value does not fall within the allowable range, the processing circuitry is configured to regenerate the attention region learning image to be used for learning of the second model. . The image processing apparatus according to, wherein the processing circuitry is further configured to:
claim 1 the learning image and the detection target image are semiconductor images. . The image processing apparatus according to, wherein
performing learning of a first model for detecting a length measurement target region included in a learning image; performing learning of a second model for detecting an attention region included in the learning image; inputting a detection target image to the learned first model and detecting a length measurement target region inferred by the first model; inputting an image of the detected length measurement target region to the learned second model and detecting an attention region inferred by the second model; and performing length measurement of the length measurement target region based on the detected attention region. . An image processing method comprising:
claim 17 the attention region includes at least one of a contour of the length measurement target region, a shape of the length measurement target region, and a part included in the length measurement target region. . The image processing method according to, wherein
claim 17 generating a length measurement target region learning image to be used for learning the first model based on the learning image; generating an attention region learning image to be used for learning the second model based on the learning image; performing learning of the first model for detecting a length measurement target region included in the length measurement target region learning image; and performing learning of the second model for detecting an attention region included in the attention region learning image. . The image processing method according to, further comprising:
claim 19 extracting the length measurement target region from the learning image, and generating the length measurement target region learning image and a first label image for identifying the length measurement target region based on the extracted length measurement target region; and extracting the length measurement target region from the learning image, and after removing noise in the extracted length measurement target region, adding an image of an attention region and an image of a non-attention region, and generating the attention region learning image and a second label image indicating the attention region and the non-attention region. . The image processing method according to, further comprising:
Complete technical specification and implementation details from the patent document.
This application is based upon and claims the benefit of priority from the prior Japanese Patent Application No. 2025-040655, filed on Mar. 13, 2025, the entire contents of which are incorporated herein by reference.
The present disclosure relates to an image processing apparatus and an image processing method.
A technique of detecting a specific shape from a detection target image using a learning image including the specific shape and a model that has learned information regarding the specific shape included in the learning image has been proposed.
However, in the existing technique, it is assumed that no noise is included in the detection target image, and in a case where noise is included in the detection target image, detection accuracy of the specific shape may decrease. In addition, in a case where the detection accuracy of a contour boundary of the detection target image is low, it is difficult to detect a detection target region itself.
Furthermore, in the existing technique, a large amount of learning images are required to learn the model, and it takes time and effort to construct the model.
perform learning of a first model for detecting a length measurement target region included in a learning image; perform learning of a second model for detecting an attention region included in the learning image; input a detection target image to the learned first model and detect a length measurement target region inferred by the first model; input an image of the detected length measurement target region to the learned second model and detect an attention region inferred by the second model; and perform length measurement of the length measurement target region based on the detected attention region. An image processing apparatus according to an embodiment includes processing circuitry configured to:
Hereinafter, embodiments of an image processing apparatus and an image processing method will be described with reference to the drawings. Although main components of the image processing apparatus will be mainly described below, the image processing apparatus may have components and functions that are not illustrated or described. The following description does not exclude the components and functions that are not illustrated or described.
1 FIG. 1 FIG. 1 1 1 1 1 is a block diagram illustrating a schematic configuration of an image processing apparatusaccording to a first embodiment. The image processing apparatusillustrated inincludes, for example, a general-purpose computer, a central processing unit (CPU), or the like. As will be described later, at least some functions of the image processing apparatusaccording to the embodiment can be implemented by hardware or software. In a case where at least some functions of the image processing apparatusare implemented by software, the functions of the image processing apparatuscan be implemented by a computer or a CPU executing a program. Hereinafter, the computer or the CPU may be called as processing circuitry.
1 FIG. 1 2 3 4 5 6 As illustrated in, the image processing apparatusaccording to the first embodiment includes a first model learning unit, a second model learning unit, a length measurement target region detection unit, an attention region detection unit, and a length measurement unit.
2 7 7 2 The first model learning unitlearns a length measurement target region model (first model) for detecting a length measurement target region included in a learning image. The learning image is read out from a learning image storage unit, for example. The learning image storage unitstores a learning image data set including a plurality of learning images. The learning image is, for example, a semiconductor image. The semiconductor image may be a cross-sectional image of a semiconductor device, or may be a layout image or an external appearance image. In addition, the learning image is not necessarily a semiconductor image, and any image such as a microscopic image can be applied. In the present specification, an example in which the learning image is a semiconductor image will be mainly described. The length measurement target region refers to a region having a characteristic shape of the learning image. The characteristic shape refers to a contour of the learning image, shapes of various parts included in the learning image, or the like. The length measurement target region model is a model that detects a length measurement target region included in the input learning image by inference processing by machine learning. The first model learning unitperforms learning of the length measurement target region model by using the plurality of learning images.
3 The second model learning unitlearns an attention region model (second model) for detecting an attention region included in the learning image. The attention region includes at least one of a contour of the length measurement target area, a shape of the length measurement target area, or various parts included in the length measurement target area.
The types of the length measurement target region model and the attention region model are not limited. For example, a semantic segmentation model such as U-Net may be used. In addition, a type of a loss function used to determine an end condition of learning of the length measurement target region model and the attention region model is also arbitrary. For example, DiceLoss may be used as the loss function.
4 4 8 6 The length measurement target region detection unitinputs a detection target image to the learned length measurement target region model (first model), and detects a length measurement target region inferred by the length measurement target region model. The detection target image is an image to be subjected to length measurement. Although the type of the detection target image is not limited, an example in which the detection target image is a semiconductor image will be mainly described in the present specification. For example, the length measurement target region detection unitselects an arbitrary detection target image from a SEM image group including a plurality of detection target images. The SEM image group is stored in, for example, a detection target image storage unit. For example, the length measurement unitmeasures a dimension of a characteristic shape of the length measurement target area.
5 4 The attention region detection unitinputs an image of the length measurement target region (hereinafter, a length measurement target region image) detected by the length measurement target region detection unitto the learned attention region model (second model), and detects an attention region inferred by the attention region model.
6 5 6 6 6 The length measurement unitperforms length measurement of critical dimension (CD) of the length measurement target region based on the attention region detected by the attention region detection unit. For example, the length measurement unitmeasures a dimension of the attention region included in the length measurement target area. The length measurement unitdoes not necessarily measure one length. The length measurement unitmay perform length measurement of each of a plurality of characteristic shapes of the length measurement target area.
1 9 10 The image processing apparatusaccording to the first embodiment may include a first learning image generation unitand a second learning image generation unit.
9 2 2 The first learning image generation unitgenerates a length measurement target region learning image to be used by the first model learning unitfor learning the length measurement target region model based on the learning image. The first model learning unitperforms learning of the length measurement target region model for detecting the length measurement target region included in the length measurement target region learning image.
10 3 3 The second learning image generation unitgenerates an attention region learning image to be used by the second model learning unitfor learning the attention region model based on the learning image. The second model learning unitperforms learning of the attention region model for detecting the attention region included in the attention region learning image.
2 FIG. 3 FIG. 3 FIG. 1 1 is a flowchart illustrating a processing operation of the image processing apparatusaccording to the first embodiment, andis a diagram illustrating an example of image processing performed by the image processing apparatusaccording to the first embodiment.illustrates an example in which the learning image and the detection target image are SEM images of the semiconductor device.
2 FIG. 3 FIG. 9 1 As illustrated in, the first learning image generation unitextracts a length measurement target region from a learning image (step S). In, the length measurement target region is indicated by a broken line.
9 2 3 FIG. Next, the first learning image generation unitgenerates a length measurement target region learning image and a length measurement target region label image (first label image) based on the extracted length measurement target region (step S). The length measurement target region label image is information for specifying a location of the length measurement target area. In, the length measurement target region is illustrated in white.
2 3 Next, the first model learning unitinputs the length measurement target region learning image to a length measurement target region model and learns the length measurement target region model (step S).
2 4 1 4 4 4 1 4 Next, the first model learning unitdetermines whether learning of the length measurement target region model has ended (step S). The processing of steps Sto Sis repeated until step Sbecomes YES. In step S, for example, when a loss function of the length measurement target region model becomes a predetermined value or less, it is determined that the processing has ended. Alternatively, when the number of repetitions of steps Sto Sreaches a predetermined number, it may be determined that the processing has ended.
1 4 5 8 5 10 In parallel with the processing of steps Sto S, the processing of steps Sto Sis performed. In step S, the second learning image generation unitextracts the length measurement target region from the learning image and removes noise in the length measurement target area.
6 10 In step S, the second learning image generation unitadds an attention region and a non-attention region to the length measurement target region from which noise has been removed and generates an attention region learning image and an attention region label image (second label image). The attention region label image is an image for identifying the attention region and the non-attention area. The attention region is, for example, a characteristic shape such as a contour or a part included in the length measurement target area. The non-attention region is, for example, noise such as dust included in the attention area. Various noises are randomly added to the non-attention area. Here, the noise is a foreign substance that is not originally expected to be included in the length measurement target area.
7 3 In step S, the second model learning unitinputs the attention region learning image to the attention region model and learns the attention region model.
8 3 5 8 8 8 4 In step S, the second model learning unitdetermines whether the learning of the attention region model has ended. The processing of steps Sto Sis repeated until step Sbecomes YES. The determination in step Smay be performed based on the same criteria as the determination in step S.
4 8 4 9 When YES is determined in steps Sand S, the length measurement target region detection unitinputs a detection target image included in the SEM image group to the learned length measurement target region model and detects the length measurement target region inferred by the length measurement target region model (step S).
5 10 Next, the attention region detection unitinputs an image of the detected length measurement target region (length measurement target region image) to the learned attention region model, and detects the attention region inferred by the attention region model (step S).
6 11 Next, the length measurement unitperforms length measurement of the length measurement target region based on the attention region detected in step S10 (step S).
4 FIG. 1 FIG. 4 FIG. 9 10 9 10 9 is a diagram for explaining processing operations of the first learning image generation unitand the second learning image generation unitinin detail. The first learning image generation unitand the second learning image generation unitperform image processing on the learning image and specify the length measurement target area.illustrates an example in which a length measurement target region is specified by a line connecting “X” and “X”, and a length measurement target region indicated by a broken line is generated. The first learning image generation unitgenerates a length measurement target region image and a length measurement target region label image indicating a location of the length measurement target region based on the length measurement target area.
10 4 FIG. 4 FIG. The second learning image generation unitgenerates an attention region mask and a noise mask for the length measurement target region by image processing or manually. The attention region mask is used to specify the location of the attention region when the attention region label image is generated. A black portion inindicates the attention area. The noise mask is processing of masking noise in the length measurement target area. A white portion inrepresents noise.
10 Next, the second learning image generation unitperforms “inpainting” processing of extracting only a contour of the length measurement target area. In the inpainting processing, the noise mask is used, and the attention region mask is not used. Noise included in the length measurement target region is removed by the inpainting processing. An image after the inpainting processing does not include noise and includes a length measurement target region and an attention area.
10 Next, the second learning image generation unitgenerates a plurality of attention region learning images by adding an image of the attention region and an image of the non-attention region to the image after the inpainting processing. The image of the attention region is selected from, for example, various parts included in an attention region image data set. The image of the non-attention region is selected from, for example, various noises included in a non-attention region image data set. The attention region image data set and the non-attention region image data set are prepared in advance based on the learning image data set. Furthermore, as will be described later, parts and noise included in the attention region image data set and the non-attention region image data set may be arbitrarily added.
10 The second learning image generation unitgenerates the attention region learning image and generates the attention region label image indicating the positions of the image of the attention region and the image of the non-attention area.
5 FIG. 1 FIG. 2 3 4 5 6 is a diagram for explaining the processing operations of the first model learning unit, the second model learning unit, the length measurement target region detection unit, the attention region detection unit, and the length measurement unitinin detail.
2 9 2 The first model learning unitadds data augmentation such as blurring to the length measurement target region image generated by the first learning image generation unitand generates an input image for the length measurement target region model. The first model learning unitperforms learning of the length measurement target region model so that the length measurement target region model outputs a correct length measurement target region label image.
3 10 3 The second model learning unitadds data augmentation such as blurring to the attention region image generated by the second learning image generation unitand generates an input image for the attention region model. The second model learning unitperforms learning of the attention region model so that the attention region model outputs a correct attention region label image.
4 The length measurement target region detection unitinputs a detection target image selected from the SEM image group to the learned length measurement target region model, and detects a length measurement target region (more precisely, the length measurement target region image) by image processing based on the length measurement target region label image inferred by the length measurement target region model and the detection target image.
5 4 The attention region detection unitinputs the length measurement target region image detected by the length measurement target region detection unitto the learned attention region model and detects the inferred attention region (more precisely, the attention region image).
6 5 The length measurement unitperforms length measurement of the attention region included in the length measurement target region by performing image processing on the attention region image detected by the attention region detection unit.
As described above, in the first embodiment, the length measurement target region learning image and the attention region learning image are generated from the learning image, the length measurement target region model and the attention region model are learned using these learning images, the detection target image is input to the learned length measurement target region model and inference is performed to detect the length measurement target region image, and the detected length measurement target region image is input to the learned attention region model and inference is performed to detect the attention region image. As a result, the length measurement of the length measurement target region can be performed with high accuracy using a small number of learning images. In addition, since the learning of the attention region model is performed by using the attention region learning image intentionally including various noises, it is possible to perform length measurement of the length measurement target region without being affected by noise even when noise is included in the detection target image, and noise resistance is increased.
6 FIG. 6 FIG. 1 FIG. 1 1 11 a a is a block diagram illustrating a schematic configuration of an image processing apparatusaccording to a second embodiment. The image processing apparatusaccording to the second embodiment illustrated inincludes a learning image generation unitin addition to the block configuration of.
11 9 10 The learning image generation unitgenerates a plurality of learning images to be used by the first learning image generation unitand the second learning image generation unit.
7 FIG. 7 FIG. 2 FIG. 7 FIG. 2 FIG. 1 0 1 1 a is a flowchart illustrating a processing operation of the image processing apparatusaccording to the second embodiment. In the flowchart of, step Sis added before step Sof the flowchart of. Step Sand subsequent steps inare similar to those in the flowchart in.
0 11 9 10 In step S, the learning image generation unitgenerates a plurality of learning images to be used by the first learning image generation unitand the second learning image generation unit, for example, by using the learning image included in the learning image data set and the detection target image included in the SEM image group.
8 FIG. 7 FIG. 8 FIG. 11 11 11 9 10 is a diagram for explaining a processing operation of the learning image generation unitin. As illustrated in, the learning image generation unitperforms similar image search by using the learning image included in the learning image data set and the detection target image included in the SEM image group to generate a learning SEM image (hereinafter, simply referred to as a learning image). The learning image generated by the learning image generation unitis supplied to the first learning image generation unitand the second learning image generation unit.
9 FIG. 7 FIG. 11 11 12 13 is a diagram for explaining the processing operation of the similar image search performed by the learning image generation unitinin detail. The learning image generation unitincludes a similarity calculation unitand an image selection unit.
12 12 12 The similarity calculation unitcalculates similarity between the plurality of learning images included in the learning image data set and the plurality of detection target images included in the SEM image group. For example, the similarity calculation unitcalculates cosine similarity of an average feature amount of all the detection target images with respect to feature amounts of respective learning images. Alternatively, the similarity calculation unitmay calculate an average value of cosine similarity of feature amounts of the respective detection target images with respect to feature amounts of the respective learning images.
13 12 13 The image selection unitselects, for example, a learning image having similarity of a predetermined value or more from among a plurality of similarities calculated by the similarity calculation unit. The predetermined value is arbitrary. As a specific example, the image selection unitselects a learning image whose similarity is included in the top 50%.
9 10 9 10 As described above, in the second embodiment, a learning image having high similarity to the detection target image included in the SEM image group is selected from the learning image data set and supplied to the first learning image generation unitand the second learning image generation unit. As a result, the first learning image generation unitand the second learning image generation unitcan generate a length measurement target region learning image and an attention region learning image suitable for the detection target image. Therefore, the inference accuracy of the length measurement target region inferred by the learned length measurement target region model can be improved.
1 11 a 6 FIG. An image processing apparatusaccording to a third embodiment has a block configuration similar to that of, and the processing operation of the learning image generation unitis different from that of the second embodiment.
10 FIG. 10 FIG. 11 11 12 14 15 is a diagram for explaining a processing operation of the learning image generation unitaccording to the third embodiment. As illustrated in, the learning image generation unitaccording to the third embodiment includes a similarity calculation unit, a part storage unitthat stores a learning image part data set, and a part selection image generation unit.
12 12 14 15 10 FIG. 9 FIG. The processing of the similarity calculation unitinis similar to that of the similarity calculation unitin. The part storage unitstores parts included in each of the plurality of learning images included in the learning image data set. The part selection image generation unitrandomly selects some parts from a plurality of parts associated with a learning image having high similarity (hereinafter, may be referred to as a representative image).
11 FIG. 11 FIG. 11 14 is a diagram for explaining a processing operation of the learning image generation unitaccording to the third embodiment in detail. As illustrated in, the part storage unitstores a structural part and a noise part for each learning image (representative image) included in the learning image data set. The structural part is a structure having a characteristic shape of the learning image. The noise part is noise such as dust included in the learning image. The noise part is a part that is not originally expected to be included in the learning image.
15 12 9 10 The part selection image generation unitgenerates a learning image by using the learning image having high similarity calculated by the similarity calculation unitas a representative image and randomly adding a structural part and a noise part associated with the representative image. The generated learning image is supplied to the first learning image generation unitand the second learning image generation unit.
As described above, in the third embodiment, since the learning image is generated by randomly adding the structural part and the noise part of the learning image having high similarity with the detection target image, a necessary and sufficient number of learning images can be easily generated without preparing many learning images in advance. In particular, in the third embodiment, the noise parts can be arbitrarily combined for each shape of the learning image, and the learning image can be diversified.
1 11 a 6 FIG. An image processing apparatusaccording to a fourth embodiment has a block configuration similar to that of, and a processing operation of the learning image generation unitis different from those of the second and third embodiments.
12 FIG. 12 FIG. 11 11 16 16 is a diagram for explaining a processing operation of the learning image generation unitaccording to the fourth embodiment. As illustrated in, the learning image generation unitaccording to the fourth embodiment generates a learning image by using a learning image generation model. For example, a multimodal model is used as the learning image generation model. The multimodal model is a model that can perform inference processing by using a plurality of types of input data and parameters.
13 FIG. 11 11 17 17 is a diagram for explaining the processing operation of the learning image generation unitaccording to the fourth embodiment in detail. The learning image generation unituses a learned multimodal model. The multimodal modelinputs a plurality of types of data and parameters, performs inference processing, and outputs an image subjected to the inference processing. User instruction information includes, for example, information for designating at least one of an attention region and noise. The noise includes, for example, structural defects or holes containing dust.
17 The learning image inferred by the multimodal modelis an image obtained by adding at least one of a new attention region and noise to the original learning image.
17 In addition, the detection target image may be given to the multimodal modelas an input parameter. As a result, it is possible to generate a learning image having high similarity with the detection target image.
17 9 10 The learning image inferred by the multimodal modelis supplied to the first learning image generation unitand the second learning image generation unit.
17 17 As described above, in the fourth embodiment, since the learning image is generated by using the learned multimodal model, it is possible to simply generate a large number of learning images suitable for the detection target image by performing learning of the multimodal modelin advance.
14 FIG. 14 FIG. 1 FIG. 1 1 21 22 b b is a block diagram illustrating a schematic configuration of an image processing apparatusaccording to a fifth embodiment. The image processing apparatusaccording to the fifth embodiment illustrated inincludes a positional deviation determination unitand a first learning image addition unitin addition to the block configuration of.
21 The positional deviation determination unitinputs a detection target image to the learned length measurement target region model and determines whether a position of a length measurement target region inferred by the length measurement target region model is in a normal range.
22 21 The first learning image addition unitadds the detection target image including the length measurement target region determined to be in the normal range by the positional deviation determination unitto the learning image data set as a new learning image.
15 FIG. 15 FIG. 21 22 21 4 21 is a diagram for explaining processing operations of the positional deviation determination unitand the first learning image addition unitaccording to the fifth embodiment. The positional deviation determination unitconfirms the position of the length measurement target region detected by the length measurement target region detection unit, and determines whether the position of the length measurement target region is in a normal range. A broken line L inindicates a center line of the length measurement target area. For example, the positional deviation determination unitdetermines whether the center line is positioned in the normal range.
22 The first learning image addition unitadds the detection target image having the length measurement target region determined to be positioned in the normal range to the learning image data set.
1 23 23 21 22 23 b 14 FIG. The image processing apparatusaccording to the fifth embodiment illustrated inmay include a position correction unit. The position correction unitcorrects a length measurement target position when the positional deviation determination unitdetermines that the position is not in the normal range. In this case, the first learning image addition unitadds the detection target image including the length measurement target region after the correction of the length measurement target position by the position correction unitto the learning image data set as a new learning image. In addition, the detection target image before correction may be deleted from the SEM image group.
16 FIG. 15 FIG. 23 23 is a diagram in which the position correction unitis added to. The position correction unitshifts the position of the length measurement target region so that the position of the length measurement target region falls in the normal range.
17 FIG. 17 FIG. 17 FIG. 2 FIG. 1 23 9 9 9 10 b a b is a flowchart illustrating a processing operation of the image processing apparatusaccording to the fifth embodiment. The flowchart ofincludes a processing operation by the position correction unit. The flowchart ofis obtained by adding processing of steps Sand Sbetween steps Sand Sof.
9 21 4 a In step S, the positional deviation determination unitconfirms a position of the length measurement target region detected by the length measurement target region detection unit, and determines whether the position of the length measurement target region is in a normal range.
9 9 23 9 10 b a a In step S, when it is determined in step Sthat the position of the length measurement target region is out of the normal range, the position correction unitcorrects the position of the length measurement target region to fall in the normal range. In a case where it is determined in step Sthat the position of the length measurement target region is in the normal range, processing in step Sdescribed later is performed.
9 c In step S, the detection target image corresponding to the corrected length measurement target region is added to the learning image data set, and the original detection target image is deleted from the SEM image group.
10 11 In step S, the image of the detected or corrected length measurement target region is input to the learned attention region model, and the attention region is detected by inference. Thereafter, length measurement of the length measurement target region is performed (step S).
As described above, in the fifth embodiment, it is determined whether the position of the length measurement target region is in the normal range, and the detection target image including the length measurement target region determined to be in the normal range is added to the learning image data set, thereby efficiently increasing the number of learning images. In addition, when it is determined that the position of the length measurement target region is out of the normal range, the position of the length measurement target region is corrected and then added to the learning image data set, so that the length measurement of the length measurement target region can be accurately performed.
18 FIG. 18 FIG. 1 FIG. 1 1 24 c c is a block diagram illustrating a schematic configuration of an image processing apparatusaccording to a sixth embodiment. The image processing apparatusaccording to the sixth embodiment illustrated inincludes a noise detection unitin addition to the block configuration of.
24 The noise detection unitinputs the length measurement target region to the learned attention region model and detects noise included in the attention region inferred by the attention region model. The noise is, for example, a foreign substance that does not originally exist in the attention area, such as dust.
10 24 The second learning image generation unitgenerates an attention region learning image and a second label image by including the noise detected by the noise detection unitin a non-attention area.
19 FIG. 20 FIG. 1 24 c is a flowchart illustrating a processing operation of the image processing apparatusaccording to the sixth embodiment, andis a diagram for explaining a processing operation of the noise detection unitaccording to the sixth embodiment.
19 FIG. 2 FIG. 10 10 10 11 a b The flowchart ofis obtained by adding steps Sand Sbetween steps Sand Sof the flowchart of.
10 24 10 10 10 10 10 11 a b a a b In step S, the noise detection unitdetects noise included in the attention area. In step S, the second learning image generation unitgenerates the attention region learning image and the second label image by including the noise (for example, dust) detected in step Sin the non-attention area. The processing of steps Sand Scan be performed in parallel with the processing of step S.
21 FIG. 24 10 24 is a diagram for explaining processing operations of the noise detection unitand the second learning image generation unitin more detail. The noise detection unitdetects various noises (for example, dust) included in the attention region and cuts out only dust.
10 24 The second learning image generation unitgenerates the attention region learning image by including the dust cut out by the noise detection unitin the non-attention area.
As described above, in the sixth embodiment, since the noise such as dust included in the attention region is detected and the detected noise is included in the non-attention region to generate the attention region learning image, it is possible to extract the noise included in the detection target image and generate a new attention region learning image. Therefore, the attention region learning image can be diversified, and the length measurement of the length measurement target region can be accurately performed even when the detection target image includes noise.
22 FIG. 22 FIG. 1 FIG. 1 1 25 26 d d is a block diagram illustrating a schematic configuration of an image processing apparatusaccording to a seventh embodiment. The image processing apparatusaccording to the seventh embodiment illustrated inincludes a length measurement value determination unitand a second learning image addition unitin addition to the block configuration of.
25 6 The length measurement value determination unitdetermines whether a length measurement value of the length measurement target region measured by the length measurement unitfalls within an allowable range. The allowable range is determined in consideration of, for example, past length measurement results.
26 25 The second learning image addition unitadds the detection target image including the length measurement target region determined to fall within the allowable range by the length measurement value determination unitto the learning image data set as a new learning image.
23 FIG. 24 FIG. 25 26 25 26 is a diagram for explaining processing operations of the length measurement value determination unitand the second learning image addition unitaccording to the seventh embodiment.is a diagram for explaining the processing operations of the length measurement value determination unitand the second learning image addition unitaccording to the seventh embodiment in more detail.
25 FIG. 25 FIG. 2 FIG. 1 12 14 11 d is a flowchart illustrating a processing operation of the image processing apparatusaccording to the seventh embodiment. The flowchart ofis obtained by adding steps Sto Safter step Sof the flowchart of.
12 25 6 In step S, the length measurement value determination unitcompares the length measurement value of the length measurement target region measured by the length measurement unitwith a design value to determine whether the length measurement value falls within the allowable range.
13 12 In step S, when the determination in step Sis YES, the detection target image including the length measurement target region is added to the learning image data set.
14 12 10 8 In step S, when the determination in step Sis NO, the second learning image generation unitregenerates the attention region learning image and the label image to be used for learning the attention region model. Thereafter, the processing of step Sand subsequent steps is performed.
6 As described above, in the seventh embodiment, when the length measurement value of the length measurement target region measured by the length measurement unitfalls within the allowable range, the detection target image including the length measurement target region is added to the learning image data set, thereby easily increasing the number of learning images. In addition, when the length measurement value of the length measurement target region is out of the allowable range, the attention region learning image is regenerated, so that reliability of the length measurement value of the length measurement target region can be improved.
26 FIG. 26 FIG. 1 FIG. 1 1 27 e e is a block diagram illustrating a schematic configuration of an image processing apparatusaccording to an eighth embodiment. The image processing apparatusaccording to the eighth embodiment illustrated inincludes a contour detection unitin addition to the block configuration of.
27 6 27 The contour detection unitdetects a contour of the length measurement target area. Note that the contour includes a shape of the length measurement target area. The length measurement unitperforms length measurement of the length measurement target region based on the contour of the length measurement target region detected by the contour detection unit.
27 FIG. 26 FIG. 27 27 6 6 is a diagram for explaining a processing operation of the contour detection unitin. Since the contour detection unitdetects the contour of the length measurement target region and then the length measurement unitperforms length measurement, the length measurement unitcan easily perform the length measurement.
1 1 1 1 1 1 e e e At least some of the image processing apparatusestodescribed in the above-described embodiments may be configured by hardware or software. In a case where at least some of the image processing apparatusestois configured by software, a program for realizing at least some functions of the image processing apparatusestomay be stored in a recording medium such as a flexible disk or a CD-ROM, and may be read and executed by a computer. The recording medium is not limited to a removable recording medium such as a magnetic disk or an optical disk, and may be a fixed recording medium such as a hard disk device or a memory.
1 1 e In addition, the program for realizing at least some functions of the image processing apparatusestomay be distributed via a communication line (including wireless communication) such as the Internet. Further, the program may be distributed via a wired line or a wireless line such as the Internet or stored in a recording medium in an encrypted, modulated, or compressed state.
While certain embodiments have been described, these embodiments have been presented by way of example only, and are not intended to limit the scope of the disclosures. Indeed, the novel methods and systems described herein may be embodied in a variety of other forms; furthermore, various omissions, substitutions and changes in the form of the methods and systems described herein may be made without departing from the spirit of the disclosures. The accompanying claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of the disclosures.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
September 8, 2025
September 17, 2026
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.