Method for Obtaining a Tilt Series of Images of a Sample at a Plurality of Tilt Angles

This application claims priority from European patent application no. EP24169352.2, filed Apr. 10, 2024. The entire disclosure of EP24169352.2 is incorporated herein by reference.

The present disclosure relates to a method and a charged particle microscope for obtaining a tilt series of images based on exposure of a region of interest (ROI) of a sample to a charged particle beam (CPB) at a plurality of tilt angles. More specifically, it pertains to a system that utilizes a charged particle optical column, a sample holder, a charged particle detector, and an imaging system to generate detailed images of samples for acquiring said tilt series of images.

In various scientific and industrial applications, it is often necessary to examine the microscopic structure and composition of samples. Traditional optical microscopes have limitations in resolution and are unable to provide sufficient detail for certain types of samples. To overcome these limitations, charged particle microscopes have been developed.

Charged particle microscopy is a well-known and increasingly important technique for imaging microscopic objects, particularly in the form of electron microscopy. Historically, the basic genus of electron microscope has undergone evolution into a number of well-known apparatus species, such as the Transmission Electron Microscope (TEM), Scanning Electron Microscope (SEM), and Scanning Transmission Electron Microscope (STEM), and also into various sub-species, such as a so-called “dual-beam” apparatus (e.g. a FIB-SEM) that additionally employs a Focused Ion Beam (FIB), allowing supportive activities such as ion-beam milling or Ion-Beam-Induced Deposition (IBID). The skilled person will be familiar with the different species of charged particle microscopy.

These charged particle microscopes generally comprise several components, which will be explained below.

Firstly, a charged particle microscope comprises a charged particle optical column that is employed to direct a charged particle beam onto the sample. This optical column is responsible for directing and controlling the charged particles to the sample to ensure accurate imaging.

Additionally, the charged particle microscope comprises a sample holder for securely holding a sample in place during the imaging process. The sample holder is generally designed to accommodate various sample sizes and shapes, providing stability and reproducibility in positioning.

Furthermore, the charged particle microscope comprises a charged particle detector to capture charged particles that interact with the sample. The charged particle detector converts the energy or intensity of the charged particles into electrical signals, which can be further processed for image generation.

The charged particle microscope also includes an imaging system that is arranged for processing signals from the charged particle detector. The imaging system receives data from the charged particle detector and generates an image signal based on the collected information.

Transmission electron microscopes (TEMs) can be used to obtain high resolution images that reveal important details of many kinds of samples, including biological samples. In electron beam tomography, multiple images of a sample are needed for image reconstruction.

Typically, in tomography, tracking the field-of-view (FOV) is a time-consuming process. There are two main methods of tracking: ‘tracking after’, where images that have already been captured are used, and ‘tracking before’, where images are captured at a nearby area before the main acquisition. However, both these methods have their drawbacks. ‘Tracking after’ can fail when the imaging dose is low, leading to low signal-to-noise ratio (SNR). ‘Tracking before’ is more reliable but adds significant time to the process.

In a specific method of tomography called fast-incremental single-exposure (FISE) acquisition, the camera is always on, and there's no chance for tracking at all. This method assumes that the area of interest will remain in the FOV, which can be difficult or even impossible in certain situations. This FISE acquisition method is described in further detail in “Rapid tilt-series acquisition for electron cryotomography.” Journal of Structural Biology. 2019 Feb. 1; 205(2):163-169.

It is an object of the disclosure to provide an improved method, in particular a method that improves the tracking of the field of view in tilt series acquisitions.

To this end, the disclosure provides a method as described in claim. The method as defined herein comprises the step of obtaining a tilt series of images based on exposure of a region of interest of a sample to a charged particle beam at a plurality of tilt angles. Additionally, the method includes changing the tilt angle at a tilt angle speed while acquiring the tilt series of images, as well as the step of tracking a position of at least a part of the sample during the changing of the tilt angle. Further, the method comprises the step of changing, based on the step of tracking the position, the relative position of the sample with respect to the charged particle beam to keep the region of interest within a field of view.

As defined herein, the method further comprises the step of starting a recovery process based on a detection of an anomaly in the tracking of the position of the sample.

The step of tracking the position is performed during the acquisition process of acquiring the tilt series, and thus the tracking allows the changing of the position based on the tracking of the position to correct for any changes, such as shifts, that might occur during the acquisition process. The tracking region thus allows for a continuous (or semi-continuous) way of tracking the position of the sample during the acquisition of the tilt series images. As defined herein, the tracking of the position may be monitored so that upon a detection of an anomaly in the tracking of the position of the sample, a recovery process may be started and performed. This allows tracking of the position of the sample to take place during the actual image acquisition process, with reduced risk of acquiring undesired images as an encountered anomaly may be corrected. With this the chances of a successful acquisition of a tilt series of images are increased.

At least some images of the tilt series of images may be used in a step of producing a tomographic image of a sample volume related to the region of interest. Thus, the method as defined herein also increases the chance of a successful tomographic image.

With this, the object as defined herein is achieved.

Further advantageous embodiments will be described below.

In an embodiment, the method comprises the step of determining a tracking accuracy. The tracking accuracy may be based on changes in the tracking of the position of at least a part of the sample during the changing of the tilt angle. The tracking accuracy may be based on a comparison of subsequent images, as will be explained in more detail later. In any event, an image can be used for determining a tracking accuracy. Tracking accuracy may include, for example, the position of the region of interest is within the image. Tracking accuracy may include, additionally or alternatively, a change in the position of the region of interest within subsequent images. Tracking accuracy may contain, in an embodiment, a ‘goodness of match’. This may, for example be used when searching for a feature in an image. This may additionally or alternatively be used when multiple matches exist with similar ‘goodness of fit’. In this latter case a comparison with the last known position of the feature may also be used to decide which is the correct match.

In an embodiment, the method comprises the step of starting the recovery process based on the tracking accuracy being outside of a target range. This may include starting the recovery process in case the position of the region of interest changes, in particularly whilst exceeding a certain threshold value, within subsequent images.

As indicated above, the method may comprise the step of using an image of the sample for detecting the anomaly. In other words, the detection of the anomaly can be based on analysis of an image of the sample. Although image-based feedback may be done in tilt tomography, it is particularly beneficial in continuous tilt tomography (CTT) as it can be used to ensure that an area of interest (such as the region of interest) does not move out of the field of view of the camera. This image-based feedback may also be referred to as image-based runout correction.

The method may comprise the step of interpreting an image stream that is captured during the acquisition of the tilt series images. The method may comprise the step of comparing subsequent images, for example coming from the image stream. From the comparison, an anomaly can be detected and subsequently the recovery process can be started.

The method may comprise the step of tracking the position of the sample based on sensor information. The sensor information may relate to the sample holder (i.e. sample stage). For example a movement sensor may be mounted close to the sample or close to the sample holder and may be arranged for giving information on x,y,z, displacements or movements of the sample, which sensor information may then be used for tracking the position of the sample during tilt changes.

In an embodiment, the recovery process comprises one or more of the following steps:

The blanking of the charged particle beam may be used for preventing exposure to the sample, in particular to a region of interest, when an anomaly is detected. Thus as part of the recovery process, the blanking of the beam may preserve the sample at a region of interest, at least until the anomaly is resolved.

In an embodiment, the recovery process comprises the step of changing the part of the sample that is being tracked.

The recovery process may include changing the magnification setting of the charged particle microscope, so that a known feature can be found in a larger field of view can be found.

In an embodiment, the method comprises the step of tracking a part of the sample that is outside of the region of interest. This may be based on images that are acquired at a different part of the sample. The method may comprise exposing a tracking region that is substantially outside of the region of interest (ROI). This ensures that the tracking process does not interfere with the imaging of the ROI, leading to less damage for the ROI and more accurate tomographic image reconstruction. By using a tracking region outside of the region of interest, it is possible to track the field of view during the acquisition of the tilt series images, leading to faster acquisition times for the tilt series with improved accuracy in view of the tracking ability during the acquisition.

In an embodiment, this step of tracking a part of the sample outside of the region of interest may be part of the recovery process. This ensures that the region of interest is not exposed after an anomaly is detected, and thus ensures the quality of the sample at the region of interest is maintained.

In an embodiment, the method comprises the step of resuming obtaining the tilt series of images. The resuming of obtaining the tilt series of images can be seen as a concluding step of the recovery process.

In an embodiment, the method may comprise the step of continuously tracking the position of the sample during obtaining the tilt series. This may comprise exposing a tracking region of the sample. The continuous exposure allows for real-time tracking of the sample, enabling precise monitoring and a timely activation of the recovery process (i.e. with minimal delay) whenever an anomaly is detected.

In an embodiment, the method can be applied to both step-by-step tomography and continuous tilt tomography. This versatility allows for the method to be utilized in various imaging techniques, providing flexibility in experimental setups.

According to an aspect, a charged particle microscope as defined in claimis provided.

The charged particle microscope as defined herein includes a charged particle optical column for directing a charged particle beam onto a sample, a sample holder for holding a sample, and a charged particle detector and an imaging system for generating an image signal based on information from the charged particle detector. Additionally, the charged particle microscope is arranged for obtaining a tilt series of images based on exposure of a region of interest (ROI) of a sample to the charged particle beam (CPB) at a plurality of tilt angles. Furthermore, the charged particle microscope is arranged for changing the tilt angle at a tilt angle speed while acquiring the tilt series of images. Additionally, the charged particle microscope is arranged for tracking a position of at least a part of the sample during the changing of the tilt angle. The charged particle microscope may be arranged for changing the relative position of the sample with respect to the charged particle beam to keep the region of interest (ROI) within a field of view (FOV).

The charged particle microscope may be arranged for, or be part of a system that is arranged for, producing a tomographic image of a sample volume (that may be related to a region of interest of the sample) based on at least some images of the obtained tilt series.

As defined herein, the charged particle microscope M is arranged for starting a recovery process based on a detection of an anomaly in the tracking of the position of the sample.

Advantages of the ability of starting a recovery process have been elucidated with respect to the method above.

In an embodiment, the charged particle microscope is arranged for performing any of the embodiments of the method as disclosed herein.

For providing real-time feedback to the microscope, the imaging system of the charged particle microscope may have a first charged particle detector output interface for outputting a first data stream of data related to said charged particle detector, as well as a second charged particle detector output interface for outputting a second data stream of data related to said charged particle detector. These two charged particle detector output interfaces can be used for providing real-time feedback (from one output interface) while maintaining high data quality and reliability (from the other output interface). By having two charged particle detector output interfaces that are arranged for providing different data steams compared to each other, the system allows to have two separate data streams that can be used for different purposes.

As an example, the first data stream may be designed to prioritize data quality and reliability, ensuring accurate and dependable data output. Thus, the first data stream can be optimized for a high image quality standard that allows for producing the tomographic image of the sample volume related to said region of interest based on at least some images of the obtained tilt series.

The second data stream may be designed to be optimized for low latency and/or fixed latency. This allows for images of the tracking region to be obtained and used for providing real-time feedback to the charged particle system. The real-time feedback can be used for providing a relative adjustment between the sample and the charged particle beam, for example by means of a stage movement and/or a deflection of the charged particle beam.

As defined herein, latency relates to the time difference between the moment a charged particle hits the charged particle detector (i.e. charged particle camera) and the moment the information from that charged particle is ready to be used by any device that is positioned downstream of the output interface. An example of such a device that is/can be positioned downstream of the output interface, is given by a data storage, a feedback processing device, a controller, etcetera. Latency can relate to the average time difference or the maximum time difference.

In an embodiment that is suitable for charged particle microscopy, the latency is in the order of half the readout integration time of the charged particle detector. If the charged particle detector, which can be a TEM camera, for example, runs at 500 fps, every pixel readout takes 2 ms. Low latency means that the time it takes for the data to come from the charged particle detector and be output through said second output interface (after which it can be used by a further processing device, for example) is in the order of the pixel readout too. This means that the low latency should be in the order of ms. Based on this example, the average latency can be about 1 ms, and the maximum latency can be about 2 ms. Higher latency, even though not optimal, can still give beneficial effects for providing real-time feedback.

As indicated above, the maximum low latency output can be related to the frame rate of the charged particle detector. For a frame rate of 500 fps, the maximum low latency is preferably in the order of 1/500=2 ms. For a lower frame rate (e.g. 250 fps) the maximum low latency can be 1/250=4 ms.

As indicated above, the average low latency output can be related to the frame rate of the charged particle detector. For a frame rate of 500 fps, the average low latency is preferably in the order of 0.5*1/500=1 ms. For a lower frame rate (e.g. 250 fps) the average low latency can be 0.5*1/250=2 ms.

This average and/or maximum latency can be used for providing real-time feedback to the charged particle microscope, with which the field of view can be accurately tracked during acquisition of the tilt series.

(not to scale) is a highly schematic depiction of an embodiment of a charged-particle microscope M according to an embodiment. More specifically, it shows an embodiment of a transmission-type microscope M, which, in this case, is a TEM/STEM (though, in the context of the current disclosure, it could just as validly be a SEM (see), or an ion-based microscope, for example). In, within a vacuum enclosure, an electron sourceproduces a beam B of electrons that propagates along an electron-optical axis B′ and traverses an electron-optical illuminator, serving to direct/focus the electrons onto a chosen part of a sample S (which may, for example, be (locally) thinned/planarized). Also depicted is a deflector, which (inter alia) can be used to effect scanning motion of the beam B.

The sample S is held on a sample holder H that can be positioned in multiple degrees of freedom by a positioning device/stage A, which moves a cradle A′ into which holder H is (removably) affixed; for example, the sample holder H may comprise a finger that can be moved (inter alia) in the XY plane (see the depicted Cartesian coordinate system; typically, motion parallel to Z and tilt about X/Y will also be possible). Such movement allows different parts of the sample S to be illuminated/imaged/inspected by the electron beam B traveling along axis B′ (in the Z direction) (and/or allows scanning motion to be performed, as an alternative to beam scanning). If desired, an optional cooling device (not depicted) can be brought into intimate thermal contact with the sample holder H, so as to maintain it (and the sample S thereupon) at cryogenic temperatures, for example.

The electron beam B will interact with the sample S in such a manner as to cause various types of “stimulated” radiation to emanate from the sample S, including (for example) secondary electrons, backscattered electrons, X-rays and optical radiation (cathodoluminescence). If desired, one or more of these radiation types can be detected with the aid of analysis device, which might be a combined scintillator/photomultiplier or EDX (Energy-Dispersive X-Ray Spectroscopy) module, for instance; in such a case, an image could be constructed using basically the same principle as in a SEM. However, alternatively or supplementally, one can study electrons that traverse (pass through) the sample S, exit/emanate from it and continue to propagate (substantially, though generally with some deflection/scattering) along axis B′. Such a transmitted electron flux enters a projection system (projection lens), which will generally comprise a variety of electrostatic/magnetic lenses, deflectors, correctors (such as stigmators), etc. In normal (non-scanning) TEM mode, this projection systemcan focus the transmitted electron flux onto a fluorescent screen, which, if desired, can be retracted/withdrawn (as schematically indicated by arrows′) so as to get it out of the way of axis B′. An image (or diffractogram) of (part of) the sample S will be formed by projection systemon screen, and this may be viewed through viewing portlocated in a suitable part of a wall of enclosure. The retraction mechanism for screenmay, for example, be mechanical and/or electrical in nature, and is not depicted here.