Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. An image registration device, comprising: a processor configured to acquire a video configured to include two or more images with different imaging times that include a target object to be imaged by a camera; acquire a simulation image relevant to the target object by extracting the target object from a three-dimensional image; generate a boundary of the simulation image by extracting the boundary of the simulation image and acquire a two dimensional boundary image showing the boundary of the simulation images, wherein the boundary image includes a closed curve or a dashed line which represents an outer circumference of the simulation images; display the video and the simulation image on display unit; and perform first registration between the target object included in the video and the simulation image in response to an instruction of the first registration input by a user, wherein in case where an instruction of at least the first registration is given, the processor displays the boundary image, which is matched manually to the target object, on the display unit instead of the simulation image, wherein in case where the first registration is ended, the processor displays the video and the simulation image on display based on the result of the first registration and receives a designation of an invariant position that does not move during a surgery inputted by the user, and wherein in case where a position of the camera is shifted from the position where the first registration is performed, the processor performs second registration between the target object included in the video and the simulation image based on the invariant position and a result of the first registration.
This invention relates to an image registration system for medical imaging, specifically for aligning real-time video of a target object (e.g., a surgical site) with a pre-acquired three-dimensional simulation image. The system addresses the challenge of maintaining accurate alignment between live video and pre-operative simulation images during surgical procedures, where camera movement or patient motion can disrupt registration. The device includes a processor that acquires a video stream from a camera, capturing multiple images of the target object at different times. It also retrieves a simulation image of the target object by extracting it from a three-dimensional image, then generates a boundary image representing the outer circumference of the simulation image as a closed curve or dashed line. The video and simulation image are displayed simultaneously. A user can manually perform initial registration by aligning the boundary image with the target object in the video. Once completed, the system displays the video and simulation image based on this alignment and allows the user to designate an invariant position—a reference point that remains stationary during surgery. If the camera position shifts, the system automatically performs secondary registration using the invariant position and the initial registration result to realign the video and simulation image. This ensures continuous accurate overlay of the simulation image on the live video, aiding surgical navigation. The system improves precision in medical imaging by combining manual and automated registration techniques.
2. The image registration device according to claim 1 , the processor further configured to: display the target object included in the video and the simulation image after the second registration on the display unit so as to be superimposed on each other.
This invention relates to image registration technology for aligning video images with simulation images, particularly in applications like augmented reality or medical imaging where precise overlay is critical. The problem addressed is the difficulty in accurately registering a target object in a video with a corresponding simulation image, ensuring proper alignment for visualization or analysis. The device includes a processor that performs a first registration to roughly align the target object in the video with the simulation image. This initial alignment may involve feature matching or other techniques to establish a coarse correspondence. The processor then performs a second registration to refine the alignment, improving accuracy by adjusting parameters such as position, rotation, or scale. This second registration may use more advanced methods like iterative optimization or machine learning-based refinement. After registration, the processor displays the target object from the video and the simulation image on a display unit, superimposed on each other. This allows users to visually confirm the alignment or use the combined image for further analysis. The system ensures that the simulation image accurately overlays the target object in the video, enhancing applications like surgical guidance, training simulations, or augmented reality displays. The invention improves upon prior methods by providing a two-step registration process that balances computational efficiency with high precision.
3. The image registration device according to claim 2 , wherein the processor displays the boundary image on the display unit, instead of the simulation image, in response to an instruction to display the boundary image at the time of the second registration.
This invention relates to image registration devices used in medical imaging, particularly for aligning and displaying boundary images of anatomical structures during medical procedures. The problem addressed is the need for precise visualization of anatomical boundaries during registration processes, ensuring accurate alignment between pre-operative images and real-time imaging data. The device includes a processor that performs image registration by aligning a simulation image, such as a pre-operative scan, with a real-time image captured during a procedure. The processor generates a boundary image representing the edges or contours of anatomical structures from the simulation image. During a second registration step, the processor can display the boundary image on a display unit instead of the simulation image in response to a user instruction. This allows medical professionals to focus on the critical boundary information without the distraction of the full simulation image, improving accuracy and efficiency in procedures like surgical navigation or radiation therapy. The boundary image is derived from the simulation image, which may be a 3D model or a 2D slice, and is overlaid or displayed separately to highlight key anatomical features. The display unit may include a monitor or a heads-up display, providing real-time feedback during the procedure. This selective display of boundary information enhances visualization and reduces cognitive load, ensuring precise alignment and better decision-making during medical interventions.
4. The image registration device according to claim 2 , wherein the boundary image is a transparent image.
This invention relates to image registration devices used in medical imaging, particularly for aligning and overlaying medical images from different sources or time points. The problem addressed is the difficulty in accurately registering images when boundaries or edges in the images are not clearly defined, leading to misalignment and inaccurate diagnostic or treatment planning. The device includes a boundary image generator that creates a boundary image from a reference image, where the boundary image highlights the edges or contours of anatomical structures. This boundary image is then used to align a target image with the reference image. The boundary image can be a transparent image, meaning it is overlaid on the target image in a way that allows the underlying target image to remain visible while the boundary contours are superimposed. This transparency enhances visualization and alignment accuracy by providing clear visual guidance without obscuring the target image. The boundary image generator may use edge detection algorithms or other image processing techniques to identify and extract boundaries from the reference image. The alignment process involves adjusting the position, rotation, or scaling of the target image relative to the boundary image to achieve optimal registration. The transparent boundary image ensures that the alignment is precise while maintaining visibility of the target image for further analysis or clinical use. This approach improves the accuracy of image registration in medical applications, such as comparing pre-operative and post-operative images or tracking changes over time.
5. The image registration device according to claim 1 , wherein the boundary image is formed of a broken line.
The invention relates to image registration devices used in medical imaging or other fields where precise alignment of images is required. The problem addressed is the need for accurate and reliable registration of images, particularly when dealing with boundaries or edges in the images. The device includes a boundary image generator that creates a boundary image representing the edges or contours of a target image. This boundary image is used to align or register the target image with a reference image, ensuring proper spatial correspondence between the two. The boundary image is formed of a broken line, which provides a clear and distinct representation of the edges or contours. This broken line structure enhances the accuracy of the registration process by making the boundary more distinguishable from the rest of the image. The broken line may be generated by detecting edges in the target image and converting them into a series of connected line segments. The registration process involves comparing the boundary image with a corresponding boundary image of the reference image and adjusting the target image's position or orientation to achieve alignment. The device may also include a registration unit that performs the alignment by minimizing the difference between the boundary images of the target and reference images. This ensures that the registered image is accurately positioned relative to the reference image. The use of a broken line boundary image improves the robustness of the registration process, particularly in cases where the images contain noise or other distortions. The invention is applicable in medical imaging, satellite imaging, and other fields where precise image alignment is critical.
6. The image registration device according to claim 1 , wherein the boundary image is a transparent image.
The invention relates to image registration devices used in medical imaging, particularly for aligning and overlaying medical images from different sources or modalities. The primary problem addressed is the accurate registration of images to ensure precise alignment, which is critical for diagnostic and treatment planning purposes. Misalignment can lead to errors in diagnosis or treatment, making accurate registration essential. The image registration device includes a boundary image generator that creates a boundary image representing the edges or contours of a target region within a medical image. This boundary image is used to align with another image, such as a live imaging feed or a previously captured image, to ensure proper registration. The boundary image is designed to be transparent, allowing the underlying image to remain visible while the boundary provides alignment guidance. Transparency ensures that the boundary does not obscure critical details in the underlying image, maintaining clarity for medical professionals. The device may also include an image alignment module that adjusts the position, rotation, or scaling of the boundary image to match the target region in the second image. This alignment process ensures that the boundary image accurately overlays the corresponding region in the second image, facilitating precise registration. The transparency of the boundary image allows for real-time adjustments while maintaining visibility of the underlying image, improving the efficiency and accuracy of the registration process. This technology is particularly useful in applications such as image-guided surgery, radiation therapy, and diagnostic imaging, where precise alignment is crucial for effective treatment and diagnosis.
7. The image registration device according to claim 1 , wherein, in a case where the target object includes at least one structure, the simulation image is an image showing three-dimensional shapes of the target object and at least the one structure.
This invention relates to image registration devices used in medical imaging, particularly for aligning simulation images with real-time images during procedures like radiation therapy. The problem addressed is the need for precise alignment of pre-procedure simulation images with live images to ensure accurate targeting of treatment areas while avoiding critical structures. The device generates a simulation image that includes three-dimensional representations of a target object (e.g., a tumor) and at least one adjacent structure (e.g., organs or blood vessels). This simulation image is then registered (aligned) with a real-time image captured during the procedure. The alignment process accounts for positional shifts, ensuring that the treatment corresponds accurately to the pre-planned simulation. The device may use image processing techniques, such as feature matching or deformation modeling, to achieve this registration. The inclusion of both the target object and surrounding structures in the simulation image improves accuracy by providing additional reference points for alignment, reducing errors caused by anatomical movement or deformation. This enhances treatment precision and safety by minimizing damage to healthy tissues.
8. The image registration device according to claim 7 , wherein the boundary image is an image showing three-dimensional shapes of a boundary of the target object and at least the one structure.
This invention relates to image registration devices used in medical imaging, particularly for aligning images of a target object, such as a patient's anatomy, with images of internal structures like tumors or organs. The challenge addressed is accurately registering boundary images—images depicting the three-dimensional shapes of the target object's boundary and at least one internal structure—to ensure precise alignment for diagnostic or treatment purposes. The device includes an image acquisition unit that captures images of the target object and the internal structure, and an image processing unit that generates a boundary image. This boundary image highlights the three-dimensional contours of the target object's boundary and the internal structure, enabling accurate spatial alignment. The device may also include a display unit to visualize the registered images, aiding in medical procedures such as radiation therapy or surgical planning. The invention ensures that the boundary image accurately represents the three-dimensional shapes of both the target object and the internal structure, improving the reliability of image-guided interventions. The system may further incorporate additional imaging modalities or tracking systems to enhance registration accuracy.
9. An image registration method using the image registration device according to claim 1 , comprising: acquiring a video configured to include two or more images with different imaging times that include a target object to be imaged by a camera; acquiring a simulation image relevant to the target object by extracting the target object from a three-dimensional image; generating a boundary of the simulation image by extracting the boundary of the simulation image and acquiring a boundary image showing the boundary of the simulation image, wherein the boundary image includes a closed curve or a dashed line which represents an outer circumference of the simulation image; displaying the video and the simulation image on display unit; performing first registration between the target object included in the video and the simulation image in response to an instruction of the first registration input by a user; displaying the boundary image, which is matched manually to the target object, on the display unit instead of the simulation image, in case where an instruction of at least the first registration is given; while an instruction of at least the first registration-is-given; displaying the video and the simulation image on display based on the result of the first registration and receives a designation of an invariant position that does not move during a surgery inputted by the user, in case where the first registration is ended; and performing second registration between the target object included in the video and the simulation image based on the invariant position and a result of the first registration in case where a position of the camera is shifted from the position where the first registration is performed.
This invention relates to a method for registering images in medical imaging, particularly for aligning video images of a target object (e.g., a surgical site) with a pre-acquired three-dimensional simulation image. The problem addressed is the need for accurate alignment of real-time video with pre-operative 3D models during surgery, especially when the camera position shifts, to ensure precise navigation and tracking of the target object. The method involves acquiring a video containing multiple images of the target object captured at different times by a camera. A simulation image of the target object is generated by extracting it from a 3D image, and its boundary is extracted to create a boundary image, which may be a closed curve or dashed line representing the outer edge of the simulation image. The video and simulation image are displayed together, allowing a user to manually perform a first registration by aligning the simulation image with the target object in the video. Once the first registration is completed, the boundary image is displayed instead of the full simulation image, aiding in fine-tuning the alignment. If the camera position shifts, the user designates an invariant position (a point that remains stationary during surgery) to perform a second registration. This second registration adjusts the alignment based on the invariant position and the initial registration result, ensuring continued accuracy despite camera movement. The method improves surgical navigation by maintaining precise alignment between real-time video and pre-operative 3D models.
10. Non-transitory computer readable medium storing an image registration program causing the image registration device according to claim 1 as a computer to execute: a procedure of acquiring a video configured to include two or more images with different imaging times that include a target object to be imaged by a camera; a procedure of acquiring a simulation image relevant to the target object by extracting the target object from a three-dimensional image; a procedure of generating a boundary of the simulation image by extracting the boundary of the simulation image and acquiring a two dimensional boundary image showing the boundary of the simulation image, wherein the boundary image includes a closed curve or a dashed line which represents an outer circumference of the simulation image, a procedure of displaying the video and the simulation image on display unit; a procedure of performing first registration between the target object included in the video and the simulation image in response to an instruction of the first registration input by a user; a procedure of displaying the boundary image, which is matched manually to the target object, on the display unit instead of the simulation image, in case where an instruction of at least the first registration is given; while an instruction of at least the first-registration is given; a procedure of displaying the video and the simulation image on display based on the result of the first registration and receives a designation of an invariant position that does not move during a surgery inputted by the user, in case where the first registration is ended; and a procedure of performing second registration between the target object included in the video and the simulation image based on the invariant position and a result of the first registration in case where a position of the camera is shifted from the position where the first registration is performed.
This invention relates to a system for registering medical images during surgery to align real-time video from a camera with pre-operative 3D simulation images. The problem addressed is the need for accurate alignment between live surgical video and pre-operative 3D models, especially when the camera position shifts during surgery. The system acquires a video stream containing multiple images of a target object (e.g., a surgical site) captured at different times by a camera. It also acquires a simulation image of the target object extracted from a 3D image. The system generates a boundary image representing the outer circumference of the simulation image as a closed curve or dashed line. The video and simulation image are displayed together, allowing a user to manually perform initial registration by aligning the simulation image with the target object in the video. Once the first registration is complete, the boundary image replaces the simulation image on the display. The user then designates an invariant position (a point that remains stationary during surgery) to assist in subsequent registrations. If the camera position shifts, the system performs a second registration using the invariant position and the results of the initial registration to realign the video and simulation image. This ensures continuous accurate alignment between the live video and pre-operative 3D model, improving surgical navigation and precision.
Unknown
April 28, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.