Optical Tracker Configured To Emit Or Reflect Polarized Light

This application claims priority to and all the benefits of European Patent Application No. 24205273, filed on Oct. 8, 2024, the entire contents of which are hereby incorporated by reference.

The present disclosure generally relates to optical trackers. In particular, an optical tracker is configured to emit or reflect polarized light, a system comprising the optical tracker, a method for determining a pose of the optical tracker, a computer program product and a data processing device are presented.

In navigated surgery and similar applications, optical trackers are used for determining positions of objects like a patient anatomy (e.g., a patient bone) or a surgical instrument. Based on the determined positions of multiple objects, relative positions between the objects may be calculated. Further, based on the relative positions, navigation instructions may be determined and used for navigation of, e.g., a surgical instrument held by a surgeon or by a robotic arm.

Optical trackers commonly comprise multiple optically detectable markers in the form of optically passive or active elements that can be captured by a camera system. For example, a passive optical tracker may comprise three or more spheres with reflective surfaces. In the case of an active tracker, the markers may comprise light emitting diodes (LEDs).

To be able to reliably determine the pose of an optical tracker in 6 degrees of freedom (DOF), the markers need to have a minimum distance from each other and at least three markers have to be visible to the camera system. As a result of these constraints, each tracker has a minimum size. In some implementations, this minimum size may lead to detrimental effects during surgery. For example, the tracker may be optically or mechanically obstructive. As a result, the view of surgeon may be obstructed and surgical access to a patient anatomy may become more difficult. Since in most surgical scenarios multiple optical trackers are used, the aforementioned problems are further increased, in particular for cases in which the surgeon has to work on relatively small or adjacent patient anatomies like a patient's vertebrae.

In such or other scenarios, it would also be desirable to increase at least one of tracking accuracy and versatility of the optical tracker.

There is a need for an technique addressing at least some of the aforementioned or other problems. In particular, there is a need for an improved optical tracker.

According to a first aspect, a method for determining a pose of an optical tracker is provided. The optical tracker has at least one planar region configured to emit or reflect polarized light and an optical marker in a fixed spatial relation to the at least one planar region. The method comprises receiving image data indicative of the marker and of polarizations of light received from the at least one planar region. The method further comprises determining, based on the image data and the fixed spatial relation between the marker and the at least one planar region, a pose of the optical tracker.

In some variants, the image data may comprise first image data captured by at least one first camera configured to detect polarizations of light received from the at least one planar region. Additionally or alternatively, the image data may comprise second image data captured by at least one second camera configured to detect the optical marker.

The at least one first camera may have a fixed spatial relation to the at least one second camera. In such a case, the pose of the optical tracker may be further determined based on the fixed spatial relation between the at least one first camera and the at least one second camera. The fixed spatial relation may be known, e.g., based on pre-calibrated camera poses. The cameras may each comprise a tracker allowing to verify the pre-calibrated camera poses and thus the known fixed spatial relation. Additionally or in the alternative, the camera trackers may be used or to determine the poses of the first camera and the second camera and based thereon the fixed spatial relation (e.g., in case the spatial relation is not known).

In some variants, the pose of the optical tracker may be determined within a tracking coordinate system. The tracking coordinate system may be defined relative to (e.g., in a fixed geometric relationship with) at least one of the at least one first camera and the at least one second camera. As an example, the tracking coordinate system may be defined to have its origin in an optic center of the at least one second camera. In other implementations, the tracking coordinate system may be defined relative to a patient tracker or a tracker of a medical imaging device. In some variants, the tracking coordinate system may be a global coordinate system in or relative to which one or more local coordinate systems (e.g., of one or more of the optical trackers presented herein, with at least one planar region configured to emit or reflect polarized light) may be registered.

In some variants, a tracking system for generating the image data may comprise multiple first cameras. In some variants, the tracking system may comprise multiple second cameras. For example, the tracking system may comprise at least one optical stereo camera to capture the at least on optical marker and at least one polarization camera.

In some variants, determining the pose of the optical tracker may comprise determining, based on the image data, a position of the optical marker. The position may be determined within the tracking coordinate system (e.g., defined relative to at least one of the at least one first camera and the at least one second camera). Additionally, determining the pose of the optical tracker may comprises determining, based on the image data, a surface normal for the at least one planar region, and determining, based on the surface normal of the at least one planar region and the position of the optical marker, the pose of the optical tracker.

In some variants, based on a first surface normal of a first planar region, the marker position and the fixed spatial relation between the first planar region and the marker, the pose of the tracker may be determined in 5 DOF excluding one rotational DOF around a rotation axis parallel to the surface normal and running through the optical marker. The 6th DOF may be determined based on at least one of known camera positions, a known or determined starting position of a surface normal, and/or a second surface normal of a second planar region and a fixed spatial relation between the marker and the second planar region.

In particular, the orientation of the tracker may be determined based on the surface normal of the at least one planar region and the position of the tracker may be determined based on the position of the optical marker and the fixed spatial relation to the at least one planar region. For example, the optical marker is spaced apart from the at least one planar region at a known distance and position relative thereto.

Determining the surface normal of the at least one planar region may be based on known techniques for determining a surface normal of an object based on polarization image data indicative of the object. Such techniques are described, for example, by Xue, F., Filin, S., and Jin, W.: “SHAPE RECONSTRUCTION OF HERITAGE ASSETS BY POLARIZATION INFORMATION”, Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLVIII-M-2-2023, 1653-1658, 2023 or in document W. Smith, R. Ramamoorthi and S. Tozza, “Height-from-Polarisation with Unknown Lighting or Albedo” in IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 41, no. 12, pp. 2875-2888, 2019. In detail, the surface normal of the at least one planar region may be determined based on diffuse reflection, specular reflection, or a combination thereof.

In some variants, the optical tracker may have at least two planar regions configured to emit or reflect polarized light. Determining the pose of the optical tracker may then comprise determining, based on the image data, a respective surface normal for the at least two planar regions, and determining, based on the surface normals, an orientation of the optical tracker. The pose of the optical tracker may then be determined based on the determined orientation of the optical tracker and the position of the optical marker.

Using optical trackers with multiple planar regions may increase at least one of the precision and the reliability of the determination of the tracker pose. In particular, providing a tracker with two planar regions allows a determination of the tracker pose in 6 DOF, as explained above. Providing a tracker with three or more planar regions may allow a determination based on multiple respective surface normals to increase the precision of the pose determination. Further, providing multiple planar regions may increase the flexibility regarding the positioning of the tracker relative to a camera system, since more positions are envisioned in which at least one planar region is in the field of view of a polarization camera.

In some variants, determining the surface normal for the at least one planar region or a respective surface normal for the at least two planar regions may comprise at least one of averaging of multiple surface normals determined for a particular one of the one or more planar regions and fitting a surface normal (e.g., pixel-by-pixel) to a basic model of the surface normal. The basic model may be based on known marker geometries. Averaging of multiple surface normals or fitting a surface normal to a basic model thereof may further increase the precision of the pose determination and may be performed based on known mathematical averaging and fitting techniques.

In some variants, the image data may be indicative of at least two optical trackers, each of the at least two optical trackers having at least two planar regions configured to emit or reflect polarized light, and an optical marker in a fixed spatial relation relative to the at least two planar regions. The orientations of the at least two planar regions relative to each other may be different for the optical trackers. For example, the two planar regions may have different angles relative to each other for the different optical trackers. The method may further comprise differentiating between (e.g., identifying) the optical trackers based at least in part on the orientation of the at least two planar regions of the respective trackers.

In some scenarios, providing optical trackers with different relative orientations between at least two planar regions thereof allows for an easy identification of the respective trackers. In some variants, each tracker may have a unique relative orientation of planar regions allowing for an unambiguous identification of the tracker based on the respective surface normals of the at least two planar regions.

In some variants, the image data may be indicative of at least two optical trackers, each of the at least two trackers having at least one planar region configured to emit or reflect polarized light and an optical marker in a fixed spatial relation relative to the at least one planar region. The planar regions may be color-coded and/or the optical markers may be active markers, wherein each active marker may have a unique emission characteristic (e.g., in regard to the emitted wavelength or in regard to a temporal emission pattern). The method may further comprise differentiating (e.g., identifying) the optical trackers based at least in part on the color-coded planar regions and/or the emission characteristics of the active markers.

Additionally or alternatively to the color-coding, the regions may be coded based on optically discernable patterns.

The techniques for identifying the trackers as described herein may be combined to increase the reliability of the identification compared to using a single one of the identification techniques or to provide more flexibility to a user, i.e., by providing a choice of identification techniques to be used.

In some variants, the at least one planar region may be configured to reflect light emitted from a light source at a known location (e.g., known within the tracking coordinate system). In such an implementation, the step of determining the pose of the tracker may be based at least in part on the known position of the light source.

The light emitted from the light source may be unpolarized and may become polarized due to the reflection from the at least one planar region. The at least one region may thus comprise or be made of dielectric, i.e., non-metallic material.

According to a second aspect, a computer program product is provided. The computer program comprises instructions which, when the program is executed by a processor, cause the processor to carry out the steps of any variant of the method described herein.

According to a third aspect, a data processing device is provided. The device comprises a processor configured to perform the steps of any variant of the method described herein.

According to a fourth aspect an optical tracker is provided. The tracker comprises at least one planar region configured to emit or reflect polarized light and an optical marker in a fixed spatial relation relative to the at least one planar region.

The optical marker may be an optically passive (e.g., light reflecting) or an optically active (e.g., light emitting) element. In one implementation, the optical tracker comprises only a single optical marker. The single optical marker may be arranged in a central portion of the optical tracker. In other implementations, the optical tracker may comprise two, three, four or more optical markers spaced apart from each other.

In some variants, the at least one planar region may comprise (e.g., may be made of) dielectric material so that the reflected light may become polarized due to the reflection.

In some variants, the optical tracker may comprise at least two (e.g., two, three or four) planar regions configured to emit or reflect polarized light, the at least two planar regions having different orientations. Optionally, the orientations of the at least two planar regions may differ between 5 and 30 degrees, e.g., relative to the marker. Additionally or alternatively, the at least two planar regions are adjacent to each other, thus allowing a compact tracker design.

In some variants, the optical tracker may further comprise an interface configured to couple the optical tracker to a surgical object. For example, the interface may be a clamp or an adhesive. In other examples, the interface may comprise a thread configured to engage with a counter-thread or to receive a screw. In general, the interface may be any known interface for attaching a tracker to an object. The interface may be in a fixed spatial relation to the optical marker and thus also the at least one planar surface. The fixed spatial relation may be known based on manufacturing data.

According to a fifth aspect, a tracking system comprising at least two optical trackers as described herein is provided. At least one of the following tracker differentiation criteria is implemented in the tracking system: i) each of the at least two optical trackers has at least two planar regions configured to emit or reflect polarized light, wherein orientations of the at least two planar regions relative to each other are different for the at least two optical trackers, ii) the optical marker of each of the at least two optical trackers is an active marker, wherein the optical markers of the at least two optical trackers have different emission characteristics, and iii) the at least one planar region of each of the at least two optical trackers is color-coded, wherein the color-coding of the respective planar region is different for the at least two optical trackers. Of course, two or more of these criteria can be combined as needed.

According to a sixth aspect, a tracking system is provided. The tracking system comprises an optical tracker as described herein, a first camera configured to detect polarizations of light reflected from the at least one planar region, and a second camera configured to detect the optical marker.

The system may further comprise a light source configured to emit light towards the at least one planar surface. The light source may be located at a distance from the tracker that is multiple times larger than the dimensions of the tracker, so that the light source may approximately be assumed as a point light source when determining the surface normal based on light emitted by the light source and reflected by the at least one planar region. The light source may be configured to emit unpolarized light.

In the following description of exemplary embodiments, the same reference numerals are used to denote the same or similar components.

1 FIG. 1 FIG. 100 100 200 shows an embodiment of a tracking system. The tracking systemofcomprises an object to be tracked. In the shown example, the object is a patient having a spinecomprising multiple vertebrae that are to be tracked for the purpose of a navigated spinal intervention. In other examples, other objects may be tracked. In particular, each object to which a tracker is attachable to may be tracked.

100 210 300 300 300 210 300 300 300 210 300 300 300 a b c a b c a b c The systemcomprises multiple optical trackers,,,attached to the patient. The dimensions of the trackers,,,may not be representative of the real dimensions of the trackers,,,and are chosen for illustrative purposes.

210 300 300 300 210 220 300 300 300 210 210 200 300 300 300 a b c a b c a b c 1 FIG. 1 FIG. The various optical trackers,,,illustrated incomprise a single optical patient trackerwith three or more optical markersand multiple optical vertebra trackers,,. In the illustration of, the patient trackeris shown as being attached to a vertebra. In other implementations, the patient trackermay be attached to a patient's skin in a vicinity of the spine, e.g., as a frame within which the optical vertebra trackers,,are provided (such as Stryker's SpineMask®).

300 300 300 310 320 310 330 300 300 300 200 330 a b c a b c Each of the optical vertebra trackers,,comprises at least one planar region, a single optical markerwith a fixed spatial relation to the at least one planar region, and an interfaceconfigured for attaching each of the trackers,,to the objectto be tracked (here: a vertebra). In the present configuration, the interfaceis configured as a clamp configured to the attached to the spinous process.

310 300 300 300 400 100 400 310 400 a b c 1 FIG. The at least one planar regionof the trackers,,shown incan be made of dielectric material and configured to reflect light emitted from a light sourcecomprised by the system. The light emitted by the light sourcemay be unpolarized light and become polarized due to the reflection at the at least one planar region. The light emitted by the light sourcemay be visible light or infrared light.

300 300 300 310 320 310 a b c 2 3 FIGS.and Different variants of the trackers,,comprising at least one planar regionand a single optical markerwith a fixed spatial relation to the at least one planar regionare shown in detail in.

2 FIG. 300 310 320 330 300 330 a a In, a first embodiment of an optical trackeris shown comprising a single planar region, a single optical markerand an interfaceconfigured for attaching the trackerto an object to be tracked. It is to be noted that the interfaceis only schematically illustrated and may be realized in accordance with the shape of the surgical object to which is will be attached.

2 FIG. 3 FIG. 310 320 330 310 320 330 0 2 0 2 310 320 330 0 2 300 a As becomes apparent from, the planar region, the single optical markerand the interfaceare spaced apart from and in fixed spatial relations to each other. Each of the planar region, the markerand the interfacedefines a respective coordinate system CSto CSas illustrated in, with an origin of the respective coordinate system CSto CSbeing located in a center of the planar region, the markerand the interface. Due to the fixed (and known) spatial relations, only one of the coordinate systems CSto CSmay need to be registered, as a local coordinate system, with or within a global (e.g., a tracking) coordinate system to enable the trackerto be used for tracking and/or navigation applications.

2 FIG. 1 FIG. 1 FIG. 1 FIG. 320 310 330 320 400 400 400 In the variant of, the optical markeris a reflective marker and located approximately in the middle between the planar regionand the interface. The optical markeris specifically configured to reflect light in the infrared spectrum. This light may be generated by the light sourceillustrated inor a separate light source. The separate light source may be configured to emit light in the infrared spectrum. In some variants, for example, the separate light source may be configured to emit light in the same spectrum as the light sourceillustrated in. In other variants the separate light source and the light sourceillustrated inmay be configured to emit light in different spectra.

3 FIG. 3 FIG. 300 300 300 300 300 300 300 310 310 310 310 310 310 310 310 310 310 310 310 310 310 310 310 310 310 310 310 310 310 310 310 300 300 310 310 310 310 300 300 310 310 310 310 310 310 310 310 2 5 b c b c a b c a b c d a b c d a b c d a b c d a b c d a b c d b c a b c d b c a b c d a b c d In, a second embodiment of an optical tracker,is shown. The second optical tracker,differs from the first optical trackerin that the second optical tracker,comprises more than one, namely four planar regions,,,. Each of the planar regions,,,is located adjacent to two other of the planar regions,,,. The planar regions,,,are angled relative to each other. The angles between the planar regions,,,may be chosen so that each of the regions,,,defines a unique surface normal. For different trackers,, the respective angles between the planar regions,,,may differ so that the trackers,may be differentiated (and possibly uniquely identified) based on the respective angles between the planar regions,,,. Further, each of the planar regions,,,shown ondefines a respective coordinate system CSto CS.

320 300 300 310 310 310 310 320 310 310 310 310 1 5 320 310 310 310 310 b c a b c d a b c d a b c d. A single passive (i.e., reflective) optical markeris located in the center of the optical tracker,and surrounded by the planar regions,,,. There exists a fixed (and known) spatial relation between the markerand the planar regions,,,. This fixed spatial relation may be defined between the origins of the respective coordinate systems CSto CS. These origins may be defined to lie in the geometric center of each of the markerand the planar regions,,,

310 310 310 310 310 310 310 310 310 310 310 310 310 310 310 310 300 300 310 310 310 310 310 a b c d a b c d a b c d a b c d b c a b c d 3 FIG. Other configurations of planar regions,,,that differ from the configuration shown inare contemplated. In particular, at least one of a different number of planar regions,,,, different geometries of the planar regions,,,, and different orientations of planar regions,,,relative to each other may be chosen for differentiating different optical trackers,. For example, a tracker (not shown) may comprise two or more planar regions,,,,arranged in a row. Further, the planar regions may be optically coded by color and/or an optical pattern (not shown).

1 FIG. 1 FIG. 100 500 510 500 320 210 300 300 300 510 310 310 310 310 310 520 500 220 320 520 500 a b c a b c d Returning to, the systemfurther comprises at least one first cameraand at least one second camera. The at least one first camerais a stereo camera configured for tracking the optical markersof the trackers,,,, for example in the infrared spectrum. The second camerais a polarization camera configured for detecting the polarized light reflected from (or actively emitted by) the planar regions,,,,. Also illustrated inis a tracking coordinate systemin which the at least one first cameratracks the optical markers,. In some variants, the tracking coordinate systemhas its origin in an optical center of the at least one first camera.

500 510 600 100 600 610 500 510 220 320 310 310 310 310 310 500 510 600 500 510 600 a b c d The at least one first cameraand the at least one second cameraare communicatively connected to a computing devicecomprised by the tracking system. The computing systemcomprises a processorconfigured to receive image data generated by the at least one first cameraand the at least one second cameraand indicative of the optical markers,and the polarization of the light reflected or emitted from the planar surfaces,,,,. In the shown example, the cameras,are directly connected to the computing device. In other variants, the cameras,may be connected to a server, a cloud or any other data storage device and the computing devicemay receive the image data from the server, the cloud, or the other data storage device.

600 300 300 300 700 300 300 300 a b b a b c 4 FIG. 4 FIG. 2 3 FIGS.and The computing devicemay process the received image data for determining a pose of at least one of the optical trackers,,will now be discussed with reference to.shows a flow diagramof a method for determining a pose of the optical tracker,,as shown in.

710 320 310 310 310 310 310 500 510 a b c d 1 FIG. In step, the method comprises receiving image data indicative of a markerand of polarizations of light received from at least one planar region,,,,. The image data may have been generated by the at least one first cameraand the at least one second cameraas explained above with reference to.

720 320 310 310 310 310 310 300 300 300 a b c d a b c. In step, the method comprises determining, based on the received image data and the fixed spatial relation between the markerand the at least one planar region,,,,, a pose of the optical tracker,,

300 300 300 600 310 310 310 310 310 510 a b c a b c d To determine the pose of the optical tracker,,, the computing devicedetermine a respective surface normal for each of the planar regions,,,,based on the image data received from the second camera. The respective surface normals may be determined based on known mathematical techniques as described above. To increase the accuracy of the determination of a respective surface normal, multiple surface normals may be determined initially, e.g., at different positions of a respective planar region (e.g., at different pixels in the image data), and averaged. Additionally or alternatively, a surface normal may be fitted pixel-by-pixel to a model of the surface normal. The model may be previously known, e.g., based on manufacturing data.

520 500 510 500 510 The surface normal may be determined in the coordinate system defined by the respective planar region(s). The respective coordinate system(s) may then be registered with the tracking coordinate system, e.g., a coordinate system defined based on a position of one of the cameras,, wherein the cameras,may have a known fixed spatial relation to each other.

320 1 500 520 Further, the position of a particular optical marker(i.e., of its coordinate system CS) may be determined based on the image data received from the first cameraand within the tracking coordinate system. The registrations between any coordinate systems described herein may be based on common image registration techniques known in the art.

300 300 300 520 320 320 310 310 310 310 310 300 300 300 320 520 a b c a b c d a b c The pose of a respective optical tracker,,may then be determined in the tracking coordinate systembased on the respective one or more surface normals, the position of the markerand the fixed spatial relations between the markerand each of the one or more planar regions,,,,. For example, the position of the respective tracker,,may be determined based on the position of its markerand its orientation may be unambiguously determined based on at least one of the orientations of at least two associated surface normals or based on the orientation of one surface normal together with a predefined or determined camera position or a starting position of the surface normal within the tracking coordinate system.

300 300 300 300 300 300 320 300 300 320 300 a b c a b c a c a c a c As has become apparent from the above description of exemplary realizations of the present disclosure, a pose of a tracker,,can accurately be determined based on image data indicative of the polarization of light reflected from a planar surface of the tracker,,and on the position of in particular less than three, e.g., a single optical markerper tracker-. Since multiple markers are no longer necessarily needed, the trackers-described herein may become more compact than conventional optical trackers with three of more spaced apart optical markers. As such, the optical and mechanical obstructions inherent to conventional optical trackers can be reduced, which has various advantages. For example, access to a surgical site may be improved and accidental mechanical collisions with markersmay be reduced. As a result, the number of re-registrations may be reduced, which may lead to a reduced duration of a surgery and thus to a reduced health risk for a patient. Further a surgeon may have a better view of a surgical site when using the optical trackers-, which may also reduce the health risk for the patient.

320 310 310 500 300 a d a c In sum, the technique presented herein allows determining a tracker pose in 6 DOF with high accuracy, even if only one optical markerand the at least one planar region,-are visible to a camera systemconfigured for generating the received image data. As a result, the optical tracker-may have a more compact design compared to existing optical trackers having at least three optical markers so that an optical and mechanical obstruction due to the use of one or more trackers is reduced.

While the present invention has been described with reference to exemplary embodiments, the invention can also be implemented in different forms.