Optoelectronic module comprising ultrasound transducer and imaging apparatus using the same

The present invention generally relates to an optoelectronic module with an ultrasound transducer and an imaging apparatus such as a medical or industrial endoscope using such optoelectronic module. More particularly, the present invention is related to a small optoelectronic module including an ultrasound transducer, an image sensor, an illumination component, and other optional component(s).

Ultrasonic images, also known as sonograms, are made by sending pulses of ultrasound into tissue using a transducer or a probe. Ultrasounds are sound waves with frequencies which are higher than those audible to humans (>20,000 Hz). The ultrasound pulses echo off tissues with different reflection properties and are recorded and displayed as an image. Medial ultrasound is conventionally used for creating images of internal organs as well as other body structures such as tendons, muscles, joints, and blood vessels. Although medical ultrasound commonly uses transducers designed to be used externally, such as through the lower abdominal wall in the case of gynecologic ultrasonography, occasionally the ultrasonic transducer is configured for insertion into an internal organ or other structure. One such example is a sonohysterogram (also known as sonohysterography or saline infusion sonography) procedure which is used for imaging of the uterus. The procedure includes inserting fluid and an ultrasound probe into the uterus and can provide sonographic images of uterine structures. However, the sonohysterogram procedure is performed “blind” or without any live visual aid during the insertion of the ultrasound probe.

An angioscope such as a coronary angioscope is a flexible endoscopic device used to visually examine the interior of blood vessels. It's inserted into an artery and provides real-time, high-resolution imaging of the vessel walls. This is especially useful for diagnosing vascular conditions like plaque buildup or blood clots. However, an angioscope cannot examine body structures surrounding the blood vessel.

Moreover, endoscopes of small size are desired in many industrial and medical applications. For example, when natural orifices and lumens of a human body are small, small endoscopes are required for insertion through such orifices and lumens to target locations within the body. For single incision laparoscopy, smaller endoscopes are preferred to provide an inside-the-body view of the surgical site, particularly when the incision itself is of minimal dimensions. Sometimes, patients may feel irritating when an endoscope is being inserted into his or her body, and a smaller endoscope may mitigate such unpleasant experience and may minimize trauma to the patient. Moreover, a physician may improve diagnostic and procedural protocols with a smaller endoscope. For example, transnasal endoscopy may sometimes replace trans-oral endoscopy.

To meet the small-size requirement, an apparent solution is to decrease the size of each individual component within the endoscope, for example, using a smaller size camera or a smaller size fiber bundle. However, there are limits to how much reduction can be achieved, and each size reduction has its cost in terms of performance and assembly complexity.

Advantageously, the present invention provides a new solution to solve all the above problems.

One aspect of the present invention provides an optoelectronic module which may be optionally bundled with an external working channel. The optoelectronic module includes a housing (or a cannula) with a longitudinal axis, an image sensor secured at a distal end of the housing, and an ultrasound transducer (or ultrasound probe) within the housing. The ultrasound transducer is positioned in its entirety behind the image sensor along the longitudinal axis.

Another aspect of the invention provides an imaging apparatus for imaging the interior surface of, and external environment surrounding, a tubular structure. The imaging apparatus includes (1) the optoelectronic module as described above for insertion into the tubular structure; (2) a first receiving device outside the tubular structure for receiving the signals from the image sensor; and (3) a second receiving device outside the tubular structure for receiving the signals from the ultrasound transducer.

The above features and advantages and other features and advantages of the present invention are readily apparent from the following detailed description of the best modes for carrying out the invention when taken in connection with the accompanying drawings.

In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It is apparent, however, to one skilled in the art that the present invention may be practiced without these specific details or with an equivalent arrangement.

Where a numerical range is disclosed herein, unless otherwise specified, such range is continuous, inclusive of both the minimum and maximum values of the range as well as every value between such minimum and maximum values. Still further, where a range refers to integers, only the integers from the minimum value to and including the maximum value of such range are included. In addition, where multiple ranges are provided to describe a feature or characteristic, such ranges can be combined.

It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only, and is not intended to limit the scope of the invention. For example, when an element is referred to as being “on”, “connected to”, or “coupled to” another element, it can be directly on, connected or coupled to the other element or intervening elements may be present. In contrast, when an element is referred to as being “directly on”, “directly connected to”, or “directly coupled to” another element, there are no intervening elements present.

Throughout the specification and claims, the following terms take the meanings explicitly associated herein, unless the context clearly dictates otherwise. The phrase “in one embodiment” does not necessarily refer to the same embodiment, although it may. Furthermore, the phrase “in another embodiment” does not necessarily refer to a different embodiment, although it may. Thus, as described below, various embodiments of the invention may be readily combined without departing from the scope or spirit of the invention.

In addition, as used herein, the term “or” is an inclusive “or” operator, and is equivalent to the term “and/or,” unless the context clearly dictates otherwise. The term “based on” is not exclusive and allows for being based on additional factors not described, unless the context clearly dictates otherwise. In addition, throughout the specification, the meaning of “a,” “an,” and “the” include plural references. The meaning of “in” includes “in” and “on.”

Additionally, all numerical values are “about” or “approximately” the indicated value, and take into account experimental error and variations that would be expected by a person having ordinary skill in the art. It should be appreciated that all numerical values and ranges disclosed herein are approximate values and ranges, whether “about” is used in conjunction therewith. It should also be appreciated that the term “about,” as used herein, in conjunction with a numeral refers to a value that may be ±0.01% (inclusive), ±0.1% (inclusive), ±0.5% (inclusive), ±1% (inclusive) of that numeral, ±2% (inclusive) of that numeral, ±3% (inclusive) of that numeral, ±5% (inclusive) of that numeral, ±10% (inclusive) of that numeral, or ±15% (inclusive) of that numeral. It should be appreciated that when a numerical range is disclosed herein, any numerical value falling within the range is also specifically disclosed.

Referring tonow, the present invention provides an optoelectronic modulethat may be optionally bundled with an external working channel(panel a) and a cross-sectional view thereof (panel b). As shown in, optoelectronic modulealone may include (i) a housing(or a cannula) with a longitudinal axis (L), (ii) an image sensorsecured at a distal end of the housing, and (iii) an ultrasound transducer(or ultrasound probe) within the housing. Ultrasound transduceris positioned in its entirety behind the image sensoralong the longitudinal axis (L).

The present invention also provides an imaging apparatusas illustrated infor imaging the interior surface of, and external environment surrounding, a tubular structure. Imaging apparatusmay include (1) the optoelectronic module () as shown inandB for insertion into the tubular structure; (2) a first receiving device () outside the tubular structure for receiving the signals from the image sensor (); and (3) a second receiving device () outside the tubular structure for receiving the signals from the ultrasound transducer ().

Ultrasound transducerserves as both the transmitter and receiver of sound waves, enabling healthcare professionals to visualize internal structures of the human body in real time. Ultrasound transduceremits sound waves (e.g. with 5-40 MHz frequency) into the human body. As the sound waves travel through different tissues, some of the waves are reflected back to the transducer. These returning echoes are analyzed by an ultrasound machine in the second receiving device () and used to construct images of the scanned area. Ultrasound transducercan turn electrical energy into sound, then upon receiving the echo turn the sound waves into electrical energy which can be measured and displayed.

Ultrasound transducermay be made from a traditional piezoelectric material such as PZT or semiconductor materials, and it may be selected from any suitable transducers such as linear array transducer, curvilinear (convex) array transducer, phased array transducer, endocavitary transducer, and 3D/4D transducer. Each transducer element can have its own cable. The cables can be bundled together and connected to an ultrasound processing unit in the second receiving device (). To reduce the number of cables running from the probe head, an ASIC can be included in the probe head. The ASIC can include high voltage switches and control circuits to drive the individual transducer elements and route echo signals to a processing unit in the second receiving device (). In such cases a reduced number of coaxial cables can then be used to carry ultrasound, transmit and receive signals, control signals and electrical power between the probe head and the ultrasound processing unit.

In one embodiment, second receiving device () includes electronics for processing the ultrasonic images from ultrasound transducer. Imaging apparatusmay be a combined ultrasound and endoscopy system (CUES). According to some embodiments, the endoscopy images and ultrasound images are displayed at real time simultaneously on a monitor or two separate monitors allowing a doctor to see both the surface and inside tissue of organs and can be transmitted electronically to other devices such as work stations that can be at remote locations and/or PACS (Picture Archiving and Communication Systems). According to some embodiments, the ultrasound image and camera image can be precisely correlated in position and orientation inside the human cavity (e.g. blood vessel).

In one embodiment, imaging apparatusincludes an external handle () linked to the ultrasound transducer () as shown in. Ultrasound handle () may be configured for grasping by a user's hand or being driven by a motor. Imaging apparatusmay further comprise an external working channelthat is bundled with the optoelectronic module (). Ultrasound handle () is configured for rotating the ultrasound transducer () about an axis (T) that is parallel with the longitudinal axis (L). It is also configured for pushing/pulling the ultrasound transducer () forwardly and backwardly (distally and proximally) within the housing () along a direction that is parallel with the longitudinal axis (L).

Therefore, ultrasound transducer () is within an ultrasound gel/fluid channelthat is within the housing (), and is rotatable about an axis (T) that is parallel with the longitudinal axis (L) by rotating external handle (), as indicated in. Ultrasound transducer () is also movable forwardly and backwardly (distally and proximally) within the ultrasound gel/fluid channel, along a direction that is parallel with the longitudinal axis (L) by pushing/pulling external handle ().

In some embodiments as illustrated inandpanel (a), optoelectronic modulemay further include an illumination component, an ultrasound gel/fluid channel, a gel/fluid outleton channel, and wiresconnecting to the image sensor.panels (a) and (b) are cross-sectional view of optoelectronic modulealong lines A-A and B-B respectively, illustrating an exemplary special arrangement of channel, image sensor, wires, and illumination component. With a gel/fluid filled within channeland outflowed from outletinto the air gap around the optoelectronic module, air gaps can be eliminated. This is preferred since sound waves used in ultrasound cannot travel well through air. Even a thin layer of air between the transducer and a surface such as skin can block the waves. The gel/liquid fills in those gaps, creating a seamless connection. The gel/liquid can act as a coupling medium, allowing high-frequency sound waves to pass from the transducer into the body and back without distortion. This ensures the returned echoes are clear enough to form accurate images. By allowing uninterrupted wave transmission, the gel/liquid helps produce sharper, more detailed ultrasound images.

Referring again to, image sensorhas a front faceand a rear side. Rear sideis positioned between front faceand ultrasound probewithin channel. In other words, ultrasound transduceris positioned entirely behind rear sidealong the longitudinal axis L.

An exemplary embodiment of imaging apparatusmay be an angioscope such as a coronary angioscope as illustrated in. Another exemplary embodiment of imaging apparatusis shown as a bronchoscope in. Still another exemplary embodiment of imaging apparatusfor imaging the interior surface of, and external environment surrounding, a tubular structureis shown as an endoscopein. When tubular structureis, for example, a lumen in the body of human or animal, endoscopebecomes an instrument useful for human medicine and veterinary medicine. However, it should be appreciated that endoscopemay be employed as an industrial endoscope, when tubular structureis a part of an industrial apparatus, an equipment, a product, a machine, a production line, and the like.

In exemplary embodiments, endoscopeincludes an optoelectronic modulewith a shell-like housing configured for insertion into tubular structurefor imaging its interior surface and the external environment surrounding it. For example, optoelectronic modulemay be inserted into a patient's body through a natural body orifice, such as the mouth, nose, urethra, bladder, vagina, or anus. Endoscopecan therefore have different configurations for use as an endoscopic ultrasound (EUS), a gastroscope, a colonoscope, endoscopic retrograde cholangiopancreatography (ERCP), and the like. Applications of endoscopeinclude diagnostic observation associated with endomaterial polyps, infertility, abnormal bleeding, and pelvic pain; and surgical procedure such as embryo growth arrest and uterine malformation etc.

Endoscopemay further include an insertion tube or shafthaving (i) a distal end connected to the proximal end of optoelectronic module, and (ii) a proximal end connected to receiving deviceincluding the first receiving device () outside the tubular structure for receiving the signals from the image sensor (), the second receiving device () outside the tubular structure for receiving the signals from the ultrasound transducer () or the combination thereof. Shaftmay be flexible, rigid, or semi-rigid, and is configured to extend proximately through the tubular structureto enable a force to be applied to optoelectronic moduleto control the movement thereof within tubular structure, and to permit retraction of optoelectronic modulefrom tubular structure.

Receiving deviceis generally located outside tubular structurefor receiving the signal from an image sensor and an ultrasound transducer (as will be described later) within optoelectronic module. For example, shaftmay include at least one electrical leadthat is coupled to optoelectronic moduleand conveys an electrical signal from optoelectronic moduleto receiving device. Shaftmay be detachably coupled to (or removably connected to) housingof receiving device, which may contain, for example, processor board, camera board and frame grabberand power source. The power sourcemay be, for example, one or more conventional dry-cell disposable batteries or lithium ion rechargeable batteries. Processor boardmay be coupled by cableto computerfor storage and retrieval of images generated by endoscope. Alternatively or in addition, housingmay include antennaand a wireless chipset, e.g., compliant with the IEEE 802.11 WiFi standards, for wirelessly transmitting the video image generated by endoscopeto computeror displaywithout cable. This arrangement may be particularly preferred for use in a physician's office because it permits the computer and display to be placed outside of the sterile field, while also allowing the physician greater maneuverability during use of endoscope.

Alternatively or in addition, computermay be programmed with image processing software that takes as input the image data output from endoscopeand generates two- or three-dimensional reconstructions of the body lumen and its surrounding tissues that may be displayed on display. A display that can display a moving image (movie), and is implemented by a CRT, a liquid crystal monitor, or the like.

In exemplary embodiments, a processor programmed with software may accept as input a plurality of still images of an object generated by the optoelectronic moduleand output for display a three-dimensional rendering of the object based on the plurality of still images. Housingmay also include switchesfor activating optoelectronic module, for switching image mode, and for activating frame grabberto create a still image from the video stream output generated by optoelectronic module.

Shaftmay be configured to couple optoelectronic moduleto the circuitry within housingin any suitable manner. For example, the availability of low-cost modular imaging system components enables the manufacture of a disposable components of the endoscopeat very low cost. In one embodiment, shaftis configured to detachably couple optoelectronic moduleto the circuitry within housing. In this manner, shaftand optoelectronic moduleare disposable, and may be detached from housingafter a single patient use, thus eliminating the need for sterilization or reprocessing and reducing contamination risks. Housingmay be disinfected for subsequent reuse with a new shaft and optoelectronic module for a different patient.

In preferred embodiments, shaftmay serve as a tether, and may include a plurality of scale markings or fiducialsthat enable a physician to measure a distance traveled by optoelectronic moduleinto tubular structuresuch as a lumen of a body.

Other known structure(s) may be built into endoscopeas desired. For example, an ergonomic handle can be used for easy operation. A mechanism can be introduced in the endoscope to curve shaft. The physician may bend or curve the shaft by pulling or loosening a wire (not shown). Modulemay be turned or maneuvered by way of a flexible shaft or cable.

Endoscopemay be operated to perform or complete selected tasks manually, automatically, or a combination thereof. Some endoscopic functions may be implemented with the use of components that comprise hardware, software, firmware or combinations thereof. While general-purpose components such as general-purpose computers or oscilloscopes can be used in endoscope, dedicated or custom components such as circuits, integrated circuits or software can be too. For example, some functions are implemented with a plurality of software instructions executed by a data processor, which is part of a general-purpose or custom computer. In some embodiments, the data processor or computer comprises volatile memory for storing instructions and/or data and/or a non-volatile storage, for example, a magnetic hard-disk and/or removable media, for storing instructions and/or data. In some embodiments, implementation includes a network connection. In some embodiments, implementation includes a user interface, generally comprising one or more input devices (e.g., allowing input of commands and/or parameters) and output devices (e.g., allowing reporting parameters of operation and results).

schematically illustrate the structural configurations of optoelectronic module. Referring to, an optoelectronic modulecomprises a housingand an image sensor. The image sensoris located within housing, and it has a front facewith a perimeter S, which may be in the range of about 2.3 mm to about 6 mm, for example, about 2.3 mm to about 4 mm, and about 4 mm to about 6 mm. The housingmay have a thickness of T thinner than 0.1 mm. The cross section of the optoelectronic module along the front facehas a perimeter H. The perimeter of a circle or ellipse can also be called its circumference. In various embodiments, S<H<1.6S, such as S<H<1.5S, S<H<1.45S, S<H<1.41S, S<H<1.3S, S<H<1.2S, S<H<1.11S, S<H<1.075S, S<H<1.05S, and S<H<1.025S.shows the cross-sectional view of various optoelectronic module having different designs of the image sensor's front face and the housing. A cross section is the intersection of a body (e.g. optoelectronic module) in three-dimensional space with a plane (e.g. front face). From design (a) to (f), H is decreased from 1.6S to 1.025S, or even down to 1.02S and 1.01S.

Despite the shape and dimension of various parts as illustrated in, the cross section of optoelectronic modulealong said front facemay have any regular shape such as circular, it may have an irregular shape as well. Although front facegenerally has a regular shape such as a polygon, e.g. rectangular and square, it may also have irregular shape.

Referring to, optoelectronic modulecomprises an image sensorwithin a cylinder housingwhile image sensorhas a non-circular front face. Image sensoris positioned at a distal end (or front tip) of optoelectronic module, and non-circular front faceis pointing forward or facing forward. In various embodiments, image sensorcan be any suitable device having a light sensitive surface (e.g. non-circular face) usable for capturing an image, for example, Charged Coupled Device (CCD) and Complementary Metal Oxide Semiconductor (CMOS) image sensors. Ultrasound transducer(or ultrasound probe) within the housingand behind the image sensoris not shown here.

As a principle of plane geometry, a minimum bounding circlemay definitely be determined based on any given non-circular planar face. It should be appreciated that the concept of “Circumscribed Circle” is different from the concept of “Minimum Bounding Circle” (AKA “Smallest Circle”). Referring to, in geometry, the circumscribed circle or circumcircle of a polygon (represented as triangle T in) is a circle denoted as CC which passes through all the vertices of the polygon (3 vertices for triangle T in this example). The center of this circle CC is called the circumcenter and its radius is called the circumradius. A polygon which has a circumscribed circle is called a cyclic polygon (sometimes a concyclic polygon, because the vertices are concyclic). All regular simple polygons, all isosceles trapezoids, all triangles and all rectangles are cyclic.

The smallest-circle problem or minimum covering circle problem is a mathematical problem of computing the smallest circle that contains all of a given set of points in the Euclidean plane. Referring again to, a minimum bounding circle denoted as MBC is the smallest circle that completely contains the polygon (e.g. triangle T) within it. Not every polygon has a circumscribed circle, as the vertices of a polygon may not all lie on a circle, but every polygon (or even any non-circular irregular 2D shape) has a minimum bounding circle, which may be constructed by a linear time algorithm. As shown in, even if a polygon has a circumscribed circle, it may not coincide with its minimum bounding circle. For obtuse triangle T, the minimum bounding circle MBC has the longest side as diameter and does not pass through the opposite vertex, and MBC is much smaller than CC.

In various embodiments, the size and dimension of optoelectronic moduleis designed based on the radius Rmbc of the minimum bounding circle MBC. As shown in, circular cylinder housingis constructed to have an internal radius which equals to Rmbc, and an external radius Rc. The thickness of housingwall is Th=Rc−Rmbc. Various components of moduleare confined within a space defined by a right circular cylinder with radius Rmbc and length L, as shown in. There is no special limitation on length L, but L should be kept as short as possible. In some embodiments, length L may be adjusted as desired. For example, when a lens is needed in front of front face, modulecan be extended forward a little bit to accommodate the lens, making length L a little bit longer. Alternatively, the lens may protrude from face.

All components of moduleexcept the protruded-out lens (if present) are configured to be within housingas defined above. Referring to, these components may include, but are not limited to, one or more illumination components, one or more optional working components, and an optical component, among others.

Although components,andcan be placed anywhere within the cylinder-shaped housing, in preferred embodiments of the invention, illumination componentsand working componentsshould block front faceas little as possible, and lensshould cover (or overlap with) faceas much as possible.

In a representative embodiment, non-circular facehas the shape of square, and the diagonal of the square equals to the diameter (2×Rmbc) of the minimum bounding circle. However, it should be appreciated that non-circular facemay have a shape of any polygon such as triangle, rectangle, pentagon, and hexagon etc. Referring to, a segment can be defined by a chord and an arc. Therefore, four segments (S, S, S, S) can be defined by 4 sides (a, b, c, d) of square-shaped faceas 4 chords, and 4 corresponding arcs (Arc, Arc, Arc, Arc) of the minimum bounding circlelying between the 4 chords' endpoints.

As shown in, one or more illumination components(,,,) can be arranged in 1, 2, 3, or all of the 4 segments (S, S, S, and S). By the same token, one or more optional working components(if present) may also be arranged in 1, 2, 3, or all of the four segments (S, S, S, and S).

Referring to, an electronic circuit board, flexible or rigid, may be configured to carry image sensormounted thereon. Circuit boardmay supply image sensorwith necessary electrical power and may derive still images and/or video feeds captured by the image sensor. Ultrasound transducer(not shown) is positioned in its entirety behind electronic circuit boardalong the longitudinal axis (L). In other words, electronic circuit boardis positioned between front faceand ultrasound probe(not shown).

Referring to, optical componentwill be described using lens or micro lensas an example. However, it should be appreciated that componentmay be a catoptric system, or may be a combination of a lens and a catoptric system. Componentmay include a plurality of optics such as lens assemblies, lenses and protective glass, and is configured to receive reflected light from target objects. A lens assembly may include a plurality of lenses, static or movable, which may provide a field of view of at least 90 degrees (90°), typically 120 degrees, and up to essentially 180 degree or 230 degrees. A lens assembly may provide a depth of field of about 2 to 200 mm.

In a preferred embodiment as shown in, lenshas the same shape and dimension as face. For example, lens, exactly like front face, may be a square lens having a perimeter S. Such a lens and such a sensor face can match (coincide) each other perfectly.

Lensmay function as an objective lens, and may comprise coatings such as chromatically correcting coating, and hydrophobic coating. In many applications, a wide-angle objective lens is normally used in order to prevent a situation in which an area of interest such as a lesion area is missing. For example, such objective lensmay have an angle of view of 170°-230°. A fish-eye lens having an angle of view of more than 180° may be employed as lens. In an operation state, lensfocuses the reflected light from the observation target, image sensordetects the focused reflected light, and an A/D converter can convert analog image signals obtained by photoelectric conversion performed by image sensorinto digital image signals.

In various embodiments, the lens has a short total optical length (track), for example, 5 mm or less. The lens may be configured to provide a large incident angle, for example, a chief incident angle larger than 20°, such as 20-40°, and provide minimal distortion (for example, less than 80%).

Referring to, one or more illumination componentscan be selected from a light source, an optical fiber optically coupled to a light source, a light diffuser (e.g. an illumination lens) optically coupled to a light source, or any combination thereof. Componentsindependently of each other emit light of same or different wavelength. Examples of light source include a light emitting diode (LED), an incandescent lamp (e.g. halogen lamp), a gas-discharge lamp (e.g. mercury-vapor lamp), a fluorescent lamp, an arc lamp, an ultraviolet lamp such as a Wood's lamp, an infrared lamp, or any combination thereof. Example of LED include a white light LED, an infrared light LED, a near infrared light LED, an ultraviolet light LED or any other LED. Optical fibers are light carriers that carry light from a remotely located light source. Exemplary light diffuser is made of polycarbonate and is coated with a reflective coating on the surface thereof. Similar to image sensor, an electronic circuit board assembly may also be configured to carry LED that is able to illuminate the viewing field of lens. In a preferred embodiment, the light source is a flashing LED, which can reduce the “LED ON” time and therefore decrease the heat generated. Flashing LED is particularly useful when module is so small that heat management becomes important, or even critical. In preferred embodiments, such a flashing LED is used with lensand facethat have the same shape and dimension (e.g. both are identical square shaped as shown in). The flashing LED may be located behind the image sensor, and deliveries the light emission to the tip of modulethrough one or more optical fibers. As shown in, a light blockcan be used to prevent any negative impact from, or any interference of, the strong light emission from the LED against the sensor. Light blockmay be, for example, a coating layer on the back of image sensor.