Apparatus and Methods for Tactile Sensing

This application claims the benefit of priority to U.S. Provisional Application No. 63/648,533, entitled “Apparatus and Methods for Tactile Sensing,” filed on May 16, 2024, the disclosure of which is incorporated by reference herein in its entirety.

This patent disclosure contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure as it appears in the U.S. Patent and Trademark Office patent file or records, but otherwise reserves any and all copyright rights.

This invention was made with government support under 80NSSC22K1179 awarded by NASA and 2036197 awarded by the National Science Foundation. The government has certain rights in the invention.

Embodiments of the disclosure relate generally to haptic technology. In some non-limiting implementations, the disclosure relates to tactile sensing.

The human skin uses a complex network of tactile sensors or mechanoreceptors to detect physical parameters from the external environment, such as pressure, vibration, texture, temperature, hardness, shape, or size. Tactile sensing technology can replicate mechanisms of these sensors so that various physical parameters can be artificially captured and processed by systems employing robotics and/or other automated devices.

Apparatus and methods are disclosed for tactile sensing techniques. In one embodiment, a sensing apparatus is provided that comprises a first sensing device; a structure configured to support the first sensing device; and an electronic device configured to process a first signal generated by the first sensing device, wherein the first sensing device is configured to generate the first signal in response to a first distributed mechanical effect with a first frequency. The first distributed mechanical effect corresponds to a change of a mechanical force. In another embodiment, the sensing apparatus further comprises a second sensing device supported by the structure, the second sensing device configured to generate a second signal in response to a second physical effect, wherein the electronic device is configured to process the first signal and the second signal. The second sensing device can have a different response time and/or a different sensitivity level compared to the first sensing device.

In a further embodiment, a sensing method is provided that comprises generating, by a first sensing device, a first signal in response to a first distributed mechanical effect with a first frequency; generating, by a second sensing device, a second signal in response to a second physical effect; and processing, by an electronic device, the first signal in combination with the second signal.

The first distributed mechanical effect with a first frequency corresponds to a change of a mechanical force, in accordance with an embodiment. The second physical effect can be a static effect with a second frequency lower than the first frequency, in accordance with another embodiment. In yet another embodiment, the first sensing device includes a polyvinylidene fluoride (PVDF) film. The PVDF film can include a plurality of tactile elements in some examples. In a further embodiment, the second sensing device includes a capacitive transducer. The second sensing device can have a different response time and/or a different sensitivity level compared to the first sensing device.

In yet another embodiment, a method for manufacturing a tactile sensing device is provided that comprises preparing a base material; evaporating at least one layer of first deposit material onto a first side of the base material; and photolithographically creating a pattern on the first deposit material using a mask, wherein the pattern includes a plurality of electrodes used to send signals for tactile sensing. In a further embodiment, the method of for manufacturing a tactile sensing device further includes evaporating at least one layer of second deposit material onto a second side of the base material. In an embodiment, the base material is polyvinylidene fluoride (PVDF) film. In another embodiment, the method further includes depositing a first layer of a compliant material onto the base material, photolithographically creating a pattern on the first deposit material using a mask, wherein the first deposit material is on the first layer of the compliant material, and depositing a second layer of the compliant material onto the first deposit material.

The present disclosure will now be described in more detail with reference to particular embodiments thereof as shown in the accompanying drawings. While the present disclosure is described below with reference to particular embodiments, it should be understood that the present disclosure is not limited thereto. Those of ordinary skill in the art having access to the teachings herein will recognize additional implementations, modifications, and embodiments, as well as other fields of use, which are within the scope of the present disclosure as described herein.

Embodiments disclosed herein are related to tactile sensing techniques. Haptic technology, or kinaesthetic communication, utilizes vibrations, forces, and motions to create a sense of touch. With development in haptic technology, physical sensory perception of human skin can be mimicked and harnessed for technological advances in fields like robotics, medical, consumer products, security systems, automotive, etc.

Human skin contains mechanoreceptors for both static and dynamic sensing. Static sensing is the detection of slowly-changing stimuli, like constant forces, while dynamic sensing is the detection of fast-changing stimuli, like vibrations and the onset of touch. Dynamic sensing plays an important role in detecting the making/breaking of contact, sensing slip, sensing surface texture, and receiving tactile feedback via a held object (e.g., tool use).

A transducer is a device that converts one form of energy into another, such as converting mechanical energy into electrical signals. Transducers used for dynamic sensing have a high frequency response and sensitivity, and often respond only to changes in signals, not static forces. For example, a piezoelectric effect is the ability of certain materials to generate an electric charge in response to applied mechanical stress. This piezoelectric effect responds to the mechanical stress instantaneously and continuously, meaning the material generates a charge as soon as the stress is applied and continues to do so as long as the stress is present. When mechanical stress is applied to a piezoelectric material, it causes a deformation in the physical structure of the material. This deformation leads to a redistribution of electrical charges within the material, resulting in an electric charge on the surface of the material. The piezoelectric material continuously responds to changes in mechanical stress. When the stress varies over time, the generated electric charge will also vary accordingly. The dynamic sensing transducers would capture real-time changes on the mechanical stress being applied and respond to these changes dynamically. For instance, a piezoelectric accelerometer detects changes in acceleration by measuring the mechanical stress applied to the piezoelectric material. As the acceleration changes, the mechanical stress on the piezoelectric material changes, generating a corresponding electric charge. The generated electric charge is converted into an electrical signal that represents the acceleration.

There remains a significant gap in exploring the complementary nature between static and dynamic sensing techniques. Static sensing measures physical quantities that are constant or change very slowly over time. For instance, static sensing can be based on resistive, capacitive, or inductive principles where the sensor's output is based on the physical quantity. Combining dynamic and static sensing at the same time is challenging due to the differing requirements for response time, sensitivity, stability, and signal processing, etc. For example, there is a lack of tactile fingers that integrate force sensing with distributed vibration sensing in the hundreds of Hz. Furthermore, research on leveraging multimodal tactile feedback to enhance manipulation skills is relatively scarce. In robotics, there are many ways to implement dynamic sensing, including accelerometers, microphones, hydrophones, piezoelectrics, and piezoresistors. Building tactile fingers with distributed vibration sensing across a complex finger-shaped surface remains a challenge. Polydimethylsiloxane (PDMS), also known as dimethylpolysiloxane or dimethicone, is a silicone polymer with a wide variety of uses including sensing applications, medical phantoms, and organ-on-chip studies, etc.

Tactile sensing technology can utilize a range of physical effects. For example, a piezoelectric transducer monitors the piezoelectric effect of an electric charge in response to applied mechanical stress. For a certain material, its property can cause charges to be generated when the material is under stress. Piezoelectric sensors measure the charge produced when the sensor is stimulated. Piezoelectricity can also be utilized for actuators.

In an embodiment, a bimodal finger is disclosed that combines distributed force and vibration sensing, as a platform for studying multimodal tactile sensing and how it can benefit manipulation. A piezoelectric polyvinylidene fluoride (PVDF) is used for vibration modality with custom electrode patterns. Capacitive sensors can be used as force modality to implement force sensing. In some embodiments, a sensor system contains custom-fabricated, taxelized PVDF films for achieving high-resolution taxelization and compact integration. In other embodiments, a sensor system contains other devices like PVDF strips as a more accessible solution (e.g., does not require cleanroom fabrication tools). Aspects of the disclosed techniques combine a distributed, high frequency (hundreds of Hz) vibration sensing transducer type with a static force sensing transducer type in a tactile finger. Compared to unimodal sensors that measure vibrations and static pressures, multi-modality provides the advantage that each modality's sensitivity can be tuned to target different stimuli. In some embodiments, PVDF and capacitive transducers are integrated together into a single tactile finger. In other embodiments, a single taxelized PVDF sheet is used with multiple independent taxels (sometimes also referred to as tactile elements) in a tactile finger with a curved finger surface.

In some embodiments, a tactile finger functions with distributed vibration sensing using PVDF. PVDF films can be custom fabricated with individualized taxels (sometimes also referred to as electrodes). PVDF responds to changes in input, and is sensitive to very light touches and vibrations (the response is “spiky”).

In some embodiments, PVDF films can be combined with capacitive sensing as an underlying static force modality. These modalities are complementary, with PVDF targeting vibrations and capacitive sensors measuring constant pressures.

In some embodiments, a tactile finger can include custom-made, taxelized PVDF film. The sensor interfaces with electronics with a heat seal connector. The electronics can include a printed circuit board (PCB), a microcontroller development system, amplifiers, heat seal connection, ports for capacitive sensors, etc.

In some embodiments, a tactile finger prototype can include PVDF strips with the electronics. In some embodiments, sensing techniques include a system that combines PVDF with capacitive sensors into one bimodal finger. Outputs of capacitive sensors are sampled with data sampling and/or conversion devices.

In some embodiments, the combined modalities are complementary. For example, PVDF targets detecting initial touch and vibrations, while capacitive sensors target static pressures. By combining these modalities, more complex manipulation tasks can be accomplished with the disclosed sensing techniques.

shows a tactile finger prototype according to an embodiment. Specifically,shows an example sensing apparatusincluding a sensing deviceand a supporting structure. The sensing devicefurther includes at least one sensing unitand a data interconnectfor the at least one sensing unit. The data interconnectincludes at least one data trace that inputs data (for instance, instructions and configuration parameters controlling the operation of the sensing device) to and/or outputs data (for instance, data representing sensed signals corresponding to the physical effect that is being sensed by the sensing device) from the at least one sensing unit. The data interconnectconnects the sensing deviceto a data and signal processing device that can include a printed circuit board (PCB), a microcontroller development system, amplifiers, heat seal connection, ports for capacitive sensors, etc.

shows a custom-made, taxelized PVDF film for a tactile finger sensor, according to an embodiment. Specifically,shows another example sensing deviceincluding at least one sensing unitand a data interconnectfor the at least one sensing unit. The data interconnectincludes at least one data trace that inputs data (for instance, instructions and configuration parameters controlling the operation of the sensing device) to and/or outputs data (for instance, data representing sensed signals corresponding to the physical effect that is being sensed by the sensing device) from the at least one sensing unit. The data interconnectconnects the sensing deviceto a data and signal processing device that can include a printed circuit board (PCB), a microcontroller development system, amplifiers, heat seal connection, ports for capacitive sensors, etc.

shows another example sensing deviceincluding at least one sensing unit, a data interconnect, and a signal processing device, according to an embodiment. The signal processing devicereceives data representing sensed signals corresponding to the physical effect that is being sensed by the sensing device, and processes the data to extract information and/or for further processing. In some examples, the electronics of the signal processing devicecan include a heat seal connector.

shows sensor interfaces with custom electronics for a tactile finger sensor, according to an embodiment. In the example shown by, an example sensing apparatusincludes a sensing device, a supporting structure, and a signal processing deviceconnected to the sensing device. The signal processing devicereceives data representing sensed signals corresponding to the physical effect that is being sensed by the sensing device, and processes the data to extract information and/or for further processing. In some examples, the electronics of the signal processing devicecan include a heat seal connector.shows another example sensing apparatus from another perspective of view, according to an embodiment.

illustrates an example circuitry for sensor interfaces with custom electronics for the signal processing device, according to an embodiment. In some examples, the circuitry can include a heat seal connection.

show a front side and a back side of resin molds for fabrication of a tactile finger sensor including the supporting structure, according to an embodiment. In some examples, the resin is encased by an elastomer polydimethylsiloxane (PDMS) material. An elastomer material is a rubbery material composed of long chainlike molecules, or polymers, that are capable of recovering their original shape after being stretched to great extents. The elastomer creates a compliant surface that distributes applied strain. The sensor can contain a stiffer outer layer of PDMS to provide a durable coating.

Referring to, in some embodiments, the process flow for fabrication of a sensing device, for instance, a taxelized PVDF films includes fabricating an array of electrodeswith corresponding traces and a back planeto complete each capacitor. For example, the array can contain around 16 electrodes and traces that run to the edge of the sensor for connection to external electronics. In an embodiment, a metal stackon each side can have around 20 nm of a chrome adhesion layer and around 150 nm of gold, which is evaporated via electron beam onto, for example, 4.5 cm×4.5 cm pieces of 100-micron thick PVDF. The array of electrodesand traces are photolithographically patterned using a photomaskto pattern photoresist and wet etching to transfer the pattern to the metal stack. A back planeis patterned using shadow mask deposition. Once fully patterned, excess PVDF is cut off with a precision knife to create the final geometry. The sensor deviceis then connected to heat seal connectors via hot bar bonding to the sensor traces for future connection to off-sensor PCBs. In an embodiment, the electrodes can also be configured to provide integral shielding. For example, a larger outside electrode can be grounded, providing a shielding functionality for the patterned electrodes on the interior. A differential or paired structure can also be considered using two layers of the sensing medium, with appropriate amplifiers for reduction of system noise. In another embodiment, the sensor layer, for example, the array of electrodesand traces are fabricated between two layers of a compliant material.

shows a tactile finger prototype with non-custom PVDF strips, according to an embodiment.shows electronic wirings for a tactile finger sensor, according to an embodiment.

illustrates an example circuitry of the signal processing devicefor processing signals from a tactile finger sensor, according to an embodiment. In some examples, the circuitry can include a microcontroller and amplifiers. The circuitry is implemented on an off-sensor PCB and can work with both a tactile finger with custom-made PVDF film and a tactile finger with non-custom PVDF strips.

illustrate signal samples from tactile finger sensors, for instance the sensing devicein response to light taps visualized at 200 Hz. In some examples, the plotted signal shows multiple transducers being stimulated from a single tap.

illustrates an electronics setup for tactile finger sensors, according to an embodiment. In the example shown in, a sensing deviceconnects to a signal processing devicevia an interconnect. The signal processing devicereceives data representing sensed signals corresponding to the physical effect that is being sensed by the sensing device, and processes the data to extract information and/or for further processing. The signal processing devicefurther includes an on-sensor circuitand an off-sensor circuit, the on-sensor circuitand the off-sensor circuitlinked with a signal processing link. The signal processing linkincludes a plurality of data lines that input data (for instance, instructions and configuration parameters controlling the operation of the sensing device) to and/or output data (for instance, data representing sensed signals corresponding to the physical effect that is being sensed by the sensing device) from the on-sensor circuitand the sensing device. In some examples, the electronics of the signal processing devicecan include a heat seal connector.

illustrates a tactile finger sensor with capacitive sensors, according to an embodiment. Specifically,shows an example sensing apparatusincluding a sensing deviceand a supporting structure. The sensing devicefurther includes at least one sensing unitthat operates at a different mode than the sensing deviceand the sensing unit(shown above as described in). In an embodiment, the sensing deviceperforms static sensing that measures a physical quantity that is constant or changes very slowly over time. For instance, static sensing can be based on resistive, capacitive, or inductive principles where the sensor's output is based on, respectively, the physical quantity of resistance, capacitance, or inductance. Such different sensing operation has different response time, sensitivity, stability, and signal processing, etc. The sensing devicecan connect to a data interconnect that inputs data (for instance, instructions and configuration parameters controlling the operation of the sensing device) to and/or outputs data (for instance, data representing sensed signals corresponding to the physical effect that is being sensed by the sensing device). The data interconnect connects the sensing deviceto a data and signal processing device that can include a printed circuit board (PCB), a microcontroller development system, amplifiers, heat seal connection, ports for capacitive sensors, etc.

illustrates an example circuitry for sensor interfaces with custom electronics for the signal processing deviceconnecting to the sensing device, according to an embodiment. In some examples, the circuitry includes ports for connection with capacitive sensors.

Referring to, in some embodiments, the finger is curved and sensorized everywhere except the back—all areas where contacts are likely. In an embodiment, there are no protrusions (.e.g., convex areas) that are unsensorized. In other embodiments, there are flat patches on the front to increase contact stability. For example, the finger comes to a tip to enable picking up small objects off of a flat surface with a pinch grasp. The overall size can be similar to a human thumb, but can be limited by the size of the PCBs and the capacitive sensors, which can have a diameter of around 8 mm. The molds can be modified to adjust the geometry in further embodiments.

shows exemplary capacitive sensors that are single-point capacitive force sensors with good SNR and limited hysteresis. They are an accessible way to integrate force sensing into a finger. With custom capacitive arrays (which is a plausible option), in some cases, it could be hard to achieve performance as good as the single-point capacitive force sensors. An aspect of these capacitive sensors is that they are around 8 mm in diameter (fairly large), and each signal has its own tail, which is to be routed to a PCB.

In other embodiments, as illustrated by, PVDF strips can be cut to a desired size. The PVDF strips are a sheet of PVDF with an electrode deposited on each side. They have two leads crimped to them, one being connected to measure a generated current (or send current, if using them as an actuator).

In further embodiments, as illustrated by, custom-made PVDF created in clean rooms provide continuous films of PVDF that are deposited with electrodes and traces in customizable patterns. The traces on the film can be connected to via a heat seal connector-a flexible connector that can be adhered by applying heat and pressure. In some examples, these connectors contain an anisotopic, conductive adhesive in the traces. When heated, the traces bond and connect to a sensor (or other interface). Because the adhesive is anisotropic, the connection is only in the direction of the traces (i.e., they don't short).

illustrates, in some embodiments, a sensing apparatus(for instance, a sensor finger) including a first sensing deviceand a second sensing device, and method of fabricating the sensing apparatus. 3D-printed moldscan be used to cast a layer of PDMS onto a supporting structure(for instance, a 3D-printed rigid skeleton). The second sensing deviceincluding, for example, capacitive sensors, is attached to the supporting structure. Then, a stiffer outer layer of PDMS is applied using a dipping process. This two-step process allows embedding the first sensing device (for instance, a PVDF sensor) between the two layers. A double-sided tape can be used to stick the capacitive sensors to the skeleton, and route the tails through the skeleton. In one example, the tails stick out the back of the skeleton during molding. Then, the molds are sprayed with a release agent (e.g., Mann Ease Release 200) to reduce adhesion between the mold and PDMS.

In an embodiment, to apply non-custom PVDF, a thin layer of PDMS is applied to the mold and is partially cured with a heat gun until it is tacky. Then, the strips are stuck to the mold in a desired configuration. Mixed and degassed PDMS is poured (e.g., 30:1 base to curing agent weight ratio) into the prepared mold and it is cured in an oven at 75 C for around 8 hours. After demolding, excess PDMS is removed from the back of the finger with a precision knife. When using custom PVDF, the sensor may be adhered to the finger with cyanoacrylate after casting the finger.

In another embodiment, the finger is dipped in stiffer PDMS (e.g., 10:1 ratio) to apply a thin outer coating on a top of the sensor, and it is cured at 75 C for around one hour (the dip process can be similar to making a candle). This procedure is completed, for example, three times to form a coating of around 1 mm. In other embodiments, instead of a dip, a gradual drizzle can be applied over the finger which lets the PDMS drip down the sides (similar to drip candy coating). This process can waste less PDMS and provide an even layer.

Referring to, in some embodiments, sensors are connected to electronics with wiring for the PVDF.illustrate a tactile finger with a custom-made, taxelized PVDF film and associated electronics, according to an embodiment.

illustrate a tactile finger with non-custom PVDF strips and associated electronics, according to an embodiment. When suspending the non-custom PVDF strips, as an example, around 6 strips can be positioned in a finger so they are suspended in the middle of the elastomer and in repeatable locations. The sensors themselves can be thin and flimsy, so they move easily after positioning them. In some examples, the leads coming off the PVDF are fairly long and take up too much space to sit in the elastomer. Instead, they dangle out the back of the finger. The wires soldered to the PVDF can be removed prior to placing in the mold. Otherwise, movement of the heavy wires can move the placement of the strip. The leads can go around the skeleton, not through it. In some embodiments, the leads are not very flexible, so they cannot be fed through a tight space without affecting the strip placement. Strips are placed within the mold before the mold is assembled. Otherwise, it can be challenging to position them. The strips and their leads should not be disturbed when assembling the mold, so there should be enough tolerance to assemble the mold without disturbing the leads. In other examples, the strips are positioned on a pre-cured finger, rather than inside the mold. However, this could be messy without a good adhesive to secure the strips in place. In further examples, a spray-on adhesive can be used that dries quickly (e.g., spray-on double-stick tape), making a “stencil” that can fit over a pre-cured finger. Then, the spray-on adhesive is used to secure the strips with the stencil before drizzling the outer PDMS coating.

In an embodiment, the suspended PVDF in a compliant structure has several advantages. For instance, the outside compliant layer to the PVDF material protects the device from scratching or other external insult, the compliance of the stack allows for sensor elements to be more useful as grippers or actuation elements, and the elastomer on the inside allows for the improved compliance upon contact to strain the PVDF and provide a useful signal.

illustrate a tactile finger with capacitive sensors, according to an embodiment.

In some embodiments, when suspending custom PVDF films, the single film is stiffer than the strips, such that using PDMS as an adhesive may not be suitable. Cyanoacrylate works as an adhesive to bond the film either to a pre-cured finger or to the inside of the mold. However, it can take longer to dry and provide a good bond. In other embodiments, double stick tape can fit even better for this, since there's no drying time. However, double stick tape can be difficult to handle and reposition. Using the spray-on adhesive mentioned above is a suitable way to stick the sample inside the mold.

Referring to, in some embodiments, the overall layout of electronics includes, for example, two printed circuit boards (PCBs): an “on-sensor” board and “off-sensor” board. The on-sensor board fastens to the back of the finger bone and connects to the sensors. The off-sensor board sits elsewhere and connects to one or multiple fingers via a cable, e.g., Flat-Flexible Cables (“FFC”). The off-sensor board contains a microcontroller and communicates with a PC computer using micro-ROS.

In the example shown in, at least one sensing device(which can be an array of sensing devices according to an embodiment) connects to a signal processing devicevia an interconnect. The signal processing devicereceives data representing sensed signals corresponding to the physical effect that is being sensed by the at least one sensing device, and processes the data to extract information and/or for further processing. The signal processing devicefurther includes an on-sensor circuitand an off-sensor circuit, the on-sensor circuitand the off-sensor circuitlinked with a signal processing link. The signal processing linkincludes a plurality of data lines that input data (for instance, instructions and configuration parameters controlling the operation of the at least one sensing device) to and/or output data (for instance, data representing sensed signals corresponding to the physical effect that is being sensed by the at least one sensing device) from the on-sensor circuitand the sensing device. In some examples, the electronics of the signal processing devicecan include a heat seal connector. In one embodiment, the on-sensor circuitof the signal processing deviceis implemented by a circuit boardand/or a circuit board. In another embodiment, the off-sensor circuitof the signal processing deviceis implemented by a circuit board.

In some embodiments, with PVDF Electronics, the PVDF is connected to the on-sensor board using either heat seal connectors or wires. Analog signals are routed through the FFC to the off-sensor board. The off-sensor board contains current amplifiers with high-impedance operational amplifiers (op amps), followed by the microcontroller's Analog-to-digital Converters (ADCs) which digitize the signal. Current amplifiers (as opposed to voltage amplifiers) can be used because their output is independent of the cable capacitance—a relevant point since long cables are used. The sample frequency can be at, e.g., around 1000 Hz.

Referring to, in some embodiments, multiple components are placed on the on-sensor PCBs, where space is limited. PVDF signals can have extremely high impedance (i.e., weak). In practice, this means that the signal is susceptible to, e.g., accumulating noise or being loaded by the amplifier/ADC. As a result, it would be helpful for the signals to be amplified and digitized as close to the transducer as possible. Different combinations of layouts for how to read the PVDF can be configured. Tradeoffs among the various configurations include balancing the amount of electronics and the signal quality.