Glove Systems

This application is a continuation of International Application No. PCT/CN2024/099228, filed on Jun. 14, 2024, which claims priority to International Application No. PCT/CN2023/124295, filed on Oct. 12, 2023, the contents of which are hereby incorporated by reference.

The present disclosure relates to the field of sensor technology, and in particular, to a glove system provided with a plurality of sensors and a method for recognizing a gesture based on the glove system.

With the gradual maturation of AR/VR technology and the rise of the meta-universe concept, intelligent electronic devices set higher requirements for human-computer interaction technology. Flexible angle sensors can be conveniently integrated into wearable devices such as smart clothing and smart gloves, so as to realize accurate recognition and restoration of human movements, and are among the important underlying technologies of the meta-universe, which have received extensive attention and research. In order to realize accurate recognition and restoration of hand movements, there is a need to capture movements of the hand, especially the fingers, in a plurality of degrees of freedom.

Therefore, it is desirable to provide an electronic device that improves the accuracy of hand movement acquisition while controlling the cost.

One of the embodiments of the present disclosure provides a glove system, including: a glove body, and a plurality of first strain sensors and a plurality of second strain sensors arranged on the glove body, when a user is wearing the glove body, each of the plurality of first strain sensors is located at an interphalangeal joint of a finger of the user and is configured to measure deformation of the interphalangeal joint in a single degree of freedom, and each of the plurality of second strain sensors is located at a metacarpophalangeal joint of the finger and is configured to measure deformation of the metacarpophalangeal joint in two degrees of freedom, the single degree of freedom and the two degrees of freedom both including a degree of freedom corresponding to a bending of the finger.

In some embodiments, the plurality of first strain sensors and the plurality of second strain sensors collectively provide data reflecting deformation of the fingers of the user in no less than 15 degrees of freedom.

In some embodiments, the glove system further includes two inertial sensors, when the user is wearing the glove body, the two inertial sensors are respectively located on two sides of a wrist joint of the user and configured to jointly measure movement of the wrist joint in three degrees of freedom.

In some embodiments, the glove system further includes a third strain sensor disposed at the wrist joint of the user and an inertial sensor disposed outside the wrist joint.

In some embodiments, a first strain sensor of the plurality of first strain sensors includes an inductive sensor, and an inductance of the inductive sensor varies with the deformation of the interphalangeal joint.

In some embodiments, a second strain sensor of the plurality of second strain sensors includes a plurality of capacitive structures, capacitances of the plurality of capacitive structures vary with the deformation of the metacarpophalangeal joint in each degree of freedom.

In some embodiments, the second strain sensor includes: a flexible substrate; a first capacitive structure and a second capacitive structure, each of the first capacitive structure and the second capacitive structure includes a multilayer structure arranged on a same side surface of the flexible substrate in a thickness direction, and layers of each of the multilayer structures are stacked along the thickness direction.

In some embodiments, the glove body includes a fabric wrapped around each finger of the hand and a fabric disposed between neighboring fingers, an elastic modulus of the fabric between the neighboring fingers is less than an elastic modulus of the fabric wrapped around each finger.

In some embodiments, the fabric disposed between the neighboring fingers include at least one of a slit structure, an elastic fabric structure, or a pleated structure.

In some embodiments, the plurality of first strain sensors are arranged only at the interphalangeal joints of the fingers that are closest to the metacarpophalangeal joints.

In some embodiments, the glove system further including a processor configured to: read data associated with the plurality of first strain sensors and the plurality of second strain sensors; and determine a gesture of the user based on the data.

In some embodiments, the processor is configured to: output a gesture representation corresponding to the gesture of the user on a display interface; and output feedback information based on a distance between the gesture representation and a preset gesture.

In some embodiments, the processor is configured to: read a mapping relationship between gestures and voice packets; and play a voice message based on the gesture of the user and the mapping relationship.

In some embodiments, the processor is configured to: read a mapping relationship between gestures and instructions; and execute an instruction based on the gesture of the user and the mapping relationship.

One of the embodiments of the present disclosure provides a gesture reproduction method including: obtaining sensor data of a hand of a user based on a glove system, the glove system includes a glove body, and a plurality of first strain sensors and a plurality of second strain sensors arranged on the glove body, when the user is wearing the glove body, each first strain sensor of the plurality of first strain sensors is located at an interphalangeal joint of a finger of the user and is configured to measure deformation of the interphalangeal joint in a single degree of freedom, and each second strain sensor of the plurality of second strain sensors is located at a metacarpophalangeal joint of the finger and is configured to measure deformation of the metacarpophalangeal joint in two degrees of freedom, the single degree of freedom and the two degrees of freedom both including a degree of freedom corresponding to a bending of the finger; determining a gesture of the user based on the sensor data; outputting a gesture representation corresponding to the gesture on a display interface; and outputting feedback information based on a distance between the gesture representation and a preset gesture.

In some embodiments, the plurality of first strain sensors and the plurality of second strain sensors collectively provide data reflecting deformation of the fingers of the user in no less than 15 degrees of freedom.

One of the embodiments of the present disclosure provides a voice playback method including: obtaining sensor data of a hand of a user based on a glove system, the glove system includes a glove body, and a plurality of first strain sensors and a plurality of second strain sensors arranged on the glove body, when the user is wearing the glove body, each first strain sensor of the plurality of first strain sensors is located at an interphalangeal joint of a finger of the user and is configured to measure deformation of the interphalangeal joint in a single degree of freedom, and each second strain sensor of the plurality of second strain sensors is located at a metacarpophalangeal joint of the finger and is configured to measure deformation of the metacarpophalangeal joint in two degrees of freedom, the single degree of freedom and the two degrees of freedom both including a degree of freedom corresponding to a bending of the finger; determining a gesture of the user based on the sensor data; reading a mapping relationship between gestures and voice packets; and playing a voice message based on the gesture of the user and the mapping relationship.

In some embodiments, the plurality of first strain sensors and the plurality of second strain sensors collectively provide data reflecting deformation of the fingers of the user in no less than 15 degrees of freedom.

One of the embodiments of the present disclosure provides a gesture manipulation method including: obtaining sensor data of a hand of a user based on a glove system, the glove system includes a glove body, and a plurality of first strain sensors and a plurality of second strain sensors arranged on the glove body, when the user is wearing the glove body, each first strain sensor of the plurality of first strain sensors is located at an interphalangeal joint of a finger of the user and is configured to measure deformation of the interphalangeal joint in a single degree of freedom, and each second strain sensor of the plurality of second strain sensors is located at a metacarpophalangeal joint of the finger and is configured to measure deformation of the metacarpophalangeal joint in two degrees of freedom, the single degree of freedom and the two degrees of freedom both including a degree of freedom corresponding to a bending of the finger; determining a gesture of the user based on the sensor data; reading a mapping relationship between gestures and instructions; and executing an instruction based on the gesture of the user and the mapping relationship.

In some embodiments, the plurality of first strain sensors and the plurality of second strain sensors collectively provide data reflecting deformation of the fingers of the user in no less than 15 degrees of freedom.

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the accompanying drawings to be used in the description of the embodiments will be briefly described below. Obviously, the accompanying drawings in the following description are only some examples or embodiments of the present disclosure, and it is possible for those skilled in the art to apply the present disclosure to other similar scenarios based on the accompanying drawings without creative labor. Unless obviously obtained from the context or the context illustrates otherwise, the same numeral in the drawings refers to the same structure or operation.

It will be understood that the term “system,” “engine,” “unit,” “module,” and/or “block” used herein are one method to distinguish different components, elements, parts, sections, or assembly of different levels in ascending order. However, the terms may be displaced by another expression if they achieve the same purpose.

As used herein, the singular forms “a,” “an,” and “the” may be intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprise,” “comprises,” and/or “comprising,” “include,” “includes,” and/or “including,” when used in this disclosure, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

Flowcharts are used in the present disclosure to illustrate operations performed by a system according to embodiments of the present disclosure. It should be understood that the preceding or subsequent operations are not necessarily performed precisely in sequence. On the contrary, the various steps can be processed in reverse order or simultaneously. Meanwhile, other operations may also be added to these processes, or a step or several steps can be removed from these processes.

is a schematic diagram illustrating an exemplary glove system according to some embodiments of the present disclosure. In some embodiments, as shown in, a glove systemmay include a glove body, a plurality of strain sensors (including first strain sensorsand second strain sensors), and a processor. The plurality of strain sensors may be distributed at different positions on the glove body.

The glove systemrefers to a system for capturing a user's hand movements (including finger movements). The glove systemmay recognize hand movements via one or more devices, such as a motion capture glove, a sign language glove, a VR glove, a force feedback glove, a haptic feedback glove, or the like.

The glove bodyrefers to a wearable member for fitting with the hand. In some embodiments, to facilitate the fit of the glove bodyto the hand joints (e.g., including interphalangeal joints, metacarpophalangeal joints, wrist joints, etc.), the glove bodyhas a structure adapted to the shape of the hand and extending along the hand joints. In some embodiments, the glove bodymay be a flexible fabric that tends to deform following the fingers (e.g., bending deformation) when subjected to an external force. In some embodiments, the glove bodymay provide support for the strain sensors to facilitate their arrangement. For example, the glove bodymay include one or more layers of flexible structures (e.g., flexible fabric). The strain sensors may be fixed on a surface of a specific flexible structure layer by means of pasting, pressing or stitching.

The strain sensor refers to an electronic device that may convert deformation in the corresponding regions of the glove bodyinto electrical signals. In some embodiments, the strain sensors (e.g., the first strain sensorand the second strain sensor) are arranged in one or more regions of the glove bodycorresponding to the user's hand joints. When the user wears the glove bodyto generate hand movement, the glove bodyis deformed by an external force. The strain sensors may convert the deformation at the hand joints into electrical signals, and the electrical signals generated by the strain sensors may reflect the deformation direction, deformation magnitude, or the like in the corresponding regions.

In some embodiments, the strain sensors include a plurality of first strain sensorsand a plurality of second strain sensors.

Each first strain sensorof the plurality of first strain sensorsmay be configured to measure deformation in a single degree of freedom. In some embodiments, the first strain sensormay be located at a position where the user's hand has fewer degrees of freedom of deformation, such as an interphalangeal joint, and be configured to measure deformation of the interphalangeal joint in the single degree of freedom. In some embodiments, the first strain sensorincludes a single-degree-of-freedom sensor, such as a single-axis sensor (e.g., an inductive sensor, a resistive bending sensor, or a capacitive stretching sensor).

Each second strain sensorof the plurality of second strain sensorsmay be configured to measure deformation in more than one degree of freedom. In some embodiments, the second strain sensormay be located at a position of the user's hand having more degrees of freedom of deformation, such as a metacarpophalangeal joint, and be configured to measure deformation of the metacarpophalangeal joint in two degrees of freedom. In some embodiments, the second strain sensorincludes a multi-degree-of-freedom sensor, such as a multi-axis sensor (e.g., a capacitive multi-axis movement sensor).

More details regarding the first strain sensor and the second strain sensor may be found in. More descriptions regarding the interphalangeal joints and the metacarpophalangeal joints may be found inand.

is a schematic diagram illustrating joints of a user's hand according to some embodiments of the present disclosure.is a schematic diagram illustrating joint positions of a user's hand according to some embodiments of the present disclosure.

As shown in, metacarpophalangeal joints,,,, andare joints between metacarpal capitula and proximal phalanges (first phalanges adjacent to metacarpal bones). Interphalangeal joints,,,,,,,, andare joints between phalanges.

As shown inand, positions,,,, andon the back of the hand (a dorsal side) correspond to the metacarpophalangeal joints,,,, and, respectively. Positions,,,,,,,, andon the dorsal side of the hand correspond to the interphalangeal joints,,,,,,,, and, respectively.

In some embodiments, the first strain sensorsare arranged on the glove bodyat the dorsal positions of the hand corresponding to the positions,,,,,,,, and, respectively. In some embodiments, the first strain sensormay be arranged at positions on the front (a palm side) or side of the glove bodycorresponding to the interphalangeal joints,,,,,,,, and. In some embodiments, the second strain sensorsare arranged on the glove bodyat positions,,,, andon the back of the hand, respectively.

Each first strain sensormay measure deformation of one interphalangeal joint among the interphalangeal joints,,,,,,,, andin a single degree of freedom. Each second strain sensormay measure deformation of one metacarpophalangeal joint among metacarpophalangeal joints,,,, andin two degrees of freedom.

In the present disclosure, degrees of freedom refer to the dimensions in which an object can move. To illustrate the degrees of freedom of a hand movement, a three-dimensional coordinate system is established for the hand, such as the coordinate system shown in. In the coordinate system, a metacarpophalangeal joint serves as the origin, finger extension direction as the Y-axis, and a direction perpendicular to the hand's plane as the Z-axis. The hand plane refers to a fitted plane of the palm region. For example, when the hand is placed flat on a table surface, either the table surface or a plane passing through the hand and parallel to the table surface may serve as the hand plane. The three-dimensional coordinate system may be used to describe the degrees of freedom of the movement of the hand. Specifically, in the three-dimensional coordinate system illustrated in, the rotation of the hand about the X-axis is defined as bending, the rotation of the hand about the Z-axis is defined as swinging, and the rotation of the hand about the Y-axis is defined as rotating. The interphalangeal joints (e.g., the interphalangeal joints,,,,,,,, and) may only perform bending and rotating. In other words, deformation of an interphalangeal joint corresponds to only one degree of freedom: bending. The metacarpophalangeal joints (e.g., the metacarpophalangeal joints,,,, and) may perform bending, swinging, and rotating. In other words, deformation of the metacarpophalangeal joint corresponds to two degrees of freedom: bending and swinging. As a result, each first strain sensormay measure deformation in the bending degree of freedom for the interphalangeal joint among the interphalangeal joints,,,,,,,, and. When any interphalangeal joint undergoes bending movement, the first strain sensormay measure deformation occurring at the corresponding interphalangeal joint on the glove body. Each second strain sensormay measure deformation in the two degrees of freedom (bending and swinging degrees of freedom) for the metacarpophalangeal joint among the metacarpophalangeal joints,,,, and. When any metacarpophalangeal joint undergoes bending or swinging movement, the second strain sensormay measure deformation occurring at the corresponding metacarpophalangeal joint on the glove body.

Continuing to refer toand, a hand has nine interphalangeal joints in total. The interphalangeal joints,,, andare distal interphalangeal joints relatively distant from the metacarpals, and the interphalangeal joints,,,, andare proximal interphalangeal joints relatively close to the metacarpals. In some embodiments, movement of the distal interphalangeal joints follows movement of previous joints (the proximal interphalangeal joints), and does not bend independently but bend together with the proximal interphalangeal joints. Therefore, the first strain sensors need not be arranged at the distal interphalangeal joints, but may be arranged only at the proximal interphalangeal joints. In some embodiments, deformation data at the distal interphalangeal joints may be generated by mapping deformation data at the corresponding proximal interphalangeal joints. The glove systemacquires deformation data for at least five proximal interphalangeal joints,,,, andin a single degree of freedom via the plurality of first strain sensors, and collects deformation data for five metacarpophalangeal joints,,,, andin two degrees of freedom via the plurality of second strain sensors. In total, the plurality of first strain sensorsand the plurality of second strain sensorscollectively provide deformation data in at least 15 degrees of freedom for the user's fingers.

By inferring or mapping deformation data at the distal interphalangeal joints using deformation data at the proximal interphalangeal joints, the first strain sensors need not be arranged at the distal interphalangeal joints. This reduces the count of strain sensors on the glove body, simplifies the glove system's structure to facilitate product cost control while maintaining deformation data acquisition accuracy.

In some embodiments, the glove systemmay further include a processor. The processoris configured to receive and process data collected by the strain sensors including the plurality of first strain sensorsand the plurality of second strain sensors. For example, the processormay obtain deformation data from the strain sensors and analyze and process the data. For example, the processormay obtain deformation magnitude and deformation angle of each finger joint through a mapping relationship (e.g., a mapping function or a trained machine learning model). In some embodiments, the processormay directly process the data collected by the plurality of strain sensors, or the processormay receive the data collected by the plurality of strain sensors and transmit the data to an external device for data processing. In some embodiments, the processormay perform corresponding functions based on data processing results, such as gesture reproduction, voice playback, gesture manipulation, etc. as described below. In some embodiments, the processormay interact with an external processing device based on data processing results. The external device includes an uplink such as a computer, a cell phone, a VR device, or the like. Interactions may include, for example, controlling a power state of the external device, adjusting volume, managing application processes of the external device (e.g., controlling movement of a game character), providing fitness status feedback, etc.

In some embodiments, the processormay be a single server or a group of servers, and the group of servers may be centralized or distributed. In some embodiments, processormay be local or remote. In some embodiments, the data collected by the strain sensor is transmitted to the processorvia a wire. In some embodiments, the processormay be implemented on a cloud platform. For example, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter-cloud cloud, a multi-cloud, etc., or any combination thereof. In some embodiments, the processormay include one or more processing chips, the processing chips analyzing and processing the received data. In some embodiments, the processoris an independent processing device. In some embodiments, the processormay be an external device or a part of the glove body. For example, the processormay be integrated into the external device or the glove body.

Typically, in order to capture a multi-degree-of-freedom movement of the fingers, gyroscopes must be arranged at non-joint positions on the fingers to measure morphology across finger regions and deduce bending states; or, single-degree-of-freedom strain sensors (e.g., the first strain sensors) may be arranged at finger joints while gyroscopes are arranged at non-joint positions on the fingers. The measurement results of the two are combined to calculate the bending of the finger. Some embodiments in the present disclosure control product costs, reduce subsequent data processing costs, and ensure measurement accuracy by using a combination of two degree of freedom strain sensors (e.g., the second strain sensors) and single degree of freedom strain sensors.

In some embodiments, the glove systemmay further acquire wrist joint movement to more comprehensively identify hand movement. For example, the glove systemmeasures the wrist joint movement using a plurality of inertial sensors or using a combination of the strain sensors and the inertial sensors. More descriptions regarding acquisition of the wrist joint movement may be found inand related descriptions thereof.

is a schematic diagram illustrating an exemplary glove system according to some embodiments of the present disclosure.

As shown in, in some embodiments, the glove systemincludes a glove body, a plurality of first strain sensors (including a first strain sensor, a first strain sensor, a first strain sensor, a first strain sensor, and a first strain sensor), a plurality of second strain sensors (including a second strain sensor, a second strain sensor, a second strain sensor, a second strain sensor, and a second strain sensor), and a processor. The first strain sensor is located in any one or more regions of the glove bodycorresponding to the user's interphalangeal joints (e.g., the interphalangeal joints,,,, and). The second strain sensor is located in any one or more regions of the glove bodycorresponding to the user's metacarpophalangeal joints (e.g., the metacarpophalangeal joints,,,, and). The processoris configured to receive and process data collected by the strain sensors.