Tactile Sensor Based on Flexible Magnetic Film
Tactile perception is one of the important capabilities of intelligent robots and human-computer interaction, but how to achieve high-precision and fast-response tactile sensing still faces many challenges. Traditional tactile sensors are often limited by signal coupling in force measurement, and it is difficult to accurately distinguish normal force from tangential force. At the same time, due to the limitation of the resolution of the sensor unit, the spatial resolution of tactile perception is also restricted.Self-decoupling and super-resolution tactile sensing technology based on flexible magnetic film provides a solution to these problems.
This technology designs a flexible magnetic film tactile sensor through an orthogonally magnetized Halbach array. When an external force acts on the sensor surface, the flexible magnetic film deforms, causing the magnetic field distribution to change.The integrated Hall sensor can capture magnetic field changes and achieve self-decoupling of three-dimensional forces through signal processing. In addition, by using a super-resolution algorithm, the sensor can achieve positioning accuracy higher than the physical resolution, greatly improving the spatial resolution of tactile perception.
On December 26, at the fifth online sharing event of "Newcomers on the Frontier" hosted by the Embodied Touch Community and co-organized by HyperAI,Dr. Yan Youcan, a postdoctoral researcher from the French National Center for Scientific Research, spoke on the topic of "Self-decoupling and super-resolution tactile sensing based on flexible magnetic films".The design and application of tactile sensors based on flexible magnetic films were shared with everyone, and the focus was on how to use the Halbach array with orthogonal magnetization to achieve self-decoupling of three-dimensional forces.
HyperAI has compiled and summarized Dr. Yan Youcan’s in-depth sharing without violating the original intention.
Design and application exploration of tactile sensors
Current status and challenges of robot tactile perception
As we all know, human hands can perform many complex operations, such as cutting potatoes and pouring water, mainly due to the rich tactile perception of the hands. This tactile perception is also crucial for robots, which can not only help robots perceive the external environment, but also interact with the environment. When tactile sensors are installed on the fingertips of robots, robots can perform certain delicate operations even without visual feedback.
However, current robots' tactile perception is still mainly limited to the fingertips compared to the human tactile system, where humans have tactile receptors distributed throughout the body.In order to give robots tactile perception capabilities, researchers have developed various types of tactile sensors, including optical, piezoresistive, and capacitive.These sensors have their own advantages and disadvantages, but the common challenge they face is how to decouple normal force and tangential force through a simple sensor structure and calibration process.
Design of tactile sensor based on flexible magnetic film
To address this problem, we designed a tactile sensor based on a flexible magnetic film (as shown in the figure below). Its structure consists of three layers: the top layer is a sinusoidally magnetized flexible magnetic film, which is a mixture of polydimethylsiloxane (PDMS) and neodymium iron boron magnetic powder; the middle layer is a flexible elastic layer; and the bottom layer is a circuit board (PCB) with a Hall sensor.When an external force acts on the sensor surface, the flexible magnetic film deforms, causing a change in the magnetic field. This magnetic field change is captured by the Hall sensor and decoupled from the external force through signal processing.
The necessity of decoupling the magnitude of external force lies in that in certain scenarios or applications, efficient control can be achieved through force feedback using simple physical principles.As shown in the figure below, in the experiment, we demonstrated adaptive grasping of eggs based on tactile feedback. When the sensor detects a downward pull force in the tangential direction, the control system will increase the holding force accordingly so that the resultant force is always within the friction cone, thereby maintaining the stability of the grasp. Without tactile feedback, the egg may slip due to lack of timely adjustment. Therefore, force decoupling is of great significance for achieving precise control.
Halbach array with orthogonal magnetization
The Halbach array magnet used in this study is unique in that it can enhance the magnetic field on one side and weaken it on the other side.As shown in the figure below, when the magnetic material is magnetized in a sinusoidal mode along the x-direction and the z-direction and superimposed, the magnetic field can be significantly enhanced on one side, while the other side is close to zero. This feature is widely used in motors, magnetic levitation tracks, refrigerator magnets and other fields.
In addition, due to the magnetic flux B below the Halbach arrayx and Bz The original magnetic flux is coupled with the x and z coordinates, so it is impossible to use this original magnetic flux to decouple the force. However, after calculation, we found that the magnetic field strength B (i.e. Bx and Bz The square root of the sum of the squares is only related to the z direction, while the quantity R related to the magnetic field directionB(i.e. Bx/Bz) is only related to the x direction (as shown in the figure below). This result is also verified in actual measurements: the value of the magnetic field intensity B is only affected by the z direction, while RB The value of is only affected by the x direction.
Based on this property, the normal force Fz It can be expressed by the magnetic field strength B, the tangential force Fx The magnetic field direction RB This represents the natural decoupling of the x-direction and the z-direction.This decoupling feature significantly simplifies the sensor calibration process. However, the limitation of this magnetization method is that the magnetic field strength distribution along the y direction is uniform, and the force in the y direction cannot be sensed.
To solve this problem, we superimposed two layers of sinusoidally magnetized magnetic films.Since the magnetic field follows the superposition principle, the superimposed magnetic field changes along the three directions of x, y, and z. As shown in the figure below, it is proved by deduction that under small deformation conditions, the superimposed magnetic field intensity B and the magnetic field direction parameter Rxz and Ryz It also has the characteristic of natural decoupling. Therefore, we can use these three parameters to deduce the magnitude of the force in the x, y, and z directions, thereby reducing the calibration complexity and achieving faster calibration.
Application of tactile sensors based on three-dimensional force decoupling
Based on the above sensing principle and force decoupling method, we can realize the measurement of distributed force.The figure below shows a tactile sensor composed of 24 sensing units, which is shaped like the cross-section of a knee joint. By calibrating the stiffness coefficient of the elastic layer in advance and measuring the displacement of each sensing unit along the x, y, and z directions, we can obtain real-time distributed force.
The figure below shows the force distribution measured by the sensor when the knee joint rotates. The x direction represents the left-right direction, the y direction represents the front-back direction, and the z direction represents the vertical direction. Comparing the sensor readings with the measured values of the ATI sensor, the combined force of the two is highly consistent.
Furthermore, based on the physical model of the sensor, expressions for its sensitivity and range can be derived.As shown in the figure below, the sensitivity S along the z directionz It indicates the degree of change in sensor response under the action of unit input (or unit pressure). The greater the change in response, the higher the sensitivity. The sensitivity in the x, y, and z directions is related to the thickness of the elastic layer, the Young's modulus of the elastic layer, and the magnetization period. At the same time, the range is also related to the above three parameters, but the sensitivity and range are affected by these parameters in opposite ways. Therefore, it is necessary to strike a balance between higher sensitivity and larger range according to specific needs.
To this end, we have designed three different sensors for different application scenarios, each with different parameters in terms of sensitivity and range.As shown in the figure below, the application of the first sensor has just been demonstrated. The actual applications of the remaining two sensors are as follows:
The first application scenario is tactile-based robotic arm teaching.As shown in the figure below, we install a 3×3 array of sensors at the end of the robot arm for teaching tasks (such as making coffee). When external forces act on the sensor, the sensor can calculate the forces and torques in the x, y, and z directions in real time. By multiplying the sensor readings with the gain matrix and adding the current position of the robot arm, the latest position of the robot arm can be obtained, thus completing the teaching operation of coffee making.
The second application scenario is knee pad tactile sensing.As shown in the figure below, the sensor uses a flexible PCB and is installed inside the knee pad to monitor the three-dimensional contact force between the knee pad and the skin when a person is walking. The experimental results show that the sensor response under different motion states (such as walking, squatting, and running) is significantly different. When the support force of the knee pad increases, the force value measured by the sensor also increases. This is because a greater force is required to overcome the support force provided by the knee pad when bending the leg.
Research and application of tactile super-resolution
Tactile super-resolution is a method for restoring tactile information with high precision by superimposing and interpolating signals from multiple sensing units or between adjacent sensing units.When the sensing unit deforms as a whole under the action of external force, we can achieve 3D force measurement through the above force decoupling algorithm. However, when an object is applied between two sensing units and the sensor surface is continuous, how to accurately infer the position and magnitude of the applied force is a key issue. To solve this problem, we propose a super-resolution model. Its goal is to accurately estimate the position and magnitude of the contact force through a super-resolution algorithm when a force is applied at any position of the sensor.
Development of super-resolution algorithms
The concept of super-resolution algorithm was first proposed by Nathan in 2015.He pointed out that when the receiving areas (i.e., perception fields) of the sensor units overlap, their physical resolution is defined as the minimum distance between two points that can be clearly distinguished. This resolution can be further improved through super-resolution algorithms. The figure below shows some representative studies from 2015 to 2024.
In our 2021 research, we proposed a super-resolution algorithm that combines qualitative and quantitative analysis.As shown in the figure below, on a 3×3 sensor array, when a small ball is pressed on its surface, we first locate the preliminary position of the small ball (located at sensor unit No. 5) based on the maximum response value. Then, by the sign of the magnetic flux in the x and y directions (positive or negative), we further infer whether it is located on the upper or lower side, and on the left or right side of the sensor unit, thereby doubling the spatial resolution of the sensor (compared to the physical resolution). Since this method is a qualitative analysis, it can be used for contact objects of different shapes.
To further improve the positioning accuracy, we use a multi-layer perceptron (MLP) model to perform regression analysis on the x and y directions to obtain a more accurate contact position. With the contact position, we can combine the reading of the magnetic flux in the z direction and determine the pressing depth through a pre-calibrated lookup table, and then calculate the force. However, this method is only applicable to single-point contact and cannot handle multi-point contact situations.
To solve the problem of multi-point contact, we propose a method based on geometric model.As shown in the figure below, when a spherical object is pressed on the sensor surface and moves from Position 1 to Position 2, sensor S1 The measured magnetic field is equivalent to the magnetic field at the horizontal position X1 Move to X2 By analyzing RB The relationship between the value and the x direction can be obtained from RB The contact position x is calculated from the curve and the force is further deduced.
The specific workflow is shown in the figure below. This method can achieve a 15-fold increase in resolution.
The figure below shows the performance of the high-resolution algorithm based on the geometric model in real-time positioning of contact positions and measuring force magnitude. We further tested objects of different shapes, including spherical and non-spherical objects (assuming that non-spherical objects can be equivalent to spherical objects of a certain diameter).This method can detect multiple points of contact at the same time, and has initially realized the application of tactile super-resolution in complex scenes. However, it is currently impossible to achieve high-resolution measurement of force distribution, which remains a challenge to be overcome in the future.
Robot tactile diagnosis based on Bayesian optimization
By combining tactile sensors and Bayesian optimization algorithms, we have achieved rapid robotic palpation (simulating a doctor's palpation, that is, pressing different parts of the body to diagnose lesions).
The figure below shows the experimental setup, where a 3D printed hard mass is used to simulate a tumor, and the silicone covering it simulates human tissue. The blue area in the lower right corner represents the ground truth of the hardness distribution of the hard mass. Here we designed the tactile sensor into an arc structure to reduce friction and increase sensitivity. The experimental goal is to find the location of the hard mass with the least number of presses and achieve accurate segmentation of the mass.
Through the Bayesian optimization algorithm, the tactile sensor can find the outline of the first hard block within 15 iterations, the second hard block within 20 times, and the boundary of the third hard block within 30 times.
Next, we cluster the hardness distribution estimates to find the center of mass of each mass, and slide the sensor in different directions from the center of mass to find the boundary points of the mass in each direction (by detecting Bx The found boundary points are then fitted with cubic spline interpolation to obtain the accurate segmentation result of the mass.
For complex shapes with their center of mass outside the body (such as C-shaped or O-shaped masses), experiments show that this method can also accurately find their boundaries.
Braille recognition and material classification
Tactile sensors can also be used for Braille recognition.When the sensor slides over the braille, the magnetic flux in the x, y, and z directions will change differently depending on the protrusions of the braille. For this reason, we trained an LSTM neural network, the input of which is the magnetic flux change of the sensor, and the output is the corresponding letter or symbol, which can achieve a recognition success rate of 97%.
In addition, fabric material classification can also be achieved using the same sensor system and LSTM neural network.By pressing the sensor on the surface of the fabric and sliding it, we found that the sensor's response along the x, y, and z directions reflects the characteristics of the fabric's hardness, friction, and roughness. Based on this characteristic, we achieved a 99% fabric recognition success rate.
Summary and Outlook
We designed highly sensitive tactile sensor hardware and developed force decoupling and super-resolution algorithms, which we applied to practical scenarios such as tactile diagnosis and material classification.However, there are still many unresolved issues in this field, including the optimization and expansion of sensor simulation; the development of general super-resolution algorithms suitable for complex contact scenarios; and the realization of full-body tactile perception similar to human skin.
In the future, we will continue to explore the potential of tactile sensors and promote their widespread application in intelligent robots and human-computer interaction.