Biosensing is an important medium for interaction between humans and machines, humans and the environment, and machines and the environment. Among them, touch can achieve accurate environmental perception and help users interact with complex environments.
In order to imitate human touch, researchers have developed various sensors to simulate the skin's perception of the environment. However, tactile sensing has high requirements and various parameters, requiring a lot of R&D experience, sufficient literature research and a lot of trial and error experiments, and the R&D cycle is very long.
To this end, researchers from Zhejiang University used support vector machines to optimize the parameters of triboelectric nanogenerator tactile sensors. The optimized tactile sensors can recognize multiple touch patterns and realize Braille recognition. This achievement is expected to provide a new solution for the development of tactile sensors.

Author | Xuecai

Editor | Sanyang

To mimic the human sensory system, researchers have developed a series of skin-like sensors to monitor touch, pressure, vibration, temperature, humidity and other information.Tactile sensors are the most important part of interacting with complex environments, has been widely used in health monitoring, intelligent robots, human-computer environment interaction and virtual/augmented reality.

The ideal sensor should have high sensitivity, wide operating range and excellent repeatabilityThis often requires selecting sensor parameters based on R&D experience or literature research, then conducting multiple trial and error experiments, and finally using the optimized sensor to perform specific tasks.

However, this feedforward design strategy isolates the design and application of sensors, increases the data burden, weakens signal characteristics, and reduces computational analysis efficiency.

Figure 1: Modular prosthesis based on tactile sensors (a) and breast examination sensors (b)

In order to improve the stability of sensor signal recognition,Machine learning algorithms are often used to analyze correlations and subtle differences between signals from different channelsRecently, Yang Geng and Xu Kaichen's research team from Zhejiang University went a step further.The design of the triboelectric nanogenerator (TENG) tactile sensor was optimized by support vector machine (SVM). The optimized sensor can accurately identify 6 dynamic touch patterns and perform text recognition and Braille recognition based on them.. The relevant results have been published in "Advanced Science".

Related results have been published in "Advanced Science"

Paper link:

https://onlinelibrary.wiley.com/doi/10.1002/advs.202303949

Experimental procedures

Overview:Optimization parameters and optimization algorithms

The performance of TENG-based tactile sensors is mainly related to the following parameters:These include the output signal type, electrode density, and the shape and distribution of the surface microstructure, which are the parameters that the SVM algorithm is to optimize.

The sensor on the finger can recognize six touch patterns, including press, tap, move up, move down, move left and move right. The recognition of these six touch patterns is the test target of the sensor, which is used to optimize the design parameters.

The SVM algorithm is used to evaluate the quality of the output signal and select the optimal design parameters. Different from the traditional experience-driven design process, the SVM design process provides a data-driven parameter optimization method that closely links the device design and algorithm.

Figure 2: Tactile sensor optimization process based on machine learning

Machine Learning Models:SVM + CNN

In this study, the scikit-learn library was used in Python to obtain SVM classifiers. There were 10 classifiers in total, and the regularization parameter ranged from 0.1 to 1.0.

At the same time, a one-dimensional convolutional neural network (CNN) was developed in Python based on the PyTorch framework for feature learning and recognition in the verification process. The CNN consists of five convolutional layers followed by a batch normalization function and a linear rectifier unit (ReLU). The model was trained using the adaptive moment estimation (Adam) algorithm with a learning rate of 0.001 and a batch size of 128.

Figure 3: CNN model diagram

Production Process:Laser direct writing + PDMS encapsulation

The production of TENG tactile sensor mainly includes 6 steps:

1. Fabricate interdigital electrodes by infrared laser direct writing technology. The thermal effect of the laser can induce carbonization of the polyimide (PI) film to obtain laser-induced graphene (LIG) patterns;

2. Spin-coat polydimethylsiloxane (PDMS) on the LIG/PI film;

3. After the PDMS solution penetrates into the porous structure of LIG, the PI film is peeled off to obtain the LIG/PDMS film;

4. Treat the film with oxygen plasma to introduce electronegative groups and remove impurities on the film surface;

5. Introduce perfluoroethylene propylene copolymer (FEP) film to enhance the triboelectric effect of the device;

6. Finally, encapsulate with PDMS and perform patterning with laser.

Figure 4: TENG tactile sensor production process

Output signal type:Current and Voltage

The output signal type is highly correlated with the detection performance of TENG, so the researchers first compared the effects of friction voltage and friction current on sensor performance.

First, the data repeatability of these two output signal typesThe distribution of the current signal is irregular and has a large deviation from the average data, while the voltage data has only a small deviation from the average data and is more repeatable.

In addition, the researchers used a linear SVM classifier to classify the data, with a classification accuracy of 94.278% for voltage signals and 95.579% for current signals.The separability and discreteness of the voltage and current signals are [3,636, 11.877] and [3.528, 11,267] respectively.

In order to eliminate the influence of data of different dimensions, the above indicators are normalized, the result of the voltage signal changes from [94,278%, 3.636, 11,877] to [0, 1, 1], and the result of the current signal changes from [95,579%, 3.528, 11,267] to [1, 0, 0]. This shows that the voltage signal has two evaluation criteria, which is more suitable for signal output.

Figure 5: Output voltage and output current response to 6 touch patterns

a&b: voltage and current signal responses to 6 touch modes respectively;

c&d: dimensionality reduction results of output voltage and current signals respectively.

Electrode density:Sparse and dense

Electrode density affects signal complexityThe researchers investigated the device performance when the electrodes were sparsely and densely distributed. After signal dimensionality reduction, it can be seen that the sparsely distributed electrode signals have greater discrimination.

The classification accuracy, separability and discreteness of the sparsely arranged electrode signals are [96.975%, 3.695, 14.759], and those of the densely arranged electrodes are [95,958%, 3.528, 11.542]. After normalization, they are [1, 1, 1] and [0, 0, 0], respectively, indicating that there are three evaluation criteria for the sparsely arranged electrode signals.

Figure 6: Signal responses of sparsely distributed and densely distributed electrodes to six touch patterns

a&b: Signal responses of sparsely distributed electrodes and densely distributed electrodes to 6 touch modes respectively;

c&d: Dimensionality reduction results of signals recorded by sparsely distributed electrodes and densely distributed electrodes, respectively.

Microstructure shape:Fingerprint and Grid

The rough microstructure of the PDMS surface can reduce the influence of viscosity on the signal and improve the signal-to-noise ratio.The researchers compared the effects of fingerprint-like and grid-like microstructures on device performance and found that the data from the fingerprint-like structure was more concentrated, while the signal from the grid-like structure was unstable.

The accuracy, separability and discreteness of the fingerprint-like and grid-like structure signals are [96.50%, 4.227, 14.958] and [95.056%, 3.725, 11.783] respectively. After normalization, they are [1, 1, 1] and [0, 0, 0], indicating that the fingerprint-like microstructure is more suitable for this experiment.

Figure 7: Signal responses of fingerprint-like microstructure (a) and grid-like microstructure (b) to six touch patterns

Microstructure distribution:Channel spacing

In the fingerprint-like microstructure, the spacing between adjacent channels also affects the signal quality.The researchers compared the performance of devices with channel spacings of 100, 200, and 300 µm. The normalized indices of the three were [1, 0.902, 1], [0.042, 1, 0], and [0, 0, 0.228], respectively, indicating that the smaller the spacing, the more conducive it is to signal output.

Figure 8: Signal responses of different channel spacing to 6 touch modes

a: Schematic diagram of the fingerprint-like microstructure channel spacing;

b&c&d: Signal responses when the channel spacing is 300 µm, 200 µm, and 100 µm, respectively.

The device parameters obtained by SVM optimization are summarized as follows:

Table 1: Summary of SVM optimization parameters and results

Experimental verification:Text Recognition

Thanks to the parameters optimized by the SVM algorithm, the TENG tactile sensor can accurately identify 6 touch modes. Therefore, in the voltage measurement mode,TENG tactile sensors can distinguish different letters and thus realize text recognition.

Figure 9: Application of TENG tactile sensor in text recognition

a&b: voltage and current output when sliding up, down, left, and right respectively;

c: Schematic diagram of the text recognition process by touch;

dh: Response signals of TENG tactile sensor to different sentences.

Bionic Verification:Braille recognition

To mimic the touch characteristics of human skin, the TENG tactile sensor was fixed on a robot hand for Braille recognition testing. First, the touch signals of 10 Braille digits were collected by the robot hand. Then, the weak signals during the sliding process were extracted through high-pass filtering.

Figure 10: Touch signals of 10 Braille digits (left) and dimensionality reduction distribution (right)

These Braille numbers have high similarity and are difficult to recognize. To this end, the researchers used CNN for feature learning and classification. The classification accuracy of CNN is 96.12%, only for the three Braille numbers 0, 3, and 8 with high similarity, the accuracy is low.

To verify its feasibility in practical applications, the researchers used the robot arm to perform real-time recognition of 11-digit Braille telephone numbers. After printing out a set of Braille telephone numbers, the robot arm was able to continuously recognize each digit, provide real-time feedback, and display it on a graphical user interface (GUI).

Figure 11: TENG tactile sensor for real-time recognition of Braille phone numbers

The above results show thatThe SVM-optimized TENG tactile sensor can effectively recognize text through contact, enhances the interaction between machines and the environment, and is expected to provide new solutions for disability assistance.

TENG tactile sensor

The earliest biosensors were based on antigen-antibody immune reactions or DNA double-strand reactions.These sensors are very accurate, but are mostly used in molecular biology and are difficult to use for large-scale biological monitoring.

After the concept of "Internet of Things" was proposed,Researchers begin developing high-throughput, low-cost, large-scale biosensors, a series of environmental sensors including tactile sensors came into being.

Figure 12: Development history of biosensors

Early tactile sensors were designed based on resistance, capacitance, piezoelectricity, and thermoelectric effects.These components will greatly increase the size of the sensor, limiting its use in multiple scenarios..

After TENG was proposed in 2012, the research and development of tactile sensors entered a new stage.Nanoscale TENGs can be embedded in various flexible substrates to form wearable sensors, which in turn give rise to concepts such as electronic skin.

However, the development of traditional TENG tactile sensors mainly relies on the experience of the research team, literature research and multiple trial and error.With the help of AI, the research and development of TENG tactile sensors can shift from experience-driven to application-driven, which can greatly reduce the R&D cycle and cost, accelerate R&D implementation, help TENG tactile sensors combine with the demand side, and provide more convenience for people's lives.

Reference Links:

[1]https://www.mdpi.com/1424-8220/17/11/2653

[2]https://onlinelibrary.wiley.com/doi/full/10.1002/adma.202008276