Single Particle Tracking at the Nanoscale, Fang Ning's Team at Xiamen University Uses AI to Play "Rock in the Cell"

In the microscopic world, every cell is a busy city, and molecules are the residents of this city. Imagine if we could track every move of these residents, perhaps we could uncover a corner of the mystery of life. This is the grand goal of scientists to perform 3D single particle tracking (SPT) in living cells. Through this technology, people can observe every move of molecules in cells, so as to understand how they interact with each other and build complex life forms.

However, accurate tracking in the microscopic world is not easy. Imagine tracking a fast-moving bullet in a gunfight movie. It is difficult enough, but molecules move much faster than bullets, and the complexity of their trajectory is beyond imagination.The challenge facing scientists is as difficult as trying to track the trajectory of each snowflake in a sky full of snowflakes.

In order to track the movement of molecules in real time and accurately in the three-dimensional space of cells,Professor Fang Ning's team at Xiamen University has developed an automated, high-speed, multi-dimensional single-particle tracking system based on deep learning.This paper breaks the limitation of nanoparticle rotation tracking in the cell microenvironment and realizes the all-round and precise tracking of single molecules/single nanoparticles in living cells at the nanoscale. It not only tracks their displacement in three-dimensional space, but also observes the rotational motion of molecules/nanoparticles for the first time. Currently, the paper has been published in the authoritative journal Nano Letters.

Research highlights:

• A single particle tracking system integrated with a deep learning algorithm was constructed to overcome the limitation of rotation tracking under low signal-to-noise ratio (S/N) conditions. 

• The system can be used to track the 3D orientation of anisotropic gold nanoparticle probes in living cells with high localization accuracy (<10 nm) and spatiotemporal resolution (0.9 ms)
• The system has better robustness and noise resistance than traditional methods, and has been proven effective by studying the movement of cargo along microtubules in living cells.

Paper address:
https://doi.org/10.1021/acs.nanolett.3c04870
Follow the official account and reply "Single Particle Tracking" to get the full PDF

SPT system: automatic, high-speed, multi-dimensional single particle tracking system

To gain a more comprehensive understanding of the dynamic processes in living cells, the study first developed a multidimensional imaging device:

As shown in the figure above, the imaging device integrates bifocal plane imaging, parallax microscopy and autotracking capabilities.

Dual focal plane imaging

After the light beam passes through the condenser, it passes through the objective lens (OBJ) and the objective scanner (OS), and by inserting a beam splitter (BS) with a reflection-transmittance ratio of 7:3 in the collection light path, the collected signal can be divided into two imaging channels (focus channel and defocus channel), thereby realizing dual-focal plane imaging. A 750mm convex lens (L1 in the figure above) is inserted into the defocus channel to establish an axial separation of about 900 nm between the focus channel and the defocus channel, thereby producing the most suitable defocus pattern.

Parallax microscope

The acquired signal is split into two mirrored and vertically arranged images by a wedge prism (WP). The device establishes the precise relationship between Δy and Δz and constructs a calibration curve by recording the distance between the two images at different z-axis positions of the probe. Then, the z-axis position of the probe is determined by calculating the distance between the two mirrored points of the probe in the xy plane.

Automatic tracking function

The device integrates an automatic feedback tracking system, which consists of a piezoelectric objective scanner (p-725.4CD) and a controller (E-709). When the z-axis movement of the probe causes the distance between the two mirror points to change, the automatic tracking program calculates the distance the target scanner needs to move based on the change value.

Model architecture: input layer + 4 convolutional blocks + 3 fully connected layers

To ensure the diversity of data distribution, this study mixed the same proportion of simulated data and experimental data for training and verification by scaling images, adding different degrees of Gaussian noise, and performing position transformations.

Inspired by the Visual Geometry Group (VGG) model, this study constructed a convolutional neural network model by mapping the input image to three-dimensional directions (azimuth angle φ and polar angle θ).

Generally speaking, the number of convolutional blocks is crucial for feature extraction of multi-layer images with background. Therefore, this study tested CNN architectures with 1-4 convolutional blocks.The results show that the CNN model with 4 convolutional blocks has the smallest error.

From the results,The CNN model of this study ultimately consists of an input layer, 4 convolutional blocks, and 3 fully connected (FC) layers.in:

• The input layer accepts a fixed-size image and converts it into a tensor to pass on.

• The 4 convolutional blocks contain multiple convolutional layers and pooling layers:  

  a. The four convolution blocks contain 64, 128, 256, and 512 convolution kernels respectively. The size of all convolution kernels is 3×3. Each convolution layer passes through batch normalization and rectified linear unit (ReLU) activation function to ensure that the model can converge faster and prevent overfitting, while also enhancing the nonlinear mapping ability of the neural network;

    b. The pooling layer reduces the computational parameters of the network while ensuring translation invariance.

• The three fully connected layers, containing 2048, 2048, and 451 neurons respectively, can integrate the features extracted by the convolutional and pooling layers into higher-level expressions, enabling the network to make more complex decisions and classifications.

By studying the loss curves of the three models CNN-exp, CNN-sim and CNN-sim+exp, the results show that for the simulated data set,The CNN model can reach convergence after 30 epochs.In contrast, it takes about 90 epochs to converge when training with the experimental dataset. In addition, the convergence speed of the CNN-sim+exp model is relatively fast.

Noise resistance evaluation and cargo movement research: CNN model has more advantages

In practical applications, high spatiotemporal resolution and cell viability will affect live cell imaging. Therefore, this study tested the noise resistance and robustness of the CNN model under different signal-to-noise ratio conditions such as 4, 2 and 1.4, and compared it with the traditional CC (correlation coefficient) method.

The research results show that when the signal-to-noise ratio is 4, both CNN and CC methods show good performance and small errors, with the errors being less than 2°; when the signal-to-noise ratio drops to 2, the error increase of the CNN method is only one-fifth of that of the CC method; when the signal-to-noise ratio is 1.4, the CC method cannot distinguish the direction of the particles, while the error of the CNN model is still within an acceptable range.

This shows that in low SNR environment, the CNN model is more noise-resistant and robust than the CC method.

The energy generated by ATP hydrolysis is the "motor" that drives intracellular molecules to transport cargo. Therefore, the characteristic translational and rotational movements of cargo can provide a lot of information about the binding state of cargo and motor proteins, and provide a new perspective for clarifying the interaction between cargo, motor and microtubules. In short, this study studied the dynamic process of motor proteins transporting cargo along the microtubule skeleton in living cells by combining an automatic high-speed multi-dimensional SPT imaging device with a deep learning model (CNN-sim+exp model).

During the entire transportation process, the cargo went through two pause stages and several active transportation stages.

In the first pause phase, the cargo has little rotational freedom, indicating that it is in a tightly attached mode. Between the two pause phases, the cargo is in an active transport phase.The automatic tracking system recorded an axial displacement of about 300 nm.This is difficult to obtain with traditional imaging methods.

In the second pause phase, the movement state of the cargo constantly switches between tight attachment and tethered rotation. In the tight attachment mode, the cargo can be tightly connected to the microtubules through a variety of motor proteins and rarely rotates freely. In the tethered rotation mode, the cargo is loosely connected to the microtubules and constantly searches for and connects to new microtubules. In general,This series of movements highlights the dynamics and complexity of intracellular trafficking and the role of kinesins in facilitating the movement of cargo along microtubule tracks.

After working in the United States for 13 years, he returned to his alma mater

Based on the in-depth research of researchers, the corresponding author of this paper, Professor Fang Ning from Xiamen University, came into our view.Professor Fang Ning is a model of "striving for success and giving back to his alma mater".

In 1998, after graduating from the Department of Chemistry of Xiamen University, Professor Fang Ning conducted doctoral and postdoctoral research at the University of British Columbia in Canada and the Ames National Laboratory of the U.S. Department of Energy, respectively under the supervision of Professor David DYChen and Professor Edward S. Yeung, an internationally renowned analytical chemist.

After working in the United States for 13 years, Professor Fang Ning has been promoted to full professor at Georgia State University. In order to make contributions to the field of optical imaging in China, Professor Fang Ning returned to China full-time in 2021.He joined the School of Chemistry and Chemical Engineering of Xiamen University as a distinguished professor, developing chemical and bio-optical imaging technologies. Relying on these groundbreaking tools, he conducted single-molecule and single-particle research in the fields of nanomaterials, catalysis, and biophysics. To date, he has published more than 90 papers in journals such as Nature Catalysis, Nature Cell Biology, Chemical Reviews, Nature Communications, Science Advances, JACS, and Angewandte Chemie.

At present, Professor Fang Ning has independently built a single molecule, single particle, and optical microscopy laboratory at Xiamen University. Focusing on the optical imaging of molecules and nanomaterials, he has developed six research directions, including single particle rotation tracking technology, Raman spectroscopy + advanced imaging, laser sheet scanning imaging, super-resolution optical imaging, total internal reflection fluorescence, and total internal reflection dark field. He has made outstanding achievements in the fields of single molecule, single particle, and optical microscopy in China.

As early as 2021,Professor Fang Ning's team has developed a new single-particle rotation tracking system and a three-dimensional angle single-particle tracking technology.Breakthroughs have been made in clarifying the mechanism of receptor-mediated endocytosis and real-time analysis of the rotational dynamics of vesicle transport in cells. Faced with the surging wave of AI, the team keenly perceived the outstanding value of AI technology in the field of optical imaging. This study is the first step in using deep learning/AI-assisted imaging to study the life processes of living cells.

Professor Fang Ning's team believes thatIntroducing AI into experiments requires breakthroughs in three major areas: automatic recognition of images, classification and prediction of movement patterns and cell behaviors.This research result is the first phase of the automatic recognition of images based on data generated by computational simulation. Currently, the team is working on the second phase of identifying and determining cellular biological processes.

There is no doubt that when all three stages are completed, researchers may be able to predict the process and results of drug delivery, which will also lead the domestic pharmaceutical industry.

References:
1.https://mp.weixin.qq.com/s/mcO_7Mg40OmyauhbeB91QA
2.https://mp.weixin.qq.com/s/NnHGnBZRRbDOI_2sZu7irA
3.https://mp.weixin.qq.com/s/CUWhLnA-HuvxdZDzMfDxAA