New Two-Chip Photonic System Enables Real-Time Reinforcement Learning for AI
Researchers at Xidian University in China have developed a groundbreaking two-chip photonic system designed to overcome significant limitations in spiking neural networks. This advancement enables real-time learning and decision-making using purely light-based processes, eliminating the delays caused by converting signals back to electronics. The technology holds transformative potential for autonomous driving, robotic systems, and embodied intelligence, offering a fast, energy-efficient alternative to traditional computing methods. Currently, photonic spiking neural systems struggle because nonlinear computation steps—essential for learning—require signal conversion to the electronic domain. This conversion introduces latency and diminishes the inherent speed and energy advantages of photonics. To solve this, the research team, led by Shuiying Xiang, has created a large-scale programmable incoherent photonic neuromorphic computing system capable of performing both linear and nonlinear computations entirely within the optical domain. The system features a 16-channel photonic neuromorphic chip with 272 trainable parameters, allowing it to process multiple optical signal streams simultaneously while adjusting connections through learning. The hardware architecture consists of two specialized components: a 16x16 Mach-Zehnder interferometer mesh chip tailored for spiking neural networks, and a distributed feedback laser array chip equipped with a saturable absorber optimized for low-threshold nonlinear spiking activation. To manage the training process, the team developed a collaborative hardware-software framework. Models are first trained globally in software, then fine-tuned directly on the chips to account for physical variations, and finally optimized again in software. This approach ensures high fidelity between the physical hardware and the theoretical models. In testing, the system successfully executed reinforcement learning tasks, demonstrating its ability to learn quickly through trial and error. The researchers evaluated the chips using two standard benchmarks: the CartPole task, where a pole must be balanced on a moving cart, and the Pendulum task, which requires swinging a pendulum to an upright position and maintaining balance. The results were impressive; the hardware decisions matched software simulations with only a 1.5% drop in accuracy for the CartPole task and a 2% drop for the Pendulum task. In some scenarios, the system achieved perfect performance on CartPole and strong results on the more complex Pendulum task. Beyond accuracy, the chips excel in performance metrics. The system achieved a computing latency of just 320 picoseconds, enabling near-instantaneous decisions. For linear computation, the energy efficiency reached 1.39 tera operations per second per watt (TOPS/W) with a density of 0.13 TOPS per square millimeter. Nonlinear computation demonstrated 987.65 gigahertz operations per second per watt (GOPS/W) and a density of 533.33 GOPS per square millimeter. These figures place the technology in the class of high-performance graphics processing units (GPUs) in terms of energy efficiency and computing density, while significantly outperforming them in speed and latency. Looking ahead, the researchers aim to scale the technology further. Their next goal is to fabricate a fully functional 128-channel photonic spiking neural network chip capable of solving complex tasks such as neuromorphic autonomous navigation. However, before the technology can be widely adopted in edge computing scenarios, a compact, hybrid-integrated large-scale chip must be demonstrated. This innovation marks a pivotal step toward making photonics a practical solution for real-world intelligent systems.
