HyperAIHyperAI

Command Palette

Search for a command to run...

Chinese SKA Node Passes Key International Grid Testing, Paving Way for Global Astronomical Data Management

The first phase of the Chinese node of the Square Kilometre Array (SKA) Regional Centre Network (SRCNet v0.1) has successfully passed the international interconnection test organized by the SKA Observatory headquarters. This achievement makes China the fifth country to pass the test, following Spain, Switzerland, Sweden, and the United Kingdom. China is also the first and only non-European node to join the SRCNet, a key component of the SKA project's core scientific data processing and analysis platform. The SKA is the largest radio telescope ever built, often referred to as the "eye in the sky" for exploring the origins and evolution of the universe. Once operational, it is expected to generate over 1,000 petabytes of scientific data annually. To manage this vast volume of data, the SKA international organization is establishing a globally distributed network of regional centers, known as SRCNet. The Chinese node of SRCNet will handle data distribution, processing, and storage, serving as a critical infrastructure for Chinese astronomers to participate in SKA research and a strategic point for China's involvement in major international scientific projects. The international interconnection test comprehensively evaluated the end-to-end data processing capabilities of the SKA regional centers. The Chinese node of SRCNet v0.1 achieved significant results in several key technical areas: System Architecture for the SKA Regional Data Center: The Chinese node adopted a cloud-native architecture for its full-stack deployment, creating a flexible computing cluster based on Kubernetes container orchestration technology. This design allows for dynamic allocation and intelligent scheduling of computing resources and can be easily migrated and replicated in other general cloud environments. The cloud-native approach has improved resource utilization by over 35%, addressing the limitations of traditional high-performance computing models. The system integrates microservices and containerized deployment, supporting multi-level service discovery, real-time fault detection, and automated maintenance, which places it at an advanced level compared to international nodes. As the earliest node to fully implement cloud-native technology in the SRCNet, the Chinese solution has not only reduced system construction and maintenance costs but also contributed to the standardization of SKA's global distributed scientific computing platform. High-speed Data Transmission System: The Chinese node built a 10 gigabits per second (Gbps) cross-continental data transmission system, ensuring stable and concurrent data exchange with major nodes in Europe and Australia. This system employs multi-level transmission optimization strategies, incorporating two optimized protocol engines and adaptive multi-threaded flow control mechanisms. These techniques overcome challenges such as latency fluctuations, packet loss, and bandwidth instability during cross-continental network transmissions. During the joint testing phase, which lasted nearly three months and involved 3.6 million data distribution and transmission tests, the failure rate was kept below 1%. The system maintained over 95% link availability and an average effective bandwidth of more than 8 Gbps, with peak speeds reaching 10.9 Gbps. This development sets a solid foundation for the global distribution of daily petabyte-level observational data from the SKA. PB-scale High-performance Distributed Storage System: To meet the SKA's immense data storage needs, the Chinese node has developed and is operating an advanced PB-scale high-performance distributed storage system. This system uses a pioneering "hot-warm" tiered architecture, leveraging high-speed NVMe solid-state drives within the computing cluster to create distributed storage. This design brings data closer to computing resources, allowing for rapid data processing for time-sensitive scientific projects, while high-performance SAS hard drives ensure optimal balance between data access efficiency and resource utilization. The system supports object storage, multiple copies for disaster recovery, automatic fault tolerance, and stringent access control, meeting the demands of secure data storage and multiple user concurrent access. The successful interconnection of the SRCNet v0.1 Chinese node marks the first step toward its full-scale construction. It holds the potential to transform China from a consumer to a producer and manager of data for large international telescopes. This achievement also provides a robust infrastructure for Chinese scientists to make original breakthroughs in cutting-edge research areas such as dark matter, dark energy, the origin of the universe, and gravitational waves. The research and development of the Chinese node were supported by the Ministry of Science and Technology, the Chinese Academy of Sciences, and the Shanghai Science and Technology Commission. Additional contributions came from the Zhejiang Lab, which was involved in the technical development and international joint testing, and the Computer Network Information Center of the Chinese Academy of Sciences, which provided network support. The international certification of the Chinese node is a testament to its technical maturity and readiness for the global SKA data processing network.

Related Links