Solving large-scale sparse linear problems is critical in fields like Electronic Design Automation (EDA), Computational Fluid Dynamics (CFD), and advanced optimization. As these applications grow in complexity, traditional solvers struggle with scalability and performance. NVIDIA’s CUDA Direct Sparse Solver (cuDSS) addresses these challenges by enabling efficient, large-scale sparse solving with minimal code changes. It supports hybrid memory mode, multi-GPU (MG) mode, and multi-node multi-GPU (MGMN) mode, allowing users to tackle problems far beyond the memory limits of a single GPU. For problems exceeding 10 million rows and a billion nonzeros, upgrading to 64-bit integer indexing is essential. Starting with cuDSS version 0.7.0, developers can use int64_t indices and CUDA_R_64I data types, enabling matrices with far more nonzeros than previously possible—though the number of rows and columns remains limited by 32-bit constraints unless using 64-bit indexing. Hybrid memory mode allows solving massive problems by leveraging both CPU and GPU memory. While data transfers between CPU and GPU introduce latency, modern NVIDIA systems with high-bandwidth interconnects like Grace Blackwell minimize this overhead. To enable hybrid mode, developers must call cudssConfigSet with CUDSS_CONFIG_HYBRID_MODE before the analysis phase. Users can also set device memory limits to control GPU memory usage. cuDSS prioritizes GPU memory usage when possible to maximize performance. Developers should query minimum device memory requirements via cudssDataGet and set limits appropriately to avoid out-of-memory errors. Multi-GPU mode (MG mode) lets developers scale across all GPUs in a single node without writing distributed communication code. cuDSS manages internal GPU communication automatically, eliminating the need for MPI or NCCL. This is especially beneficial on Windows systems where MPI-aware CUDA support is limited. MG mode improves performance through strong scaling—solving fixed-size problems faster by using more GPUs. Performance gains are evident even at modest GPU counts, as shown in benchmarks using 31-million-row matrices on DGX H200 nodes. To set up MG mode, developers first detect available GPUs and assign device indices. Then, cudssCreateMg initializes the solver handle. The solver configuration must include device count and indices via cudssConfigSet. Hybrid memory limits must be set individually per device using cudaSetDevice before calling cudssDataGet or cudssConfigSet. This ensures optimal memory allocation across all GPUs. For problems too large for a single node, MGMN mode enables distributed solving across multiple nodes. This requires a communication layer—such as CUDA-aware Open MPI or NVIDIA NCCL—configured via CUDSS_DATA_COMM. Developers define the communicator pointer, and cuDSS handles all underlying communication. Custom communication layers are also supported through a shim abstraction. MGMN mode supports both pre-distributed input and automatic distribution, with developers responsible for placing data correctly across nodes and devices. Performance depends heavily on CPU-GPU-NIC binding, which must be carefully optimized. While powerful, MGMN mode has current limitations and requires careful setup. In summary, cuDSS empowers engineers and scientists to solve increasingly large sparse linear systems efficiently. With support for 64-bit indexing, hybrid memory, multi-GPU, and multi-node scaling, it provides a flexible, high-performance solution for next-generation engineering and scientific computing. Developers are encouraged to explore the advanced features and logging capabilities in the cuDSS documentation to debug and optimize their code.