Polars GPU Engine Tackles Large Datasets: Strategies for Handling Data Beyond VRAM Limits

In fields like quantitative finance, algorithmic trading, and fraud detection, data practitioners often need to process massive datasets exceeding hundreds of gigabytes (GB) quickly and efficiently. Polars, a rapidly growing data processing library, addresses this challenge with its GPU engine, powered by NVIDIA cuDF, which significantly accelerates compute-bound queries. However, a common issue arises when the volume of data surpasses the available VRAM (dedicated GPU memory), which is usually smaller than system RAM. To tackle this, Polars offers two key options within its GPU engine: Option 1: Unified Virtual Memory (UVM) Unified Virtual Memory (UVM) is a technique that extends the GPU's memory capacity by allowing it to utilize system RAM. This means that when your dataset exceeds the VRAM, the excess data is spilled over to the system RAM, preventing out-of-memory errors and enabling you to work with larger datasets. The GPU can seamlessly access this data from system RAM as needed, although there is some performance overhead due to data migration. To mitigate this overhead, developers can use the RAPIDS Memory Manager (RMM), a library that provides fine-grained control over GPU memory allocation. With RMM, you can optimize the allocation and management of memory, reducing the performance impact. For a detailed understanding of UVM's performance and configuration, refer to the Polars documentation titled "Introducing UVM for Larger than VRAM Data on the Polars GPU Engine." Option 2: Multi-GPU Streaming Execution For datasets that reach terabyte (TB) scales, Polars offers an experimental multi-GPU streaming execution feature. This approach distributes the workload across multiple GPUs, enhancing processing efficiency and scalability. The multi-GPU streaming executor works by partitioning the data and processing it in batches. It takes the optimized internal representation (IR) graph generated by Polars and rewrites it for batched execution. These partitions are then independently processed, allowing for parallel workloads and faster data handling. The streaming executor supports both single-GPU and multi-GPU execution via the Dask synchronous and distributed schedulers, respectively. Users can control various parameters, such as join strategies and partition sizes, to fine-tune performance. Tests have shown that the streaming executor performs exceptionally well on large datasets. For instance, it processed all 22 queries on the 3 TB PDS-H benchmark in just a few seconds. For hands-on experience, users can explore the example notebook provided and try the multi-GPU streaming execution on their datasets. For a comprehensive look at how the streaming executor functions, watch the NVIDIA GTC Paris session, "Scaling DataFrames with Polars." Choosing the Right Approach Both UVM and multi-GPU streaming execution are effective solutions for handling datasets larger than your GPU VRAM in the Polars GPU engine. The choice between them depends on your specific requirements: UVM: Best for single-GPU setups and moderately large datasets where simplicity and ease of use are crucial. It requires minimal code changes and can be optimized with RMM. Multi-GPU Streaming Execution: Ideal for datasets on the order of hundreds of GBs to a few TBs, where maximum performance and scalability are necessary. It provides more advanced control over memory and execution but is experimental and may require more setup. Industry Insights and Company Profiles The integration of UVM and multi-GPU streaming execution in Polars demonstrates the library's commitment to addressing real-world challenges faced by data scientists and engineers. Polars is known for its high performance and user-friendly API, making it a preferred choice for those working with large datasets. NVIDIA's support through cuDF and RAPIDS highlights the growing importance of GPU-accelerated data processing. Companies like Meta and Amazon, which have invested in Polars, recognize the potential for speeding up data-intensive workflows, particularly in areas like AI and machine learning. Industry experts laud Polars' approach to managing large datasets, noting that these features will likely become standard tools in the data processing toolkit. They predict that as AI models grow even more complex and data volumes continue to rise, the ability to handle large datasets efficiently will be a critical differentiator for data processing libraries.