Rafay Systems, a leading provider in cloud-native and AI infrastructure management, has announced the general availability of its Serverless Inference offering. This new feature aims to streamline the deployment of Generative AI (GenAI) models for NVIDIA Cloud Providers (NCPs) and GPU Cloud providers, making it easier for enterprises to adopt and benefit from cutting-edge AI technologies without the burden of complex infrastructure management. Rafay’s Serverless Inference Offering The Serverless Inference offering is a token-metered API designed to run both open-source and privately trained or tuned Large Language Models (LLMs). Many NCPs and GPU Cloud providers currently use the Rafay Platform to deliver multi-tenant, Platform-as-a-Service (PaaS) experiences, enabling their customers to seamlessly consume compute resources and AI applications. With this new feature, these providers can now offer Serverless Inference as a turnkey service at no additional cost, significantly accelerating the time-to-market and maximizing return on investment (ROI). According to industry projections, the global AI inference market is poised to reach $106 billion by 2025 and $254 billion by 2030. Recognizing the potential, Rafay’s Serverless Inference is specifically designed to help NCPs and GPU Clouds tap into this rapidly growing market by addressing key challenges such as automated provisioning, developer self-service, and rapid deployment of new GenAI models. Key Benefits and Capabilities Seamless Developer Integration: The offering includes OpenAI-compatible APIs, which require zero code migration for existing applications. Secure RESTful and streaming-ready endpoints enable developers to integrate GenAI workflows into their applications swiftly, accelerating the time-to-value for end customers. Intelligent Infrastructure Management: Rafay's platform features auto-scaling GPU nodes that dynamically allocate models, optimizing resources across multi-tenant and dedicated isolation options. This eliminates over-provisioning while ensuring strict performance Service Level Agreements (SLAs). Built-in Metering and Billing: The token-based and time-based usage tracking provides granular consumption analytics, integrating with existing billing platforms via comprehensive metering APIs. This enables transparent and consumption-based pricing models, giving customers clear insight into their resource usage. Enterprise-Grade Security and Governance: The platform ensures robust security through HTTPS-only API endpoints, rotating bearer token authentication, detailed access logging, and configurable token quotas per team, business unit, or application, meeting stringent enterprise compliance requirements. Observability, Storage, and Performance Monitoring: Rafay offers end-to-end visibility with logs and metrics stored in the provider’s own storage namespace. It supports high-performance backends like MinIO and Weka, ensuring complete transparency in infrastructure and model performance. Centralized credential management further enhances this transparency. Immediate Availability and Upcoming Features: The Serverless Inference offering is available now to all Rafay Platform users delivering multi-tenant GPU and CPU-based infrastructure. Fine-tuning capabilities, which will further streamline the process of optimizing AI models, are slated to be rolled out soon. Strategic Vision and Industry Impact Haseeb Budhani, CEO and co-founder of Rafay Systems, emphasizes the importance of rapid integration and consumption of GenAI models. He noted, "With the new Serverless Inference offering, our customers and partners can now deliver an Amazon Bedrock-like service, enabling access to the latest GenAI models in a scalable, secure, and cost-effective manner. Developers and enterprises can integrate GenAI workflows into their applications in minutes, not months." Budhani sees this as a significant step in helping NCPs and GPU Clouds transition from offering mere GPU-as-a-Service to providing comprehensive AI-as-a-Service solutions. This shift is crucial as many enterprises are now focusing on developing agentic AI applications to enhance their business offerings. Industry Insights and Expert Evaluations Industry insiders view Rafay’s Serverless Inference as a game-changer in the AI market. The ability to deliver GenAI models through a simple, token-metered API without the need for extensive infrastructure management is seen as a major breakthrough. This not only accelerates the adoption of AI but also reduces operational overhead and costs, making it an attractive solution for both startups and established enterprises. Gartner has recognized Rafay as a Cool Vendor in Container Management, highlighting the company’s innovative approach to infrastructure orchestration. Additionally, GigaOm named Rafay a Leader and Outperformer in the GigaOm Radar Report for Managed Kubernetes, underscoring its strong position in the tech industry. Company Profile Founded in 2017, Rafay Systems is dedicated to transforming CPU and GPU-based infrastructure into a strategic asset for enterprises and cloud service providers. The company’s GPU PaaS™ stack simplifies complex infrastructure management, enabling self-service workflows for platform and DevOps teams within a single, multi-tenant offering. Rafay’s platform also helps optimize resource costs and accelerate the delivery of cloud-native and AI-driven applications. Customers like MoneyGram and Guardant Health rely on Rafay to underpin their modern infrastructure strategies and AI architectures. For more information, visit Rafay’s website at www.rafay.co, and follow them on social media platforms like X and LinkedIn.