AT&T has made a strategic shift from using ChatGPT to employing open-source AI models for managing its customer service calls, a move that has proven both cost-effective and efficient. The company receives around 40 million customer service calls annually, which contain valuable insights into customer issues and satisfaction. Previously, these calls were transcribed and manually sorted into 80 categories by employees, a process that was time-consuming and prone to delays. The manual categorization aimed to minimize customer churn, a term used to describe the rate at which customers leave a service. Hien Lam, a senior data scientist at AT&T, detailed the transformation during a presentation at Nvidia's GTC Conference in March. Initially, the company used ChatGPT to automate the call categorization process. While ChatGPT provided high-quality outputs and helped save 50,000 customers annually, it was prohibitively expensive and often faced availability issues for the required graphics processing units (GPUs). To address these challenges, Lam and Ryan Chesler, a principal data scientist at H2O.ai, collaborated to develop a more flexible and cost-efficient solution. They theorized that by combining multiple open-source AI models with specific skills, they could achieve similar accuracy at a fraction of the cost while ensuring data privacy. The team selected and fine-tuned three open-source models for their system. The first model, a basic one, was capable of handling simpler tasks such as sorting calls into roughly 25% of the categories. For example, it could easily identify calls mentioning competitor names. The second model, Danube, a smaller yet robust model developed by H2O.ai, managed about half of the calls after being tailored to AT&T’s requirements. The most complex calls, requiring deeper analysis, were processed by Meta’s Llama 70B model, which is larger and more resource-intensive but used sparingly to keep costs down. This open-source patchwork solution significantly reduced processing time and expenses. According to Lam, it now costs only 35% of what AT&T was spending on ChatGPT, achieving 91% relative accuracy. The processing time has also been cut from 15 hours to just under five hours per day’s worth of call summaries. With the success of this initial implementation, the team is now focused on improving the system further. Their next goal is to process call summaries in real-time, allowing immediate feedback and action. "Because it takes 4.5 hours for a full day, we are looking to do it real-time after you hang up with AT&T. We could get those outputs immediately," Lam said. Industry experts view this approach as a pioneering step in leveraging open-source AI to optimize customer service operations. The ability to control costs and maintain data privacy while achieving high accuracy and speed sets a benchmark for other companies in the telecommunications sector. AT&T’s strategic move highlights the growing trend towards custom AI solutions that balance performance with operational efficiency, potentially giving the company a competitive edge in retaining customers and addressing issues promptly. AT&T, founded in 1885, is one of the largest telecommunications companies globally, offering a wide range of services including mobile, broadband, and television. The company’s commitment to innovation, particularly in AI, underscores its aim to stay at the forefront of technological advancements and customer service excellence. H2O.ai, a leading open-source AI platform, has also gained recognition through this collaboration, showcasing the effectiveness of open-source models in practical business applications.