In the semiconductor industry, the precision of chip manufacturing is paramount, with even the smallest defect capable of rendering a device nonfunctional. As devices grow more complex, traditional methods of defect classification are increasingly inadequate. Historically, convolutional neural networks (CNNs) have been the go-to solution for automated defect classification (ADC), but they face significant limitations in data demands, semantic understanding, and adaptability. To overcome these challenges, NVIDIA is pioneering the use of generative AI, vision language models (VLMs), and vision foundation models (VFMs) to transform defect detection in semiconductor fabrication. CNNs require large volumes of labeled data for each defect type, making them impractical for rare or emerging defects. They also lack the ability to interpret context, perform root-cause analysis, or integrate multimodal data. Frequent retraining is needed to adapt to new tools, processes, and product lines, which slows down production and increases costs. These constraints often lead to reliance on manual inspection, which is slow, inconsistent, and unscalable. NVIDIA’s new approach leverages VLMs and VFMs to address these shortcomings. VLMs like Cosmos Reason combine visual perception with natural language reasoning, enabling systems to not only classify defects but also generate explanations, answer questions, and compare results against known good patterns. This allows for more intelligent, context-aware analysis of wafer map images, where spatial distribution of defects is critical. For example, a VLM can identify a center ring defect and attribute it to chemical contamination, providing actionable insights for process engineers. The workflow for using Cosmos Reason begins with a curated dataset, such as the public WM-811k wafer map dataset. After data preparation, the model is fine-tuned using supervised learning. The result is a system that achieves over 96% accuracy in defect classification—far surpassing zero-shot performance—while significantly reducing the need for manual labeling and visual inspection. At the die level, where defects are smaller and more complex, NVIDIA’s approach uses vision foundation models like NV-DINOv2. These models are pre-trained on massive, diverse image datasets and then adapted to the semiconductor domain through self-supervised learning (SSL). The process involves three stages: starting with a general VFM, adapting it to domain-specific images using unlabeled data, and finally fine-tuning with a small set of labeled examples through linear probing. This method dramatically reduces the need for labeled data. In one case, a leading manufacturer used one million unlabeled images and just 600 labeled samples to improve defect detection accuracy from 93.84% to 98.51%. The use of SSL not only enhances model performance but also increases yield and productivity, with reported gains of up to 9.9%. The end-to-end workflow, powered by the NVIDIA TAO Toolkit, includes data preparation, domain adaptation with SSL, task-specific fine-tuning, and deployment via ONNX and TensorRT for high-speed inference. The system can be integrated into real-time inspection pipelines using DeepStream, enabling continuous monitoring across front-end and back-end processes. These advances are paving the way for smart fabs—manufacturing environments where AI-driven systems continuously learn, adapt, and improve. Beyond defect detection, video analytics agents based on NVIDIA’s Blueprint for Video Search and Summarization are being used to monitor plant safety, enforce PPE compliance, and improve operational efficiency. By combining generative AI with accelerated computing, NVIDIA is enabling faster, more accurate, and more scalable ADC systems. These innovations reduce time-to-deployment, simplify model maintenance, and support rapid adaptation across diverse fab environments. As semiconductor manufacturing continues to push the limits of physics, AI-powered vision models are becoming essential tools for maintaining yield, quality, and competitiveness.