Artificial intelligence tools designed for general use can now be exploited to embed stealthy, dangerous security flaws into computer chip designs, according to a new study from the NYU Tandon School of Engineering. The research, published in IEEE Security & Privacy, reveals that widely available large language models like ChatGPT can assist both skilled and unskilled individuals in creating hardware Trojans—malicious modifications hidden in chip designs that can leak data, disable systems, or grant attackers unauthorized access. To test this risk, researchers ran a two-year competition called the AI Hardware Attack Challenge, part of CSAW, an annual student cybersecurity event hosted by NYU’s Center for Cybersecurity. Participants were tasked with using generative AI to insert exploitable vulnerabilities into open-source hardware designs, such as RISC-V processors and cryptographic accelerators, and then demonstrate successful attacks. Jason Blocklove, a Ph.D. candidate in NYU Tandon’s Electrical and Computer Engineering Department and lead author of the study, said AI significantly lowers the barrier to creating such attacks. “AI tools definitely simplify the process of adding these vulnerabilities,” he noted. Some teams built fully automated systems that analyzed hardware code, identified weak points, and inserted custom malicious logic with minimal human input. Others used AI to better understand complex designs and craft targeted attacks. The most effective attacks included backdoors for unauthorized memory access, mechanisms to extract encryption keys, and logic that caused system crashes under specific conditions. Alarmingly, two teams with little to no prior experience in chip design—both undergraduate students—produced vulnerabilities rated as medium to high severity. Despite built-in safeguards in most large language models, researchers found these protections easy to bypass. One winning team used prompts that framed malicious requests as academic exercises, tricking the AI into generating exploitable code. Others discovered that requesting responses in less common languages could evade content filters altogether. The danger lies in the permanence of hardware flaws. Unlike software bugs, which can be patched with updates, errors in manufactured chips cannot be fixed without physically replacing the hardware. “Once a chip has been manufactured, there is no way to fix anything in it without replacing the components themselves,” Blocklove said. “That’s why researchers focus on hardware security. We’re identifying risks that may not exist yet but could become real threats.” The study follows earlier work by the same team showing that AI can also help design functional chips—demonstrated in their “Chip Chat” project. This duality highlights a growing concern: the same tools that could democratize chip development may also empower malicious actors to launch sophisticated attacks. The researchers stress that commercially available AI models represent just the beginning of potential threats. More advanced, open-source models not yet widely used could pose even greater risks. They call for stronger safeguards in AI systems and better verification tools to detect and prevent hardware-level attacks before chips are produced.