Large language models (LLMs) can make errors not because they lack knowledge, but because they rely too heavily on grammatical patterns learned during training, according to a new study from MIT. This overreliance on syntax—rather than true understanding—can lead to incorrect or misleading answers, creating reliability and safety risks in real-world applications. The research, led by Marzyeh Ghassemi, an associate professor in MIT’s Department of Electrical Engineering and Computer Science, and her team, reveals that LLMs often learn to associate specific sentence structures with certain topics. For example, a model may learn that questions like "Where is Paris located?" follow a particular grammatical pattern and that the answer is typically a country. As a result, when presented with a nonsensical but structurally similar question—such as "Quickly sit Paris clouded?"—the model may still respond with "France," not because it understands the question, but because it recognizes the familiar syntax. This phenomenon, the researchers call "syntactic shortcut learning." It occurs because LLMs are trained on vast amounts of internet text, where certain grammatical structures are consistently linked to specific domains. Over time, models internalize these patterns and may use them as a shortcut to generate answers, even when the meaning of the question is meaningless or misleading. To test this, the team designed experiments using synthetic data where only one syntactic template was present per domain. They replaced words with synonyms, antonyms, or random terms while preserving the sentence structure. Despite the loss of semantic meaning, many LLMs—including GPT-4 and Llama—still produced correct answers. However, when the syntax was changed—even if the meaning stayed the same—the models frequently failed, showing that their performance depended more on grammar than on understanding. The findings also highlight a serious security concern. The researchers demonstrated that by crafting questions using syntactic templates associated with safe training data, they could trick models into generating harmful content, even when those models were specifically trained to refuse such requests. This suggests that current safety mechanisms may be vulnerable to manipulation through linguistic patterns. To address this, the team developed an automated benchmarking procedure to measure how much a model relies on these incorrect syntax-domain associations. This tool could help developers identify and fix such vulnerabilities before deploying models in critical applications like healthcare, finance, or customer service. The researchers emphasize that while this issue stems from how models are trained, it has real-world consequences. As LLMs are increasingly used in safety-critical contexts, understanding and mitigating these hidden failure modes is essential. Future work will focus on improving training data diversity and exploring how to build more robust, linguistically aware models, especially for complex reasoning tasks. Experts outside the study, like Jessy Li of the University of Texas at Austin, have praised the work for bringing much-needed attention to the role of linguistic structure in model safety—an area that has been overlooked but is crucial for building trustworthy AI.