StoryScope Maps AI Fiction’s Narrative Idiosyncrasies

Researchers have developed a method to identify AI-generated fiction by analyzing story structure rather than word choice, suggesting that machine-written narratives follow measurably different plotting patterns. The approach, introduced in a paper titled StoryScope: Investigating Idiosyncrasies in AI Fiction and linked to work by researchers including Russell, Rajendhran, Pham, Iyyer and Wieting, compares human and AI short stories using a dataset of 10,272 prompts and 61,632 stories, pairing each human-written story with AI-generated versions from Claude, DeepSeek, Gemini, GPT and Kimi. The stories average about 4,753 words, allowing analysis of long-form narrative structure. Rather than detecting common vocabulary, punctuation or sentence predictability, the team extracted 304 narrative features describing plot causality, subplot presence, timeline jumps, character agency, theme explanation and emotional description. A classifier trained only on these narrative features reached a 93.2% macro-F1 score distinguishing human from AI stories, and 68.4% macro-F1 identifying the source among human authors and five AI systems. The findings support the intuition that AI fiction often feels overly tidy, explanatory or uniformly structured, even when surface style is polished. The study found three recurring patterns. AI stories explicitly stated their themes in 77% of cases, versus 52% for human stories. They were more likely to avoid subplots, with 79% having none compared with 57% of human stories, and more often resolved conflicts through protagonist choices, 69% versus 46%. AI narratives also leaned heavily on bodily sensations and physical metaphors to convey emotion, appearing in 81% of cases compared with 38% of human-written examples. Human fiction more often used nonlinear timelines, unresolved tensions and ambiguous motivations. The researchers tested whether these signals survive editing by applying the LAMP rewriting method to 278 Gemini-generated stories. Although the revisions changed stylistic tells, classification performance remained high, falling only from 95.5% to 93.9% macro-F1. This suggests structural patterns can persist after superficial cleanup, though the experiment was limited to one model and one editing approach. The authors caution that their results apply primarily to long-form fiction and a controlled prompt-based dataset, not to emails, essays, social media or all real-world AI-assisted writing. Features were assigned with language models, and high classification scores do not prove individual authorship. The broader implication is that generative AI evaluation may need to examine how models organize narratives, not just whether their sentences sound plausible. Future improvements may depend less on avoiding predictable words and more on learning when not to follow obvious story paths.