One-sentence Summary

Researchers from UCLA, USC, and UC Davis introduce a computational framework that analyzes narratives through story arcs, turning points, and affective dimensions, revealing that LLM-generated stories lack the diversity and tension of human narratives, and demonstrating that explicitly integrating these discourse features yields over $40\%$40% improvement in neural storytelling.

Key Contributions

• The paper introduces a computational framework for analyzing narratives through story arcs, turning points, and affective dimensions, using expert and automatic annotations to measure narrative structure, emotional arousal, and valence.

• The paper presents a quantitative comparison of human-written and LLM-generated stories, along with narrative reasoning measurements, finding that human stories are more suspenseful, arousing, and structurally diverse, while many LLM stories are homogeneously positive and weak in turning points such as setbacks and climaxes.

• The paper shows that explicit incorporation of story arcs, turning points, and affective features improves neural storytelling, with reported improvements above 40 percent in diversity, suspense, and arousal.

Introduction

As large language models are increasingly used for storytelling, understanding their narrative structure matters because stories shape audience engagement, emotional response, and social meaning. Existing evaluations often focus on fluency, coherence, or broad judgments of creativity, while prior work notes global gaps such as plot holes, repetitive themes, and weak pacing, but lacks a quantitative framework for discourse-level analysis. The authors address this by analyzing narratives at three levels: story arcs, turning points, and affective signals such as arousal and valence. They build a movie synopsis dataset with human and automated annotations, use it to compare LLM and human storytelling and comprehension, and introduce discourse-aware prompting that improves diversity, suspense, and engagement.

Dataset

• The authors build a narrative dataset from film plots, using recent English-language films listed on Wikipedia as the primary source. The raw records include film titles, genres, release dates, and plot synopses.

• To reduce popularity bias and potential data leakage, the authors filter out well-known movies by using Wikipedia page length as an approximate popularity signal. This filtering step leaves 819 human-written film synopses.

• To further reduce contamination risk, the authors rephrase each synopsis title and initial setting by changing unique identifiers, including proper nouns. They then use GPT-4 to generate a paired machine-written synopsis for each film, using the rephrased title, initial setting, and genre as input. The final corpus contains 1638 synopses in total, comprising human-written and GPT-4-written paired narratives.

• The human and machine narratives are roughly comparable in length. Across the narratives, the average plot contains 705.6 words and 37.8 sentences, making the data longer and more complex than many other quantitative narrative datasets.

• For annotation, the authors select 440 narratives for in-depth human labeling. Each selected narrative is annotated by three annotators with English or story-analysis backgrounds. The annotation tasks assign a story arc to each synopsis and locate five sentential turning points.

• The turning-point schema adapts prior story structure annotations by allowing flexible positioning for turning points TP3 and TP4, and optional missing or multiple positions for TP2, TP3, and TP4 when annotators are uncertain. Reported inter-annotator agreement is 0.90 using Spearman correlation for one task and 0.62 using Cohen kappa for the other.

• The authors use the data to analyze discourse structure in film narratives, comparing human-written and GPT-4-written story plots. They also apply an agentic arousal and valence analysis: GPT-4 identifies the main character, infers three emotion adjectives for each sentence, and the authors map those adjectives to arousal and valence scores using the NRC VAD lexicon. Per-sentence emotion scores are averaged, interpolated across narrative length, and aggregated into mean curves for comparison.

Method

The method is built as a controlled narrative analysis and generation pipeline. The authors first construct a paired human and machine narrative corpus. They collect recent English-language film synopses, reduce the likelihood of data leakage by filtering popular films with Wikipedia page length as a proxy, and anonymize the remaining narratives by altering proper nouns and unique identifiers. The revised titles, initial settings, and genres are then given to a large language model to generate machine-written synopses paired with the human-written ones. This produces a comparison set in which human and machine narratives are similar in length and grounded in the same initial story setup.

The annotation module converts raw narratives into structured discourse annotations. Annotators label each story with a story arc and identify the sentence positions of five turning points. The annotation schema adapts the turning point definition to longer and more complex stories by allowing flexible positions for some turning points and, in uncertain cases, missing or multiple positions. Annotators are trained and evaluated through qualification tasks, and each narrative is reviewed by multiple annotators. The resulting annotations support later measurements of narrative structure, pacing, and arc distribution.

For affective measurement, the authors use an agentic procedure based on large language model inference. Given a narrative, the model first identifies the main character. For each sentence $s_i$si​, the model infers three emotion descriptors $W_i = \{w_{i1}, w_{i2}, w_{i3}\}$Wi​={wi1​,wi2​,wi3​} that characterize the protagonist’s emotional state at that point in the plot. These descriptors are mapped to arousal and valence scores using a valence lexicon normalized to $[0,1]$[0,1]. For each sentence, the average arousal and valence values are computed, yielding $A(s_i)$A(si​) and $V(s_i)$V(si​). For a narrative with $N$N sentences, these values are plotted against the relative sentence position $\frac{i}{N}$Ni​, interpolated to form smooth curves, and averaged across narratives to produce aggregate arousal or valence trajectories.

The generation and improvement module uses discourse features to guide story writing. Instead of producing a story in a single step, the authors adopt a planning-first approach in which the model first generates an outline and then expands it into a full narrative. Several variants are designed with increasing degrees of explicit discourse structure. The basic variant asks the model to produce an outline and then expand it. A stronger variant instructs the model to include key turning points in the outline, such as the point of no return, the major setback, and the climax, together with their definitions. Another variant replaces some machine-generated turning points with human-written equivalents. These variants test whether reasoning about narrative turning points improves suspense, emotional engagement, plot coherence, and overall preference.

The pipeline is further extended with an arc-enhanced generation method. The observed bias in machine-generated narratives motivates explicit control over story arc type. The human-written and machine-written stories show markedly different distributions of arc categories, with machine-generated outputs concentrating more heavily in certain arc types and displaying fewer varied inflections. Refer to the figure below.

Based on this motivation, the arc-enhanced variant instructs the model to write a story with a specified arc, a specified number of major rises and falls, and explicit descriptions of the protagonist’s initial and ending states. The method is evaluated along two directions. First, the authors measure how reliably the model follows the requested arc structure. Second, they compare narrative diversity between arc-enhanced stories and baseline outline-only stories across themes, conflict types, and characters. The full workflow therefore connects corpus construction, discourse annotation, affective trajectory measurement, and structure-aware prompt generation, with the goal of improving machine storytelling by making pacing, turning points, and arc selection more explicit.

Experiment

The experiments compare human and LLM narratives, benchmark narrative comprehension, and test discourse-aware story generation. They find that LLMs mishandle pacing and suspense, overuse positive arcs, and still comprehend narrative structure less well than humans, although adding turning point or arc information can improve understanding. Planning with explicit turning points and story arcs yields more engaging, logical, and diverse stories, while annotators prefer concrete twists and well-motivated plots over overly straightforward positive outcomes.

Turning points are ordered narrative events that shape plot progression from setup through commitment, setback, and resolution. The early types mainly support exposition and rising action, while setback and climax are especially important for determining the story arc. Adding turning points improved generated narratives over outline-only baselines, with self-generated turning points giving the most balanced flow. Early turning points introduce goals, stakes, and commitment. Setback and climax most strongly shape the story arc. Turning point guidance improved narratives over outline-only generation. Self-generated turning points were more balanced than forced human turning points.

Humans identify narrative turning points more accurately than all tested language models across the five categories. Gemini and Claude are the strongest machine systems, with GPT-4 following behind, while GPT-3.5 and Llama3 trail. The results show that current language models still struggle to match human narrative comprehension when identifying turning points without story arc context. Human annotators achieve the highest overall accuracy on turning point identification. Gemini and Claude lead the machine models, with average accuracy above 30%. GPT-4 performs moderately, while GPT-3.5 and Llama3 show weaker results. All evaluated language models remain below human performance.

Human evaluations compare plain outline-based stories with versions that explicitly use turning points in planning. Turning-point-aware approaches are judged better on suspense, emotion provocation, and overall preference than the outline-only baseline. Self-generated turning points are especially favorable because they improve storytelling while preserving narrative flow. Turning-point-aware generations receive more best ratings and fewer worst ratings than outline-only stories. Self-generated turning points produce the most balanced results, with fewer low ratings than human-provided turning points. Human-provided turning points can strengthen emotional engagement, but they may disrupt logical flow when enforced too strongly.

GPT-4 showed limited success when asked to generate stories following specified narrative arcs, with overall accuracy below half in this evaluation. Arcs with one inflection point were handled more reliably than arcs with more complex or multiple plot changes. The model struggled most with negative plot progressions, humble starts, and complex narrative dynamics. GPT-4 had poor accuracy in following requested story arc types. Arcs with one inflection point were more likely to succeed than arcs with two or more plot turns. Negative, humble-start, and complex arcs were harder for the model to generate reliably.

Arc-Enhanced stories show higher win rates than Outline-Only stories across most diversity dimensions, especially theme, conflict, and character. The overall comparison also favors Arc-Enhanced, while setting is the least differentiated dimension. The pattern suggests that explicit story-arc guidance improves several aspects of narrative diversity beyond outline-only generation. Arc-Enhanced stories outperform Outline-Only stories on theme, conflict, and character diversity. Overall comparison favors Arc-Enhanced stories in the diversity evaluation. Setting diversity shows the smallest separation between the two approaches.

The experiments evaluate whether explicit narrative turning points and story arcs improve large language model storytelling, covering turning point identification, human narrative preference, and diversity outcomes. Across evaluations, explicit turning point and arc guidance generally improves narrative quality, suspense, emotional engagement, overall preference, and diversity over outline only baselines, with self-generated turning points producing the most balanced and coherent stories. However, all tested language models remain weaker than humans at identifying turning points, and GPT-4 struggles with complex, negative, or multiple inflection narrative arcs.