One-sentence Summary

Researchers from the University of Liverpool and Mohamed bin Zayed University of Artificial Intelligence propose Generative Montage, a Writer-Editor-Director framework that enables colluding LLM agents to manipulate victim beliefs using only truthful evidence fragments distributed through public channels, exploiting overthinking tendencies to achieve $74.4\%$74.4% attack success rates and cascading deception exceeding $60\%$60% across 14 model families.

Key Contributions

• A formal threat model for cognitive collusion in LLM agents defines how colluding agents steer a victim’s posterior belief toward a fabricated hypothesis using only truthful evidence fragments, local truth constraints, and global lie conditions rather than fabricated data, covert channels, or backdoors.
• Generative Montage, a Writer-Editor-Director framework, constructs deceptive narratives through adversarial debate and coordinated posting of evidence fragments, exploiting causal reasoning and overthinking tendencies to induce internally plausible but false conclusions.
• CoPHEME, a dataset derived from real-world rumor events, supports simulated attacks across 14 LLM families, showing attack success rates of 74.4% for proprietary models and 70.6% for open-weights models, greater susceptibility in reasoning-specialized models, and downstream deception rates above 60%.

Introduction

Large language models increasingly act as reasoning cores for autonomous agents that analyze fragmented, real-time information streams on platforms such as social media. The authors leverage this reliance on narrative coherence as an attack surface, showing that agents can overinterpret truthful but unrelated fragments and infer false causal relationships, enabling misleading “lying with truths” that bypasses conventional fact checking. Prior collusion work mainly focuses on hidden channels such as backdoors or steganographic communication, leaving a gap in understanding how public, factual evidence can be coordinated to manipulate agent beliefs. The authors formalize Cognitive Collusion Attacks, propose Generative Montage, a multi-agent framework using Writer, Editor, and Director agents to curate and sequence truthful fragments into deceptive narratives, and introduce CoPHEME with experiments showing that LLM agents are highly susceptible to this form of narrative manipulation.

Dataset

• The authors use CoPHEME, a dataset adapted from the PHEME dataset, to simulate narrative manipulation by separating factual evidence from fabricated conclusions.

• The dataset is built around a “lying with truths” structure, where agents can cite truthful tweets while forming misleading or false narratives.

• Evidence Pool: Tweets annotated as “true” or “non-rumors” are used as factual raw material. These tweets are filtered by a Local Truth constraint, LT=1, and serve as the evidence pool for colluding agents.

• Target Fabrications: “False” and “unverified” rumors are selected to form fabricated narratives. These items are chosen based on historical cascade size and further filtered by semantic deduplication to emphasize high-impact and non-redundant narrative campaigns.

• The authors use the dataset as a testbed rather than describing full training splits or mixture ratios. The data supports experiments that test how models or agents can use truthful evidence to support misleading conclusions.

• The description does not report subset counts, exact cascade size thresholds, or detailed metadata schema.

Method

The authors model the deception setting with atomic evidence fragments, belief spaces, and causal graphs. Each evidence fragment $e_i$ei​ is treated as factually correct and has a timestamp $t_i$ti​. An agent’s belief space $\mathcal{H}^{a_i}$Hai​ contains candidate explanations that connect these fragments into narratives, and it is split into true hypotheses $\mathcal{H}_r$Hr​ and fabricated hypotheses $\mathcal{H}_f$Hf​, with $\mathcal{H}_r \cap \mathcal{H}_f = \emptyset$Hr​∩Hf​=∅ and $\mathcal{H}_r \cup \mathcal{H}_f = \mathcal{H}^{a_i}$Hr​∪Hf​=Hai​. Each hypothesis induces a directed causal graph $G=(V,E)$G=(V,E), where nodes represent events and edges represent causal dependencies. The real world is represented by $G^{*}=(V,E_{\mathrm{real}})$G∗=(V,Ereal​), while a spurious reality is represented by $\hat{G}=(V,\hat{E})$G^=(V,E^), where $\hat{E}=E_{\mathrm{real}}\cup E_{\mathrm{false}}$E^=Ereal​∪Efalse​ and $E_{\mathrm{false}}\cap E_{\mathrm{real}}=\emptyset$Efalse​∩Ereal​=∅. A false narrative emerges when $E_{\mathrm{false}}\neq\emptyset$Efalse​=∅, meaning the agent adopts causal links that do not exist in the true graph even though the observed fragments may all be truthful.

Generative Montage is a multi-agent framework that operationalizes coordinated narrative production to induce a victim agent’s belief in a fabricated hypothesis. The pipeline separates explicit collusion, where attacker-controlled agents intentionally build and distribute the deceptive narrative, from implicit collusion, where otherwise benign agents internalize the narrative and later amplify it. Refer to the framework diagram.

The first stage is adversarial narrative production by the Writer, Editor, and Director. The Writer acts as a narrative synthesizer that generates a coherent draft $\mathcal{N}$N strictly from a factual evidence pool $\mathcal{E}_{pool}$Epool​. Rather than fabricating false fragments, the Writer uses contextual ambiguity and plausible generalization to make the fabricated hypothesis appear more likely than the true hypothesis. This is formalized as a constrained generation task:

$$\mathcal{N}^{*} = \operatorname*{argmax}_{\substack{\mathcal{N}\sim\mathcal{E}_{pool}\\ P(\mathcal{N}\mid G^{*})=1}} \frac{P(H_f\mid\mathcal{N})}{P(H_r\mid\mathcal{N})}.$$N∗=N∼Epool​P(N∣G∗)=1​argmax​P(Hr​∣N)P(Hf​∣N)​.

The constraint $P(\mathcal{N}\mid G^{*})=1$P(N∣G∗)=1 preserves factual integrity, while the objective increases the semantic posterior odds of the target lie relative to the true explanation.

The Editor then fragments the synthesized narrative into discrete semantic slices and reorders them into a time-ordered evidence stream $\vec{S}=\{(p_i,t_i)\}$S={(pi​,ti​)}. The purpose is not to change facts, but to arrange truthful fragments so that implicit semantic associations and temporal proximity encourage the victim to infer nonexistent causal edges. The Editor searches over valid logical permutations $\Pi(\mathcal{N})$Π(N) to maximize the cumulative probability of spurious causal relations induced by the sequence:

$$\vec{S}^{*} = \operatorname*{argmax}_{\vec{S}\in\Pi(\mathcal{N})} \sum_{(p_i,p_j)\in E_{\mathrm{false}}} P(p_i\rightarrow p_j\mid\vec{S}).$$S∗=S∈Π(N)argmax​(pi​,pj​)∈Efalse​∑​P(pi​→pj​∣S).

This captures the core montage effect: individual fragments remain locally truthful, but their juxtaposition compels the victim to construct a global causal story that is absent from any single evidence item.

The Director manages the adversarial refinement process by acting as a simulated victim. It evaluates intermediate outputs $\mathcal{O}\in\{\mathcal{N},\vec{S}\}$O∈{N,S} using a dual-loop optimization process. The Writer-Director loop refines the narrative draft, while the Editor-Director loop refines the fragment ordering. The Director assigns one of three states to an output through a gating function:

$$\delta(\mathcal{O}) = \begin{cases} \text{ACCEPT}, & \text{if } \hat{P}(H_f\mid\mathcal{O})>\tau \text{ and } \forall e\in\mathcal{O},\ \mathrm{LT}(e)=1,\\ \text{REJECT}, & \text{if } \exists e\in\mathcal{O},\ \mathrm{LT}(e)\neq 1,\\ \text{REVISE}, & \text{otherwise, generating critique } \mathcal{C}. \end{cases}$$δ(O)=⎩⎨⎧​ACCEPT,REJECT,REVISE,​if P^(Hf​∣O)>τ and ∀e∈O, LT(e)=1,if ∃e∈O, LT(e)=1,otherwise, generating critique C.​

Here, $\hat{P}(H_f\mid\mathcal{O})$P^(Hf​∣O) estimates how strongly the output induces the fabricated hypothesis, $\tau$τ is an acceptance threshold, and $\mathrm{LT}(e)=1$LT(e)=1 denotes that fragment $e$e satisfies factual correctness. This design simultaneously enforces deceptive persuasiveness and factual integrity.

Once the Director approves the sequence, the Publisher stage performs decentralized injection. The approved montage $\vec{S}$S is mapped onto a network of sybil publisher accounts $\mathcal{B}=\{b_1,\ldots,b_m\}$B={b1​,…,bm​}, where each fragment is assigned to a specific account. The injection process is defined as:

$$\mathcal{P}_{\mathrm{pub}} = \Phi(\vec{S},\mathcal{B}) = \left\{ (p_i,t_i,b_{\pi(i)}) \right\}_{i=1}^{|\vec{S}|}.$$Ppub​=Φ(S,B)={(pi​,ti​,bπ(i)​)}i=1∣S∣​.

The assignment strategy $\pi(i)$π(i) distributes fragments across multiple channels while preserving the designed temporal structure in the victim’s observable feed. This makes the injected evidence appear as a naturally flowing stream rather than a centrally controlled manipulation.

The victim stage then triggers implicit collusion through cognitive steering. The adversarial feed $\mathcal{P}_{\mathrm{pub}}$Ppub​ is mixed with normal information $\mathcal{F}_{\mathrm{normal}}$Fnormal​ to form the victim’s total information environment $\mathcal{F}=\mathcal{P}_{\mathrm{pub}}\cup\mathcal{F}_{\mathrm{normal}}$F=Ppub​∪Fnormal​. The method leverages the victim’s tendency to construct coherent explanations from fragmented inputs, shaping the posterior distribution so that the fabricated hypothesis becomes the most plausible interpretation:

$$H_f \approx \operatorname*{argmax}_{H} P_{\mathcal{M}}(H\mid\mathcal{F}).$$Hf​≈Hargmax​PM​(H∣F).

The victim does not passively receive a false statement. Instead, its own reasoning machinery infers a spurious causal structure from locally truthful fragments and treats the resulting conclusion as a self-derived belief.

Finally, the framework produces a cascade effect through belief transfer. After internalizing the fabricated conclusion $\hat{H}_{\mathrm{vic}}$H^vic​, the victim publishes it to public channels as a trusted output. Downstream agents or decision makers $\mathcal{A}_{\mathrm{down}}$Adown​ then consume the conclusions of multiple victim agents rather than the raw evidence stream. Because these outputs are endorsed by agents that appear to have independently analyzed the data, they create trust amplification:

$$P_{\mathcal{A}_{\mathrm{down}}}(H_f\mid\{\hat{H}_{\mathrm{vic}}^{(i)}\}_{i=1}^{K}) \gg P_{\mathcal{A}_{\mathrm{down}}}(H_f\mid\vec{S}).$$PAdown​​(Hf​∣{H^vic(i)​}i=1K​)≫PAdown​​(Hf​∣S).

As the number of compromised agents $K$K grows, the global information environment converges toward the fabricated hypothesis:

$$\lim_{t\to\infty}P(H_f\mid\mathcal{I}_{\mathrm{global}})\to 1.$$t→∞lim​P(Hf​∣Iglobal​)→1.

This describes how the system transforms an originally fabricated causal narrative into a collectively ratified consensus, even though each individual published artifact may remain factually grounded or appear as an independent analytical conclusion.

Experiment

The experiments evaluate cognitive collusion in a simulated social media ecosystem built from truth-labeled evidence and high-impact fabricated narratives, where colluding agents manipulate neutral LLM analysts and their outputs pass through downstream majority-vote and judge-based verification layers. Results show that adversarial montage sequences can steer a broad range of model families to adopt false narratives with high confidence, and that enhanced reasoning may amplify rather than resist this vulnerability. Ablation and sequence-length analyses indicate that specialized multi-agent coordination, iterative refinement, strategic evidence ordering, and a moderate amount of distributed posts are key to maximizing deception, while downstream fact-checking remains largely insufficient once analysts are misled.

Across the evaluated rumor events, the attacks show broad susceptibility among victim models, with high overall success rates and elevated confidence in the adopted false narratives. Open-weights models with enhanced reasoning are generally more vulnerable than their base or smaller counterparts, while proprietary models exhibit the opposite trend. This suggests that stronger causal inference can increase exposure to adversarial narrative manipulation rather than reduce it. Most tested models are highly vulnerable, forming false beliefs with high confidence and frequent high-confidence success. Reasoning-enhanced open-weights models are more susceptible than base or smaller models, whereas proprietary models show the reverse pattern. Events requiring rapid causal synthesis are more vulnerable than complex political narratives with variable rates. High-confidence outcomes indicate a failure mode in which agents internalize misleading narratives without detecting manipulation.

Adding Chain-of-Thought prompting raises victim susceptibility for both tested 7B open-weight models in the Charlie Hebdo scenario. The reasoning distilled model starts from a higher direct susceptibility and receives the larger increase when prompted to show its reasoning. The pattern suggests that explicit reasoning can amplify adversarial narrative effects instead of mitigating them. Chain-of-Thought prompting increases attack success against both Qwen2.5-7B-Instruct and DS-R1-Distill-Qwen-7B. DS-R1-Distill-Qwen-7B remains more susceptible than Qwen2.5-7B-Instruct under both direct and reasoning prompts. Enhanced reasoning appears to be an attack surface under adversarial cognitive manipulation.

Ablations on the Charlie Hebdo event show that the full multi-agent attack achieves the highest attack success rate, while removing components lowers effectiveness. The largest drop occurs when multi-agent coordination is replaced by a single model, indicating that specialization and collaborative optimization are central to the attack. Collapsing the multi-agent system into a single model causes the largest performance loss. Removing adversarial debate reduces attack success noticeably. Removing sequential editing causes a smaller but still clear decline. The hardened metrics show the same downward trend as attack success decreases.

The experiments evaluate adversarial narrative attacks against language models across rumor events and a targeted scenario, including direct prompting, chain-of-thought prompting, and component ablations. They show that models generally form false beliefs with high confidence, with reasoning-enhanced open-weight models being more vulnerable than base or smaller models, while proprietary models exhibit the opposite pattern. Explicit reasoning can amplify rather than reduce manipulation, and full multi-agent coordination, adversarial debate, and sequential editing are important for attack effectiveness.