One-sentence Summary

Ball and Hackemann argue that modern AI alignment methods, originally designed for safety, are dual-use technologies that inadvertently build a censor’s toolkit for informational dominance, a risk amplified by rapid AI adoption, economic power asymmetries, and rising authoritarianism, and they urge the community to mitigate this dual-use potential.

Key Contributions

• The paper maps alignment techniques such as pretraining data filtering and inference-time controls to documented cases of misuse for censorship and manipulation, demonstrating how these safeguards can become instruments of informational dominance.
• It identifies dual-use risks exacerbated by the convergence of AI as a primary information source, market concentration, and global democratic backsliding, which transforms alignment tools into instruments that may serve both protection and oppression.
• The work proposes mitigation strategies including standardized censorship and manipulation benchmarks, transparency and auditing mechanisms, and preserving competitive pluralism to prevent informational monopolies, urging the community to move beyond perfunctory ethics statements.

Introduction

AI alignment techniques, designed to steer models toward desired behaviors such as safety and harmlessness, are purpose-agnostic tools: whoever defines the target values determines whether the system serves protection or informational control. Recent research on AI risks has largely remained hypothetical and focused on general system-level harms, lacking a systematic treatment of how alignment methods themselves can be repurposed for large-scale censorship and manipulation. The authors bridge this gap by mapping alignment stages to their dual-use potential, documenting real-world instances where state and corporate actors already exploit these mechanisms, and analyzing the convergence of growing AI reliance, concentrated model provision, and global democratic decline that amplifies the threat. They propose a risk-mitigation framework centered on verifiable alignment through standardized benchmarks, competitive model pluralism, and genuine researcher reflection on dual-use consequences.

Method

The control stack for modern frontier language models is typically organized into three sequential stages: pre-training data curation, post-training alignment, and inference-time interventions. Each stage applies a distinct set of techniques to shape model behavior, and each carries a different risk profile for deliberate misuse. The overall pipeline can be understood as a layered filtering-and-steering mechanism that progressively molds both the knowledge base and the surface behavior of the model.

Pre-Training Data Filtering Before any model training begins, the raw corpus is subjected to an extensive cleaning process. The goal is to raise data quality and purge unwanted material. The paper describes two broad classes of filtering. Heuristic methods rely on simple rules such as deduplication, keyword-based “dirty word” counting, and domain block lists to remove adult content, personally identifiable information, or boilerplate. Model-based filtering employs trained classifiers to recognize more abstract patterns of unsafe or low-quality text. These classifiers are themselves language models fine-tuned to detect content that violates predefined quality or safety standards. The combination creates a dataset that reflects the judgments embedded in both the heuristic rules and the classifier training data. From a dual-use standpoint, the same machinery can be repurposed to excise entire topics, viewpoints, or historical facts. Heuristic filtering makes it straightforward to delete specific terms or domains, while model-based filtering can suppress subtler concepts. However, the barrier to entry is high: altering pre-training data and retraining from scratch demands full access to the training infrastructure, massive compute, and deep technical expertise. The impact is, however, fundamental because knowledge missing from the pre-training corpus cannot be generated by the model later unless explicitly injected through post-training or in-context information.

Post-Training Preference Alignment After pre-training, the base model typically undergoes alignment to make it helpful, honest, and harmless. The authors detail two major families of techniques. The first is Reinforcement Learning from Human Feedback (RLHF). A pool of annotators provides preference comparisons between model responses, a reward model is trained to predict those preferences, and then the base policy is optimized with algorithms such as Proximal Policy Optimization (PPO) or Group Relative Policy Optimization (GRPO) to maximize the learned reward. The second family is guideline-based alignment, which removes the need for ongoing human preference collection. Constitutional AI, for instance, uses a written set of principles to automatically generate critiques and revisions; the resulting preference data is used in the same RL step. Deliberative Alignment, a more recent variant, teaches the model to reason over explicit safety policies before responding, without requiring human-written chain-of-thought demonstrations. These post-training methods confer a significant amount of control to the entity that designs the preference data, selects the annotator pool, or writes the guidelines. An actor can curate a preference dataset that systematically favors an ideology, or brief annotators to punish disfavored viewpoints. Guideline-based approaches further lower the technical overhead, because modifying a policy document is faster than recollecting human feedback and retraining a reward model. The required resources are moderate: fine-tuning a model is far cheaper than pre-training but still demands access to model weights and sufficient compute. While post-training changes are more superficial than pre-training interventions and can be partially undone by adversarial attacks, they are effective at enforcing specific refusals and steering expressed opinions.

Inference-Time Control Even after a model is fully trained and aligned, providers deploy runtime mechanisms that can substantially alter its behavior. The most lightweight component is the system prompt, a hidden instruction prepended to every user interaction that defines the model’s role, priorities, and content boundaries. In addition, safety classifiers may be placed before or after generation to detect and block disallowed outputs. These classifiers can range from keyword filters to trained neural models. Inference-time interventions do not modify model parameters, so they offer the lowest barrier to implementation: they require negligible compute, can be changed instantly without retraining, and need no specialized expertise to alter a system prompt. This makes them the most accessible lever for rapid, surreptitious shifts in alignment. However, the control is superficial. System prompts shape responses through context, and classifiers act as a post-hoc gate; neither erases the underlying knowledge or tendencies encoded in the model. Nevertheless, high-precision classifiers can effectively censor outputs in real time, making it possible to intercept and rewrite answers that depart from a desired narrative.

Experiment

Evaluation reveals that inference-time controls such as system prompts and classifiers offer a low-cost, flexible alignment layer that can be easily misused, as demonstrated by Grok’s prompt-induced antisemitic responses and Yi-large’s real-time output filtering to suppress criticism. At the same time, the lack of transparency around proprietary alignment practices prevents independent scrutiny, creating an urgent need for standardized benchmarks that can detect political bias and information suppression across diverse contexts.

Alignment techniques vary widely in how accessible they are and how deeply they shape model behavior. Pre-training filtering is resource-intensive and hard to modify but yields fundamental changes, whereas inference-time controls are easy to implement with minimal expertise and produce only superficial modifications. These disparities create dual-use risks, as lightweight, accessible interventions can be readily exploited for censorship without deep technical effort. Inference-time control requires only runtime access and low-to-moderate expertise, making it the easiest technique to modify, yet its changes remain superficial. Pre-training filtering demands very high computational resources and high expertise, making modification difficult, but it enables fundamental shifts in model behavior. Post-training alignment occupies a middle ground: it needs model weights, moderate-to-high resources and expertise, and offers moderate ease of modification with persistent effects. Real-time output filtering, a common inference-time control, has been observed in deployed models suddenly suppressing disfavored content, illustrating the concrete dual-use potential of these easy-to-modify methods.

Alignment techniques show a clear spectrum in resource requirements and effect depth: pre-training filtering is computationally heavy and hard to alter but yields fundamental model changes, whereas inference-time controls need minimal expertise, are easily modified, and produce only superficial adjustments. Post-training alignment sits in between, requiring model weights and moderate resources with persistent effects. This disparity introduces dual-use risks, as lightweight inference-time methods can be readily exploited for censorship; real-time output filtering has been observed in deployed models suddenly suppressing disfavored content, illustrating this concrete danger.