One-sentence Summary

The authors propose a suite of minimal algorithmic tasks abstracting open-ended creative skills, showing that next-token prediction is myopic relative to multi-token approaches such as teacherless training and diffusion models, and that injecting noise at the input layer (seed-conditioning) matches or surpasses output-layer temperature sampling for eliciting diverse, coherent output.

Key Contributions

• A minimal algorithmic benchmark is introduced for open-ended creativity, abstracting tasks that discover new connections in abstract knowledge graphs or construct new patterns, such as wordplay, analogies, research, math problem design, and protein design. This benchmark enables controlled measurement of originality and diversity.

• Experiments on the benchmark show that next-token learning is myopic for creative planning, while multi-token approaches, including teacherless training and diffusion models, produce more diverse and original outputs and improve diversity-accuracy tradeoffs.

• Seed-conditioning injects noise at the input layer to elicit randomness and is reported to maintain coherence as well as, or in some conditions better than, output-layer temperature sampling while increasing diverse and creative generation.

Introduction

The authors leverage minimal, controlled algorithmic tasks inspired by cognitive science to study creativity in language models, especially open-ended generation where outputs must be coherent, diverse, and original rather than merely correct. This matters for applications such as scientific ideation, novel training data generation, and test-time scaling methods that rely on diverse exploration, but prior work often evaluates real-world creativity subjectively and struggles to verify originality when models memorize training data. The authors also argue that standard next-token prediction and output-level randomness through temperature sampling are poorly aligned with tasks that require latent planning and creative leaps of thought. Their main contribution is a suite of open-ended tasks isolating combinational and exploratory creativity, along with an objective “algorithmic creativity” metric, and they show that multi-token approaches, such as teacherless training or diffusion, outperform next-token prediction, while seed-conditioning can induce useful diversity.

Dataset

• The dataset is synthetic and graph-based, created for two algorithmic creativity tasks: Sibling Discovery and Triangle Discovery. It is not drawn from an external corpus; the authors construct graph structures and sample valid examples from them.

• Sibling Discovery uses a fixed bipartite graph with parent vertices and child vertices. Each example is a triplet (gamma, gamma', Gamma) where gamma and gamma' are siblings that share parent Gamma. The authors define coherence by requiring that both siblings are connected to the same parent, then sample from a uniform distribution over all valid sibling-parent triplets.

• In Sibling Discovery, the model receives independent and identically distributed examples of valid triplets. The graph itself is not provided in context, so the model must infer and store graph adjacency relationships in its weights. The triplets are formatted with the parent last, making the task harder for standard next-token prediction because the parent, or punchline, is generated after the siblings.

• Triangle Discovery uses a knowledge graph G = (V, E). Each example is a triplet (v1, v2, v3) that forms a triangle, meaning all three pairwise edges between the vertices must exist in the graph. This is a higher-order version of Sibling Discovery because the model must coordinate three remembered edges rather than one parent-child relationship.

• For Triangle Discovery, the authors treat different permutations of the same triangle as one unique example. This filtering or uniqueness rule prevents duplicate triangles from being counted separately. Exact string formatting details are deferred to the paper appendix.

• The data is used as a procedural task distribution rather than a real-world annotated dataset. The model observes valid examples during training and is evaluated by generating novel valid triplets from memorized graph structure. Sibling Discovery focuses on single-hop parent recall, while Triangle Discovery tests multi-edge planning.

• Processing details include coherence filtering, uniqueness filtering for triangle permutations, and deliberate prompt formatting that hides the graph from the model. The provided text does not specify exact dataset size, vocabulary, train-test split ratios, or mixture weights across the two tasks.

Method

Method and training design

The authors construct an open ended algorithmic framework for evaluating creative generalization by forcing models to discover latent relational structures that are not directly visible in the training context. The central idea is to encode the creative process as a hidden combinatorial rule. In combinational creativity, the model must recall entities and arrange them under relational constraints, thereby discovering a graph-like structure stored in weights rather than exposed in context. In exploratory creativity, the model must generate new valid structures within a known rule space, such as graphs whose edges can be rearranged into a coherent circuit or line.

A representative task is circle construction. The model outputs an adjacency list

and the string is considered coherent only if there exists a resolving permutation $\pi$π such that

for distinct indices $j_1,\ldots,j_n$j1​,…,jn​. The permutation $\pi$π represents the latent creative structure that turns a seemingly disordered edge list into a valid cyclic path. To measure novelty correctly, examples generated by the same resolving permutation but using different vertex labels are treated as duplicates. This prevents superficial surface variation, such as changing characters or symbols while preserving the same plot dynamics, from being counted as creative progress. A simpler line construction variant uses the same logic, except the resolved permutation forms a path that terminates in a dead end rather than returning to the first vertex.

The training pipeline is designed so that the model observes only token sequences and must infer the underlying hidden structure. These tasks are permutation invariant, meaning no token order is privileged over another, and the latent creative rule must be recovered from distributed evidence across the entire sequence. This makes the learning problem different from algorithmic tasks where the reasoning path is nearly visible at the token level.

For models trained with next token prediction, the authors use a standard supervised teacher forcing objective. Given a prompt $\boldsymbol{p}$p and target sequence $\boldsymbol{s}$s, the model predicts each token $s_i$si​ using the prompt and previous target tokens:

The intended reasoning process, however, is more naturally described by a latent plan $z$z and a plan conditioned distribution over coherent outputs:

In such a formulation, a model can first select a latent structure and then generate the surface sequence conditionally on that structure. For example, if a parent node must be generated after two sibling nodes, the efficient generative rule is to plan the parent first and then emit the children, requiring roughly one training example per valid parent child edge. A next token learner, by contrast, may exploit the prefix as input and infer the parent from the visible sibling pair. This shortcut, sometimes described as a Clever Hans style solution, can dominate early learning and reduce useful gradient signal for the latent plan. Afterward, the model may learn the second sibling through a more complex pairwise next token conditional, which can require far more training examples than the latent plan approach.

To study whether this difficulty is specific to autoregressive factorization, the authors also train multi token transformer variants using teacherless training. In this setting, the model predicts all target tokens simultaneously from the prompt and masked placeholder positions. The exact masking schedule is not central to the method, but the objective encourages the model to produce a complete structured completion without relying on a revealed ground truth prefix. In practice, the training procedure combines this simultaneous prediction objective with the standard next token prediction objective, allowing the model to learn both local sequential regularities and more holistic sequence completion behavior.

A discrete diffusion model is used as an additional non autoregressive comparison. The model is trained on masked or corrupted versions of the target sequences and learns to recover the original coherent output from varying degrees of damage. This produces a training regime that interpolates between fully masked reconstruction and partially corrupted completion. During inference, the model begins from a fully masked or corrupted token sequence and iteratively revises subsets of tokens until a complete coherent output emerges. This iterative correction process contrasts with standard transformer generation, which proceeds token by token.

For transformer models, inference is performed by extracting independent samples using autoregressive decoding. This applies both to models trained with next token prediction and to multi token transformer models trained with the combined objective. Generation can be done with greedy decoding or nucleus sampling, depending on whether the goal is maximally coherent output or greater sample diversity.

To control diversity more explicitly, the authors introduce seed conditioning. Each training example is paired with an arbitrary random prefix, called a seed, which carries no direct semantic meaning. The model learns to generate outputs conditioned on this seed. At test time, fresh random seeds are used to elicit different completions. This design gives the model a separate control variable that may encourage one latent plan per seed, rather than asking a high temperature softmax to simultaneously represent many competing creative choices. Conceptually, seed conditioning can also be viewed as a controlled way to simulate prompt phrasing variation, where different seeds induce different internal trajectories while keeping the underlying task fixed.

Experiment

The experiments evaluate algorithmic creativity and summarization using multi-token, next-token, teacherless, diffusion, and seed-conditioned Transformer setups, including larger Gemma models and smaller Transformer and diffusion baselines. Multi-token prediction significantly improves creativity, especially for larger models, and reduces memorization-related failure, while teacherless objectives show smaller direct creativity gains for small Transformers but improve diversity and summarization quality. Seed-conditioning is another important diversity mechanism for Transformers, increasing creative outputs without relying on temperature and complementing, rather than replacing, training-objective effects.