One-sentence Summary

Researchers from Sapienza University of Rome and Babelscape propose $\textit{xCoRe}$xCoRe, a unified three-step pipeline (mention detection, in-context clustering, cross-context merging) that jointly handles $\text{short-}$short-, $\text{long-}$long-, and $\text{cross-document}$cross-document coreference resolution, achieving state-of-the-art results on cross-document benchmarks and strong performance on long documents while retaining top-tier results on traditional datasets.

Key Contributions

• The paper introduces cross-context coreference resolution as a unified framework covering medium-size, long-document, and cross-document settings, and presents xCoRe, a three-step pipeline that extracts mentions, builds clusters within individual contexts, and merges clusters across contexts in a single pass.
• The architecture is modular and end-to-end, eliminating handcrafted features, separately trained components, and threshold tuning that limited prior cross-document systems; this design enables joint training on shared long- and cross-document resources, increasing data availability and benefiting the more challenging cross-document task.
• The model achieves new state-of-the-art results on cross-document benchmarks such as ECB+ and SciCo, along with strong performance on long-document data and top-tier results on traditional medium-size datasets; the models and code are publicly released.

Introduction

Coreference resolution (CR) identifies and groups mentions that refer to the same entity. While modern neural models reach near-human performance on standard benchmarks like OntoNotes and PreCo, CR remains challenging in two settings: very long documents, where models must maintain coherence across extended inputs, and cross-document coreference, which requires resolving entities across multiple texts.

The technical stakes are practical. Most existing CR systems rely on Transformer architectures with quadratic self-attention complexity, so they struggle with long inputs. Recent work segments documents into windows, trading performance for efficiency. Cross-document methods that encode texts separately cannot exceed 35 CoNLL-F1 on ECB+, and many require gold mentions or separate training for mention extraction and clustering, making end-to-end use impractical.

The authors identify a shared limitation: models fail to resolve coreference across disjoint contexts. They formalize this as cross-context coreference resolution and introduce xCoRe, an end-to-end architecture that operates in three stages: within-context mention extraction, within-context mention clustering, and cross-context cluster merging. This design leverages the observation that models already perform well within single documents and learns to merge local clusters across contexts in a single pass. The formulation enables training on shared long- and cross-document resources, improving data availability. xCoRe achieves new state-of-the-art scores on cross-document benchmarks and top-tier results on long-document datasets.

Dataset

The authors construct their experimental setup around two primary evaluation scenarios, each with distinct datasets and processing rules.

Cross-Document Setting

• They train on the well-established ECB+ and SciCo training sets and evaluate on the corresponding official test sets.
• To ensure comparability with prior work, they use gold topic information and exclude singleton mentions, as these have been shown to skew benchmark results.
• For ECB+, they focus strictly on entity coreference resolution and do not use additional document context, a configuration referred to as the Cybulska setting. This differs from some previous studies that incorporated extra surrounding context from the original source documents.
• In all cross-document experiments, they follow the convention of inputting only documents that belong to a single topic, leveraging the provided gold topic structure.
• See Table 1 for full dataset statistics.

Long-Document Setting

• For long-document coreference, they train comparison systems on LitBank and the silver-quality training set of BookCoref.
• Models trained on LitBank are tested on Animal Farm and the LitBank test set.
• Models trained on BookCoref are evaluated on its manually annotated test set.
• When testing on long documents (Animal Farm and BookCoref), they use a within-window size of w = 4000 tokens to match previous experimental setups.

Medium-Size Benchmarks

• They also include results on medium-size benchmarks such as OntoNotes and PreCo for broader comparison.

Method

The authors introduce xCoRe, a unified coreference system designed to seamlessly handle short, long, and multi-document inputs through a novel cross-context formulation. The model processes a set of generic contexts, which can represent individual documents or fixed-size windows of a long document. The architecture follows a three-step discriminative pipeline that first constructs coreference clusters within local contexts and then merges them across contexts in a single forward pass. Refer to the framework diagram for the overall architecture.

The pipeline begins with a within-context coreference resolution step for each input context. This phase is divided into mention extraction and mention clustering. For mention extraction, the model adopts a start-to-end strategy. A Transformer-based encoder computes hidden representations for the tokens within a context. The system identifies potential mention starts and subsequently predicts their corresponding ends, constrained by the nearest end-of-sentence token. This process yields a set of candidate mentions for each context.

Following extraction, the model performs mention clustering. Each mention is represented by the concatenation of its start and end token hidden states. The system employs a set of linear classification layers to detect whether a candidate mention is coreferent with an antecedent mention. This results in a set of local coreference clusters for each individual context.

The core innovation of the system lies in the cross-context cluster merging strategy. After all local clusters have been extracted across all contexts, the model computes a representation for each cluster using a single-layer Transformer. It then calculates pairwise coreference probabilities between clusters from different contexts. The probability is computed using a linear classification layer defined by the following equations:

$$\mathcal{L}(x) = W \cdot (ReLU(W' \cdot x))$$L(x)=W⋅(ReLU(W′⋅x)) $$p_{cm}(\mathcal{W}_a^{c_i}, \mathcal{W}_b^{c_j}) = \mathcal{L}(hs(\mathcal{W}_a^{c_j}) \| hs(\mathcal{W}_b^{c_i}))$$pcm​(Waci​​,Wbcj​​)=L(hs(Wacj​​)∥hs(Wbci​​))

where $W$W and $W'$W′ are learnable parameters, and $hs$hs denotes the hidden state representation of a cluster. The model calculates this probability for pairs of clusters from distinct contexts and merges those with a probability greater than $0.5$0.5. This technique is order-invariant, making it applicable to both sequential long documents and unordered cross-document settings.

During inference, the system addresses the quadratic memory complexity of encoding long sequences by splitting documents into fixed-size windows. Training presents a unique challenge, as learning cross-context merging requires exposure to multi-context examples, while learning within-context resolution benefits from long single-context examples. To resolve this, the authors employ a dynamic batching training strategy. For single-document datasets, the model samples $n$n contiguous contexts from a training document, adjusting the length of each context to fit memory constraints. For cross-document datasets, the batch is constructed by collecting $n$n distinct documents. This approach allows the model to learn from varied input configurations without exceeding memory limits.

Experiment

The evaluation compares xCoRe against cross-document systems such as PMCoref and CDLM, long-document systems like Longdoc, Dual-cache, and Maverick, and also probes cluster merging, pretraining, and step-wise oracle variants. xCoRe's cluster merging strategy reliably outperforms concatenation and mention-to-antecedent baselines, with larger gains as contexts increase. On cross-document benchmarks, xCoRe surpasses the previous state of the art and benefits notably from pretraining on LitBank, while on long-document benchmarks it matches or exceeds Maverick on several datasets but trails Longdoc on BookCoref due to splitting long chains. Error analysis shows mention extraction is the main bottleneck in cross-document settings, whereas cluster merging is the limiting factor in cross-context scenarios.

The datasets used in the experiments vary widely in size and annotation richness across cross-document, long-document, and medium-size coreference settings. Cross-document datasets are organized by topics and are relatively smaller, while long-document and medium-size benchmarks contain much larger token counts and training splits. The presence of singleton mentions also differs substantially across datasets, influencing experimental choices such as using gold topics and excluding singletons. Cross-document datasets have explicit topic structures, with SciCo containing over ten times more topics than ECB+. Long-document datasets range from a single test document for Animal Farm to over eleven million tokens for BookCoref, showing a large scale disparity. Medium-size benchmarks have the largest training splits, with PreCo including over 36,000 training documents. Several datasets, including Animal Farm, BookCoref, and OntoNotes, contain no singleton mentions, while others like PreCo and LitBank have many.

The cluster merging strategy (xCoRe) consistently outperforms both the append-based and mention-to-antecedent alternatives across all tested settings, with the performance gap widening as the number of document segments or sampled documents increases. The results highlight that locally extracted cluster merging is more robust than traditional approaches, especially under fragmented or partial document conditions. xCoRe achieves the highest CoNLL-F1 scores across all split and sampling configurations, surpassing both xCoRe-append and xCoRe-m2a. The advantage of xCoRe over alternatives grows with more splits or fewer documents, indicating its reliability under increasing context fragmentation. xCoRe-append degrades sharply when contexts are split or sampled, while xCoRe-m2a maintains moderate performance but remains below xCoRe.

The proposed xCoRe model outperforms the previous state-of-the-art PMCoref on cross-document benchmarks, with further gains from pretraining on LitBank. The results show consistent improvements in both predicted and gold mention settings across ECB+ and SciCo. On ECB+, the LitBank-pretrained xCoRe improves over PMCoref by 8.7 CoNLL-F1 points in the predicted mention setting. Additional pretraining on LitBank yields notable gains over the non-pretrained version, including +2.1 on ECB+ and +2.7 on SciCo. On SciCo, the pretrained xCoRe surpasses the previous best score by 7.2 points, while models using additional context, such as CDLM, show high gold-mention scores.

xCoRe shows robust performance across long-document benchmarks, achieving the highest score on Animal Farm and matching the state-of-the-art on LitBank. On BookCoref, it slightly outperforms Maverick but falls short of Longdoc, with error analysis indicating that frequent chain splitting hurts its overall score. Additional cross-document pretraining does not yield meaningful gains in this setting. xCoRe surpasses all comparison systems on Animal Farm, improving over Dual-cache by 5.9 CoNLL-F1 points. On LitBank, xCoRe scores 78.2, closely aligning with Maverick's 78.0. On BookCoref, xCoRe edges out Maverick but trails Longdoc; its within-window predictions are better, but it occasionally splits long coreference chains. Hierarchical underperforms on long documents because filtering singleton mentions from small contexts accumulates errors.

The step-wise oracle analysis shows that providing gold mentions yields large gains across most datasets, especially in cross-document settings, indicating mention identification is a primary bottleneck. However, adding gold clusters on top provides only marginal gains, highlighting cluster merging as the limiting factor in cross-context scenarios. On BookCoref, gold mentions help less because its annotation focuses on characters, making mention detection easier. Using gold mentions instead of predicted ones substantially boosts CoNLL-F1 across datasets, with the largest improvements in cross-document benchmarks. Oracle mention clustering (gold clusters) adds only modest gains after gold mentions, implying cluster merging is the main remaining challenge in cross-context tasks. BookCoref is the exception where gold mentions provide limited benefit, as its character-only annotation simplifies mention detection.

The experiments evaluate xCoRe across cross-document, long-document, and medium-size coreference benchmarks with varied scale and annotation characteristics. The cluster merging strategy consistently beats append-based and mention-to-antecedent alternatives, with its advantage growing under heavier context fragmentation or sampling. xCoRe surpasses the previous state-of-the-art on cross-document tasks (ECB+ and SciCo), with noticeable gains from LitBank pretraining, and performs competitively on long documents, achieving top scores on Animal Farm while matching Maverick on LitBank, though frequent chain splitting limits its BookCoref result. Oracle analysis reveals that mention identification is the dominant bottleneck in most settings, but cluster merging becomes the limiting factor once gold mentions are provided, particularly in cross-context scenarios.