One-sentence Summary

VIDRAFT and Ginigen AI introduce FINAL Bench, the first benchmark for systematic evaluation of functional metacognition in LLMs, utilizing 100 expert-level tasks with hidden cognitive traps and a 5-axis rubric to expose a critical declarative-procedural dissociation in which Error Recovery is the dominant bottleneck (94.8% of MetaCog gain) and a strong anticorrelation (Pearson $r = -0.777$r=−0.777) exists between baseline score and metacognitive improvement.

Key Contributions

• FINAL Bench is introduced as a benchmark of 100 expert-level tasks across 15 domains and 8 cognitive types, each with hidden cognitive traps to evaluate functional metacognition in LLMs; nine state-of-the-art models are tested under Baseline and MetaCog conditions, and the full dataset and scoring code are released.
• A 5-axis rubric disentangles declarative metacognition (Metacognitive Accuracy) from procedural metacognition (Error Recovery), providing the first quantitative measurement of the Declarative-Procedural Gap, the dissociation between verbalizing uncertainty and acting to correct errors.
• Empirical analysis of 1,800 evaluations shows that 94.8% of metacognitive improvement comes from Error Recovery, all nine models exhibit a mean accuracy-recovery gap of 0.392 at baseline, and baseline score is strongly anticorrelated with metacognitive gain (Pearson r = −0.777, p < 0.001), establishing Error Recovery as the primary bottleneck in LLM reasoning.

Introduction

Recent reasoning models exhibit emergent self-correction, yet AI evaluation remains fixated on final-answer accuracy, ignoring whether a model knows it is wrong. Existing benchmarks are saturated in knowledge and reasoning, and even nascent self-awareness probes only assess binary detection without measuring the full detect-acknowledge-correct pipeline. The authors address this gap by introducing FINAL Bench, a benchmark of 100 expert-level tasks embedded with cognitive traps and scored on a five-axis rubric that separates Metacognitive Accuracy (declarative self-monitoring) from Error Recovery (procedural correction). This enables the first quantification of the Declarative-Procedural Gap in LLMs, revealing that error recovery dominates performance while models consistently verbalize uncertainty they fail to act upon.

Dataset

The FINAL Bench dataset is a benchmark built around four design principles aimed at measuring metacognitive capabilities in language models. Here is a concise breakdown of its composition, sources, processing, and usage.

• Composition and sources The benchmark consists of 100 tasks. Each task is constructed to contain hidden cognitive traps — stimuli that deliberately trigger metacognitive failure. The tasks are not sourced from existing corpora but are purpose-built by the authors to satisfy the functional measurement principle (observing only behavioral patterns) and the trap-embedded principle. The dataset is split into two comparative conditions: a baseline condition and a MetaCog condition, which isolates the causal effect of metacognitive support.

• Key details for each subset

• Baseline condition: Tasks are presented without any additional metacognitive scaffolding. The model must answer directly, exposing its raw metacognitive awareness (MA) and error correction (ER) behaviors.

• MetaCog condition: The same tasks are presented with a metacognitive support intervention. The difference in performance between conditions quantifies the model’s ability to leverage metacognitive cues.

• Both subsets share the same 100 tasks; the split is purely by condition, not by task identity. No explicit size differences are given because the number of tasks remains constant.

• Data processing and schema Task design follows the declarative–procedural separation principle: each task is structured to independently score MA (knowing something is wrong) and ER (actually correcting the error). This means every task yields two separate metrics, and the gap between them is a key output. Processing involves embedding traps that are not trivial to detect, ensuring the measurement is based on observable behavioral patterns. No additional metadata or cropping strategies are described; the entire construction is oriented toward creating a controlled experimental setup.

• How the paper uses the data The authors use FINAL Bench solely for evaluation, not for training. Both baseline and MetaCog conditions are run on a model, and the resulting MA and ER scores are compared. The design enables a causal analysis: the difference between conditions reveals the impact of metacognitive support, while the MA–ER gap quantifies the model’s internal misalignment between knowing and doing. No training split or mixture ratios apply, as the dataset is a benchmark.

Method

The authors introduce FINAL Bench, a benchmark designed to systematically measure the metacognitive capabilities of language models. Its construction is guided by four core design principles. First, only observable behavioral patterns are measured, adhering to a strict functionalist stance (P1). Second, every task embeds a hidden cognitive trap engineered to trigger metacognitive failure, so that successful navigation requires genuine self-monitoring (P2). Third, metacognitive awareness (MA) and error recovery (ER) are scored independently, enabling a quantitative separation of “knowing one is wrong” from “actually correcting the error” (P3). Fourth, comparative conditions are built into the evaluation protocol, with a Baseline and a MetaCog condition that serve to isolate the causal effect of metacognitive support (P4).

The benchmark consists of 100 tasks spanning 15 domains, three difficulty grades, and eight different metacognitive failure types as defined by the TICOS framework. Each task provides a prompt of 100–500 words that masks a hidden trap, together with an expected metacognitive behavior, its TICOS classification, and a difficulty grade. The TICOS taxonomy classifies every task into one of eight functional metacognitive types, ensuring systematic coverage of distinct cognitive failure patterns.

Performance is quantified through a five-axis rubric that yields a single FINAL Score. The five axes are Prompt Quality (PQ), Metacognitive Awareness (MA), Error Recovery (ER), Intent Detection (ID), and Failure Classification (FC). The overall score is computed as a weighted average across tasks, with higher difficulty grades receiving proportionally larger weights. Formally,

$\text{FINAL\_Score} = \frac{\sum \left( \text{weighted\_score}_i \times \text{grade\_weight}_i \right)}{\sum \text{grade\_weight}_i}$FINAL_Score=∑grade_weighti​∑(weighted_scorei​×grade_weighti​)​

where the per-task weighted score is a linear combination of rubric dimensions:

$\text{weighted\_score} = 0.15 \cdot \text{PQ} + 0.20 \cdot \text{MA} + 0.25 \cdot \text{ER} + 0.20 \cdot \text{ID} + 0.20 \cdot \text{FC}.$weighted_score=0.15⋅PQ+0.20⋅MA+0.25⋅ER+0.20⋅ID+0.20⋅FC.

All evaluations are performed by a tri-model LLM-as-Judge ensemble comprising GPT-5.2, Claude Opus 4.6, and Gemini 3 Pro. Each judge scores every response independently in a blinded setup using Structured Output mode. The ensemble mean achieves strong alignment with human ratings, as evidenced by a Cohen’s $\kappa = 0.87$κ=0.87.

The evaluation protocol defines two distinct modes to assess the influence of external metacognitive scaffolding. In the Baseline condition, models receive a single API call without any self-correction prompting, reflecting their raw default behavior. The MetaCog condition, in contrast, wraps the model in an external self-correction scaffold that follows a three-phase pipeline: (1) Initial Reasoning, (2) Critical Self-Review, and (3) Corrective Revision. The metric $\Delta_{\text{MC}} = \text{MetaCog Score} - \text{Baseline Score}$ΔMC​=MetaCog Score−Baseline Score then captures the pure effect of the scaffold, independent of the model’s original capabilities.

Experiment

FINAL Bench evaluates black-box LLMs on multiple metacognitive axes by comparing nine models across 100 tasks in baseline and MetaCog conditions. The baseline leaderboard reveals a universal metacognitive awareness–error recovery gap, with error recovery being the weakest dimension for all models. Applying a self-correction scaffold improves error recovery dramatically but barely alters declarative metacognition, making this procedural gain the dominant source of overall improvement and causing harder tasks to benefit most. This pattern is interpreted as a monitoring-control dissociation, where current models can detect but not autonomously correct their mistakes.

FINAL Bench evaluates metacognitive abilities in large language models across 15 diverse domains and five axes, contrasting with prior benchmarks that relied on narrower or binary assessments. Experiments on nine state-of-the-art models show that error recovery is consistently the weakest axis, yet it captures nearly all improvement when a self-correction scaffold is applied. The scaffold's effect reveals a 15-fold procedural-declarative gap, and harder tasks exhibit disproportionately larger gains, pointing to a monitoring-control dissociation in current models. Error Recovery is the lowest-performing axis for all nine models, with a mean score of 0.302 and a floor effect in four-fifths of baseline evaluations. Without a scaffold, every model shows a large gap favoring Meta-Awareness over Error Recovery (mean difference 0.392). Metacognitive scaffolding reverses the Meta-Awareness–Error Recovery gap, making it negative for all nine models. 94.8% of the scaffolded improvement is concentrated in the Error Recovery axis. The scaffold boosts Error Recovery by 0.533 (procedural) while Meta-Awareness improves by only 0.035 (declarative), a 15-fold difference. Baseline performance and scaffold gain are strongly anticorrelated (r = −0.777), meaning harder tasks benefit the most from the scaffold.

The framework distinguishes surface self-reflection (monitoring accuracy, MA) from behavioral self-correction (error recovery, ER), while embedding-space uncertainty is deferred to open-source studies. Baseline experiments show that all nine models exhibit a large gap between strong monitoring and weak error recovery, but a self-correction scaffold reverses this gap, with 94.8% of the metacognitive gain coming from ER. The improvement is strongly anticorrelated with baseline performance, and the dissociation parallels classic monitoring–control distinctions in cognitive psychology. All nine models show a universal MA > ER gap in baseline, with ER being the lowest axis and stuck at 0.25 in 79.6% of evaluations. Under the MetaCog condition, the MA–ER gap reverses, and 94.8% of the overall gain originates from the error recovery axis alone. The self-correction scaffold increases procedural ER by +0.533 but declarative MA by only +0.035, a 15× differential. MetaCog gains are strongly anticorrelated with baseline scores (r = –0.777), so harder tasks yield dramatically larger improvements.

All evaluated models show a floor effect in error recovery at baseline, with a large gap between declarative monitoring ability and procedural error recovery. Applying a self-correction scaffold boosts procedural error recovery by 0.533 while improving declarative monitoring by only 0.035, reversing the gap and demonstrating that 94.8% of the gain originates from the error recovery axis. This asymmetry mirrors the cognitive dissociation between monitoring and control, indicating that current language models can detect errors but cannot yet effectively correct them. Error recovery is the lowest axis for all models, with a mean baseline score of 0.302 and a floor effect in 79.6% of evaluations. All models exhibit a universal gap between monitoring and error recovery of 0.392, with monitoring consistently outscoring error recovery. Under the self-correction scaffold, the gap between monitoring and error recovery reverses, and 94.8% of improvement comes from error recovery. The scaffold's procedural self-correction gain is 15 times larger than its declarative monitoring gain, highlighting a stark procedural-declarative asymmetry. Harder tasks yield larger scaffold-driven gains, with a strong negative correlation (r = -0.777) between baseline performance and improvement.

All nine models show a pronounced error recovery floor effect, with ER substantially lower than monitoring accuracy and a universal positive MA–ER gap. Under metacognitive scaffolding, this gap reverses, and the overwhelming majority of improvement comes from ER alone, while MA barely changes. The pattern reveals a declarative-procedural dissociation: models can monitor their own errors but struggle to correct them without extra intervention. Every model scored lowest on error recovery, with ER values around 0.25 in roughly 80% of baseline evaluations. The monitoring accuracy–error recovery gap is universally positive across all models, averaging 0.392. Under metacognitive conditions, the MA–ER gap becomes negative for all models, and 94.8% of the gain originates from the error recovery axis. Self-correction scaffolding increases ER by +0.533 but monitoring accuracy by only +0.035, a 15-fold differential. Baseline performance and metacognitive improvement are strongly anticorrelated (Pearson r = –0.777), so harder tasks gain substantially more from scaffolding. Token competition may cause small declines in identification and fact-checking scores when self-correction is applied.

Under the MetaCog self-correction scaffold, Kimi K2.5 tops the leaderboard with a FINAL score of 78.54, propelled by the highest error recovery among all models. Across all nine models, error recovery dominates every other axis, creating a universal negative monitoring–recovery gap where procedural control outstrips declarative monitoring. Every model records a negative MA–ER gap, meaning error recovery surpasses monitoring accuracy after self-correction, reversing the baseline pattern. Kimi K2.5 achieves the best FINAL score and the highest error recovery (0.908), while GPT-5.2 leads on monitoring accuracy and factual correctness but lags in error recovery.

This benchmark evaluates metacognitive abilities across 15 diverse domains and five axes, contrasting with earlier narrower assessments. Experiments on nine state-of-the-art models reveal that error recovery is consistently the weakest axis, but a self-correction scaffold reverses the large baseline gap between monitoring accuracy and error correction, with nearly all improvement concentrated in procedural self-correction. This dissociation shows that models can detect errors yet struggle to correct them without external intervention, and harder tasks benefit disproportionately from scaffolding.