Metacognition-Bench, released by ginigen-ai, is a benchmark dataset for evaluating the metacognitive abilities of large language models. It aims to measure the model's functional metacognitive ability to detect and correct its own reasoning errors, rather than simply assessing the accuracy of the final answer. Related research papers include... FINAL Bench: Measuring Functional Metacognitive Reasoning in Large Language Models . This dataset contains 300 metacognitive trap questions, covering 121 fields including mathematics, physics, biology, law, medicine, economics, statistics, ethics, and computer science. The difficulty levels are divided into three levels: A (cutting-edge), B (expert), and C (core), encompassing eight metacognitive behavior types: self-correction, trap escape, transition detection, conflict resolution, incremental discovery, multi-constraint handling, expert panel, and uncertainty decision-making. Each question embeds a hidden trap to test the model's performance when faced with inertial but erroneous reasoning paths. The dataset uses two task formats—multiple-choice and open-ended generation—for independent scoring to objectively measure the model's vulnerability to adversarial reasoning traps and the adapter's error-correcting gain.

Data fields:

• task_id: A unique identifier (format META-001 to META-300)
• domain: the domain to which the question belongs
• grade: Difficulty level (A/B/C)
• ticos_type: Metacognitive behavioral types (8 types in total)
• difficulty: Qualitative difficulty description
• prompt: Task prompt word
• expected_behavior: The specific points where correct reasoning and intuition are prone to error.
• hidden_trap: Hidden error reasoning paths
• ticos_required: Required TICOS metacognitive code