One-sentence Summary

Peking University et al. introduce K12-KGraph, a curriculum-aligned knowledge graph extracted from People’s Education Press textbooks covering mathematics, physics, chemistry, and biology across primary, middle, and high school with nine node types (Book, Chapter, Section, Concept, Skill, Experiment, Exercise, Figure, VisualElement) and fourteen relation types, alongside K12-Bench, a 23,640-question multi-select benchmark that probes curriculum cognition through five task families (Ground, Prereq, Neighbor, Evidence, and Locate), and K12-Train, a KG-guided fine-tuning corpus of 7,335 samples (2,267 text-only and 5,068 multimodal pairs), demonstrating that even strong proprietary and open-source models reach only 57% (Gemini-3-Flash) and 46% (Gemma-4-31B-IT) exact match on K12-Bench with Prereq and Neighbor being hardest, yet domain-specific supervision from K12-Train consistently outperforms equally sized subsets of eight mainstream instruction-tuning corpora (OpenHermes, Infinity, UltraChat, WizardLM, DataFlow, LMSYS, SmolTalk, Tulu-3) on GaokaoBench and EduEval, and for VLMs, K12-Train-Full achieves best overall performance on Gaokao-MM, MDK12-medium, and K12Vista, confirming the sample efficiency and complementarity of structurally grounded curriculum data.

Key Contributions

• The paper introduces K12-KGraph, a curriculum-aligned knowledge graph extracted from official Chinese K–12 textbooks, encoding structural relationships such as prerequisite chains, concept taxonomies, and visual grounding across four subjects with nine node types and fourteen relation types.
• This work constructs K12-Bench, a 23,640-question multi-select benchmark that probes curriculum cognition through five graph-derived task families, and experiments show that even strong models achieve low exact match (e.g., 46% for Gemma-4-31B-IT), with prerequisite and neighbor tasks proving especially challenging.
• The paper develops K12-Train, a KG-guided SFT corpus (7,335 samples with text-only and multimodal splits) and demonstrates its sample efficiency: under a strictly matched 2,300-sample budget, the text-only subset outperforms eight mainstream instruction-tuning corpora on GaokaoBench and EduEval, while the full corpus surpasses larger general-purpose datasets on multimodal educational benchmarks, revealing complementary value of textual and visual supervision.

Introduction

Large language models now perform at or above human levels on K‑12 exam benchmarks, but these benchmarks only assess factual recall. They ignore curriculum cognition, the structured understanding that a skilled teacher possesses: why one topic must precede another, how experiments link to theory, and where each idea sits across chapters and textbooks. This gap leaves today’s educational AI unable to reproduce the pedagogical reasoning that supports real tutoring. The authors tackle the problem by building K12‑KGraph, a large‑scale, multimodal knowledge graph extracted from official Chinese K‑12 textbooks. From this single graph they derive both a benchmark (K12‑Bench) that measures curriculum reasoning across five task families, and a training set (K12‑Train) that explicitly teaches structural knowledge. Experiments show that even the strongest current models achieve only modest performance on curriculum cognition, and that graph‑guided fine‑tuning is markedly sample‑efficient, outperforming mainstream instruction‑tuning corpora on downstream educational tests.

Dataset

The authors construct a heterogeneous knowledge graph, K12-KGraph, from K–12 textbooks, and then derive two downstream resources: a multi-select benchmark (K12-Bench) and a supervised fine-tuning dataset (K12-Train).

K12-KGraph: base knowledge graph

• Source: OCR-parsed structured Markdown from textbook PDFs. The table of contents is used to split content into per-section files; each image is associated with its section text.
• Schema: 9 node types (including Concept, Skill, Experiment, Exercise, Figure, VisualElement, Book) and 14 edge types. Nodes carry typed properties: for example, a Concept includes name, definition, importance, and optional formula/aliases; an Experiment includes instruments, student-performed flag, process, phenomena, and conclusion. Container nodes (Chapter, Section) and their relations are structural and not included in summary statistics.
• Construction pipeline: Five automatic stages.
  1. OCR-based parsing to Markdown.
  2. Section splitting and image-text alignment.
  3. GPT-5.2 extraction of nodes and edges as JSON, with evidence citations and confidence scores.
  4. Bottom-up merging: book-level deduplication of same-name concepts/skills and remapping of edges, then subject-level reconciliation (e.g., “velocity” across grades). Light-normalized name matching followed by expert review resolves cross-book aliases.
  5. Depth-first cycle detection on is_a and prerequisites_for subgraphs; manual resolution ensures valid DAGs.
• Quality control: All triples are manually verified by 12 domain experts (Fleiss’ κ = 0.84). Edge confidence fields and evidence backlinks to textbook excerpts are collected. The extraction prompt restricts outputs to “truly important, clearly presented” knowledge and actively discourages hallucination.

K12-Bench: multi-select curriculum benchmark

• Design: 23,640 multi-select questions across five task families (9 subtasks): Knowledge Grounding, Prerequisite Reasoning, Neighbor Recommendation, Experiment Evidence Chain, and Cross-Chapter Indexing. Each question lists four candidates, and the model must output the full set of correct labels (cardinality 1 to 3). Questions and distractors are deterministically derived from K12-KGraph subgraphs; correct answers are true graph neighbors, distractors are sampled from structurally proximate but non-answer nodes (e.g., 2-hop neighborhoods or taxonomic siblings).
• Quality filters: Character 3-gram cosine similarity removes surface-form near-duplicates; a GPT-5.2 pedagogical filter discards distractors synonymous with any correct answer. A stratified spot check found 98.4% of sampled items fully correct. Because K12-Bench is directly derived from the verified graph, benchmark quality is tightly coupled to graph quality.
• Usage: Models receive only the question text (no graph context) to probe parametric curriculum knowledge.

K12-Train: KG-guided supervised training data

• Generation paths:
  • Node-grounded QA: Qwen3-235B-A22B generates a question–answer pair from a node’s typed properties (e.g., definition/formula for Concept, procedural steps for Skill, stem with reasoning for Exercise). Supervision teaches node content.
  • Edge-grounded QA: The same LLM, prompted with relation-specific templates (e.g., “Why must one learn A before B?” for prerequisites_for), produces QA pairs that articulate the relation. Supervision teaches graph structure.
  • Exercise-assessment QA: For tests_concept and tests_skill edges, deterministic templates are filled directly, ensuring full factual grounding.
• Cropping and filtering: Each node’s property set is cropped to only the fields relevant for the target QA type. Edges with confidence below a specified threshold are filtered before prompting to prevent the LLM from drifting into unrelated attributes.
• Splits and composition:
  • K12-Train-Text (2,267 pairs): purely textual QA pairs, source-balanced by random subsampling from the KG-derived pool. Includes 450 Exercise examples (with step-by-step reasoning), 356 test-edge samples, 695 node-grounded pairs (from Concept/Skill/Experiment), and 766 relation-grounded pairs (from is_a, prerequisites_for, relates_to, verifies).
  • K12-Train-MM (5,068 pairs): relation-grounded QA pairs that require visual input, covering illustrates, refers_to, requires_figure, and supports_edge, selected based on edge confidence (1.0 for the first three, 0.8 for requires_figure). This partitioning controls for data quality and mitigates natural skew in the raw KG pool.
• Validation: Every QA pair is checked for JSON structure, non-empty fields, and language consistency.
• Usage: Both subsets are used for supervised fine-tuning (SFT) of educational LLMs; the text-only branch is shown to be highly effective despite modest size because of its structural grounding.

Method

The authors construct the K12-KGraph through a five-stage automated pipeline, supplemented by human-in-the-loop quality control.

First, an OCR-based parser converts K12 textbook PDFs into structured Markdown, preserving heading hierarchies, mathematical formulas, raw text, and image assets. Next, a table-of-contents parser processes the generated Markdown to produce a manifest and splits the content into per-section files, associating each image with its corresponding section text.

For the core knowledge extraction phase, the authors prompt a large language model with schema-aware instructions to emit nodes and edges as structured JSON for each section. The model is instructed to output evidence citations from the original textbook sentences and confidence scores for every edge. The extraction prompt explicitly discourages hallucination, restricting outputs to truly important and clearly presented knowledge.

Following extraction, per-section graphs are merged bottom-up. A book-level pass assigns globally unique IDs, deduplicates concepts and skills with the same name, and remaps edges. A subject-level pass then reconciles entities across different books within the same subject. Finally, the authors run depth-first cycle detection on the is_a and prerequisites_for subgraphs to ensure structural validity, manually resolving any violations to yield valid Directed Acyclic Graphs for taxonomic and prerequisite relations. All extracted triples undergo manual verification by domain experts, and hierarchical deduplication utilizes light-normalized name matching followed by expert review to reconcile cross-book aliases.

Experiment

The evaluation first benchmarks a range of open-source and proprietary LLMs on K12-Bench, a multi-select question set derived from a curriculum knowledge graph, revealing that even the strongest models struggle with structural curriculum understanding, particularly prerequisite and neighbor relations, and small models perform near random. Fine-tuning experiments show that supervised training on K12-Train, a KG-guided dataset covering both node- and edge-grounded reasoning, consistently boosts performance on educational benchmarks such as GaokaoBench and EduEval across text-only and multimodal settings, outperforming general instruction-tuning datasets of comparable size. The gains arise from structural grounding and curriculum alignment, which impart transferable reasoning skills rather than mere fact memorization, as evidenced by improvements in non-trained subjects, and text-only and multimodal supervision sources are complementary.

The textual component of the graph encompasses 48 textbooks across mathematics, physics, chemistry, and biology. Concepts are by far the most common node type, while prerequisite, associative, and taxonomic edges form the bulk of the relations. Cross-subject leads_to links greatly outnumber the total of within-subject leads_to edges, and experiment nodes appear only in the three science subjects, not in mathematics. Concepts dominate the node set, dwarfing skills, exercises, and experiments in every subject. Prerequisite edges are the most frequent relation, followed by associative (rel_to) and taxonomic (is_a) links. Experiment nodes are completely absent from the mathematics subgraph, while present in all three science subjects. The total number of leads_to edges is predominantly cross-subject, as within-subject sums amount to less than 100. Exercises are linked far more often to concepts than to skills, with tested-concept edges outnumbering tested-skill edges by a wide margin.

The multimodal knowledge graph is largest for Mathematics, which contains substantially more nodes and edges than Physics, Chemistry, or Biology. Concept-to-visual-element edges are the most abundant relation type across all subjects, while reference edges are rare or entirely absent in some subjects. Edges involving figure and visual-element nodes overlap heavily, indicating dense interconnection between textual concepts and visual materials. Mathematics accounts for over half of the total nodes and edges in the multimodal component, reflecting its heavier use of figures and diagrams. Concept-to-visual-element edges dominate the multimodal relations, comprising the vast majority of all subject-level edge counts. Reference edges are extremely sparse; Physics and Biology have none, while Mathematics and Chemistry have only a few dozen each. The sum of edges incident to figure nodes and visual-element nodes exceeds the total number of edges, showing that most edges involve both a figure and a visual element. Chemistry, despite having the fewest nodes among the subjects, records the highest number of illustrates edges, surpassing even Mathematics.

Even the strongest proprietary model achieves only 57% exact match on K12-Bench, and open-source models lag considerably, with the best open-source model reaching 46%. Prerequisite and neighbor tasks are consistently the hardest, with exact match scores below 19% for all open-source models, indicating a lack of structured reasoning about curriculum relationships. Overall F1 scores are higher, peaking around 73%, showing that models often produce partially correct answer sets without fully capturing the correct combination of options. The random baseline yields an overall exact match of 6.7%, providing a lower bound for multi-select guessing. LLaMA-3-8B-Instruct obtains only 7.2% overall exact match, nearly indistinguishable from the random baseline and indicating no curriculum knowledge. Prerequisite and neighbor tasks are the most challenging: the best open-source model scores under 19% exact match on both, far below its performance on evidence and locate tasks. Ministral-3-14B-Instruct leads open-source models with 37.5% overall exact match and 67.4% F1, indicating frequent partially correct selections. The strongest open-source model, Gemma-4-31B-IT, reaches 46.4% exact match, still well below the best proprietary model at 57.1%. Overall F1 peaks around 73% across all models, suggesting that even when exact match is low, models often identify some correct labels but struggle to produce the complete correct set.

K12-Train-Text yields the highest overall GaokaoBench scores on both base models, exceeding the best baselines by notable margins. The curriculum-structured training improves objective and subjective question performance, and its cross-subject transfer—boosting subjects like Chinese and Humanities Math without direct supervision—indicates the acquisition of a generalizable reasoning style rather than content memorization. K12-Train-Text achieves the highest overall total on Qwen3-4B-Base (1009.96) and Llama3.1-8B-Base (625.49), surpassing the strongest baselines by +24.1 and +32.4 points respectively. On Qwen3-4B-Base, K12-Train-Text attains the best objective score rate (81.8%) and subjective score rate (89.5%). SFT on K12-Train, synthesized solely from math, physics, chemistry, and biology subgraphs, produces the highest Chinese and Humanities Math scores, evidencing transferable structured reasoning. The gains remain consistent across both base models under the same 2,300-sample budget, demonstrating robust improvements from curriculum-structured training.

After supervised fine-tuning with only 2,300 samples, K12-Train-Text achieves the highest overall average scores on EduEval for both Qwen3-4B-Base and Llama3.1-8B-Base backbones, surpassing the strongest baselines and delivering the best Ethics performance on Qwen3-4B-Base. The gains are smaller than those seen on GaokaoBench but remain consistent across architectures, demonstrating robust transfer of curriculum-structured training. On Qwen3-4B-Base, K12-Train-Text attains the top average score (66.76), edging out WizardLM (66.70), and records the strongest Ethics results among all configurations. On Llama3.1-8B-Base, K12-Train-Text reaches the highest average score (40.90), beating OpenHermes (40.05) by nearly one point under the same training budget. The pattern of improvement holds across both base models, showing that curriculum-structured training transfers consistently even when absolute margins are smaller than on GaokaoBench.

A curriculum knowledge graph built from 48 math and science textbooks evaluates structured reasoning, with concept and prerequisite edges dominating the textual component and concept-to-visual-element edges dominating the multimodal component. On K12-Bench, open-source models perform near random on prerequisite and neighbor tasks, while even the best proprietary model only reaches 57% exact match, highlighting the benchmark's difficulty. Fine-tuning on a small set of curriculum-structured training samples (K12-Train) yields consistent improvements on GaokaoBench and EduEval, including cross-subject transfer to Chinese and Humanities Math, indicating the acquisition of a generalizable reasoning style rather than content memorization.