One-sentence Summary

Beihang University et al. introduce CyberFactory, a unified open-source framework that converts public CVEs into executable, verifiable task instances and synthesizes skill-guided agentic trajectories for proof-of-concept generation, vulnerability patching, and CyberQA, and the resulting model OpenAegis attains 58.1% Pass@1 on CyberGym under a one-hour budget, improving 28.5 points over its Qwen 3.5 base model and outperforming evaluated general-purpose backbones under the same scaffold.

Key Contributions

• CyberFactory is an open-source framework that transforms public CVE artifacts into executable and verifiable task instances, unifying data construction for proof-of-concept generation, vulnerability patching, and cybersecurity question answering.
• A reusable vulnerability-analysis skill guides the teacher through source inspection, domain-guided exploration, and evidence-based validation, producing agentic trajectories with execution feedback that teach coherent security-analysis procedures without requiring the skill at inference time.
• The resulting model, OpenAegis, reaches 58.1% Pass@1 on CyberGym under a one-hour budget, improving over its Qwen 3.5 base model by 28.5 points and outperforming the evaluated general-purpose backbones under the same scaffold.

Introduction

The growing cybersecurity capabilities of large language models bring both promise and risk, driving demand for reproducible methods to inspect vulnerabilities and automate defensive actions. Prior open-source efforts offer strong models or isolated task coverage but lack a unified, reproducible training recipe; they do not provide scalable agentic data or inject domain-specific procedural guidance into model training. The authors introduce CyberFactory, an end-to-end open-source framework that converts real-world CVE artifacts into executable, verifiable instances spanning vulnerability detection, patch generation, and cybersecurity question answering. A reusable vulnerability-analysis skill guides a tool-interactive teacher agent through source inspection, domain-prior-driven exploration, and evidence-based validation, producing grounded supervision trajectories. The resulting data trains OpenAegis, a model that internalizes this prior-guided workflow and achieves 58.1% Pass@1 on the CyberGym benchmark, outperforming substantially larger baselines without needing the skill at inference time.

Dataset

The authors use several data sources to build datasets for vulnerability-related tasks, mainly for proof-of-concept construction, patch generation, and question answering.

• PoC construction instances

  • Sources: ARVO, OSS-Fuzz, and from-the-wild CVEs.
  • ARVO instances are the easiest to create because they provide pre-patch and post-patch Docker images plus a groundtruth PoC.
  • OSS-Fuzz instances provide only the commit that introduces the vulnerability. The authors use a binary search method to locate the corresponding fix commit.
  • From-the-wild CVEs are the hardest. They provide affected software version ranges and metadata such as CWE types. The authors retain CVEs with CWE types, locate fix commits from the version ranges, and use the pre-patch version as the vulnerable image and the post-patch version as the fixed image.
  • A PoC is valid only if it triggers the target behavior in the pre-patch build and does not trigger it in the post-patch build.
  • During instance verification, the authors filter out instances that inference models can already solve, keeping only challenging instances. Extra vulnerability-specific signals such as vulnerability type or crash information are used only for verification and are discarded during data synthesis and training.

• Descriptions for PoC instances

  • The authors classify fix-commit messages with an LLM.
  • High-quality commit messages are kept as-is.
  • Low-quality commit messages are rewritten into vulnerability descriptions using vulnerability-specific evidence and the fix commit.

• Patch generation data

  • Patch generation instances follow the same approach as CVE-Factory.
  • They use vulnerability-fix corpora from CVE records and neural repair formulations.

• Question-answering data

  • QA data uses an answer-first strategy.
  • Trusted answers come from execution-derived results such as crash locations and test results, structure-derived facts such as changed functions and call-graph relations, and text-extracted answers from authoritative reports.
  • The authors apply automatic checks with an LLM judge to verify that answers can be traced to the source, are not leaked in the question, and identify a unique answer.
  • Ambiguous or invalid samples are regenerated once or discarded.

• How the data is used

  • PoC construction instances are used to create executable environments and corresponding vulnerability descriptions, which serve as prompts.
  • Patch generation data is used to build instances for patch generation.
  • QA data is used to improve trustworthiness in question-answering tasks.
  • The provided text does not specify exact dataset sizes, training split ratios, or mixture ratios.

Method

The authors introduce a comprehensive data synthesis pipeline to support the training of OpenAegis. This pipeline spans vulnerability detection, vulnerability fixing, and cybersecurity question answering.

As shown in the figure below:

The end-to-end workflow, termed the Cyber Factory, orchestrates the process from source adaptation through multi-task training and evaluation.

The pipeline begins by adapting diverse vulnerability sources, including ARVO, OSS-Fuzz, and wild CVEs. A safety filter processes these inputs to yield normalized evidence. For proof-of-concept construction, the authors reconstruct executable instances by pairing pre-patch and post-patch Docker images. For OSS-Fuzz vulnerabilities, a binary search method locates the specific vulnerability fix commit. Wild CVEs undergo a three-step pipeline that retains those with Common Weakness Enumeration types, locates fix commits within affected version ranges, and verifies instances by filtering out those that baseline models can already solve. To generate high-quality vulnerability descriptions, the system classifies fix commit messages using a large language model and supplements low-quality messages with vulnerability-specific evidence.

To ensure data reliability, the framework implements rigorous quality calibration. Behavior checks validate that candidate proof-of-concept inputs trigger the target vulnerability in the pre-patch build but not in the post-patch build. A difficulty check filters out instances that are either too easy or unsolvable. For patch generation, the authors build upon established vulnerability fix corpora. In the question-answering module, an answer-first strategy is adopted to guarantee trustworthiness. Trusted answers are derived from execution results, structural facts, and authoritative reports. An LLM-based judge then verifies that answers are traceable, not leaked in the question, and uniquely identifiable. Following calibration, a sanitization engine processes private evidence, such as crash locations and fix commits, to generate sanitized and agent-visible tasks comprising code, descriptions, and executables.

The final stage focuses on synthesizing trajectories and training the model. To prevent inefficient exploration during rollouts, the authors encode a task-independent vulnerability analysis skill. This skill directs the model to inspect targets, utilize appropriate testing techniques, and revise its approach upon validation failure, transferring this procedural knowledge into the model parameters via supervised fine-tuning. To handle long-horizon contexts that approach the 256K token limit, a structured compaction protocol is applied. When context usage reaches 90 percent, the accumulated trajectory is compressed into a continuation state that retains verified evidence, failed attempts, and pending actions while discarding redundant logs. The synthesized trajectories for proof-of-concept generation, patching, and question answering are then utilized to train and evaluate OpenAegis across multiple cybersecurity tasks.

Experiment

The evaluation uses the CyberGym task, where agents must synthesize a proof-of-concept input that triggers a vulnerability on a pre-patch build but not on the patched build, under a one-hour budget and with a differential oracle. OpenAegis, fine-tuned from Qwen 3.5-397B-A17B on trajectories generated with a vulnerability-analysis skill, achieves the highest success rate, substantially outperforming its base model and larger general-purpose agents. Analysis shows that the skill elicits a systematic exploration-and-validation workflow from the teacher, which supervised fine-tuning internalizes into OpenAegis, leading to more consolidated tool use, stronger instrumentation, and more selective submission without requiring the skill at inference time.

OpenAegis achieves the highest Pass@1 (58.1%) among evaluated models. Specialization on the Qwen 3.5 backbone yields a 28.5-point improvement, demonstrating that training trajectories contribute substantially beyond the base checkpoint. The model also outperforms larger general-purpose systems, surpassing GLM 5.2 by 14.8 points and Kimi K2.7 by 6.4 points, despite using fewer parameters. OpenAegis attains the highest Pass@1 (58.1%), outperforming models with up to 1T total parameters. Fine-tuning the same Qwen 3.5 backbone lifts Pass@1 by 28.5 points, the largest gain among all comparisons. OpenAegis surpasses GLM 5.2 and Kimi K2.7 by 14.8 and 6.4 points, respectively, while using fewer parameters.

Compacting context at 90% utilization achieves the highest Pass@1 and the lowest context exhaustion rate among the evaluated strategies. The advantage is especially large on long-horizon tasks requiring more than 40 tool interactions, where compact execution substantially reduces context-exhaustion failures. Adjusting the trigger threshold indicates that 90% provides a practical trade-off, as earlier compaction reduces effectiveness and later compaction leaves insufficient room for follow-up tool calls. Compact execution at 90% context utilization attains the highest overall Pass@1 (58.1%) and the lowest context exhaustion rate (7.0%). Simple truncation performs worst, with the lowest Pass@1 (45.6%) and the highest context exhaustion rate (24.5%). On tasks with more than 40 tool interactions, compact execution improves Pass@1 by 8.5 points and reduces context exhaustion by 11.7 points compared to full history. Triggering compaction at 80% instead of 90% yields lower Pass@1 and higher context exhaustion, confirming 90% as a favorable operating point.

Applying the vulnerability-analysis skill to GLM 5.2 at inference time improved Pass@1 even though each skill-guided attempt received only one quarter of the wall-clock time. The comparison used one 60-minute attempt without the skill and five 15-minute attempts with the skill, so it is not compute-matched. The result suggests substantially higher synthesis throughput rather than an equal-compute improvement. Skill-guided GLM 5.2 achieved a higher Pass@1 despite using shorter individual attempts and only one quarter of the run time per attempt. The without-skill condition used one 60-minute attempt, while the skill-guided condition used five 15-minute attempts, so the gain reflects synthesis throughput rather than equal-compute improvement. The improvement accrues during data construction and does not introduce an inference-time dependency into OpenAegis.

Injecting the vulnerability-analysis skill shifts the model from ad-hoc direct attempts to systematic domain-guided exploration and evidence-based validation. Exploration and validation coverage become nearly universal, and per-trajectory call counts rise by orders of magnitude, while operations per shell call slightly decrease. Exploration coverage jumps from negligible to nearly every trajectory with the skill. Validation coverage rises from near zero to almost all trajectories. The number of exploration and validation calls per trajectory increases dramatically. Operations per shell call slightly drops, indicating the skill does not directly drive action consolidation.

Explicitly providing the domain skill to the teacher model greatly increases exploration and validation calls per trajectory. The fine-tuned OpenAegis model, which never sees the skill at inference, exhibits a similarly elevated pattern, indicating that the exploration-and-validation workflow has been internalized through training. Skill injection in GLM 5.2 raises exploration and validation calls from near zero to over two per trajectory. OpenAegis achieves exploration and validation frequencies comparable to the skill-guided teacher without receiving the skill at inference time. Base models perform almost no exploration or validation, underscoring the behavioral shift driven by skill elicitation and subsequent internalization.

The experiments evaluate OpenAegis on Pass@1 and context management, showing that fine-tuning a Qwen 3.5 backbone yields large gains and outperforms larger general-purpose models despite using fewer parameters. Compact execution at 90 percent context utilization achieves the best Pass@1 and lowest context exhaustion, with the largest benefits on long-horizon tasks requiring many tool interactions. Inference-time injection of a vulnerability-analysis skill improves synthesis throughput by eliciting systematic exploration and validation, and the fine-tuned OpenAegis model internalizes this behavior without needing the skill at inference. Base models rarely explore or validate, underscoring the behavioral shift from skill elicitation and subsequent training.