Command Palette
Search for a command to run...
WeVisForensics : un agent auto-évolutif pour la détection omni-modale de vidéos générées par IA
WeVisForensics : un agent auto-évolutif pour la détection omni-modale de vidéos générées par IA
Yipeng Du Yuxi Li Ying Zhang Chong Sun Chen Li Jing Lyu Ying Tai
Résumé
Les progrès récents en génération vidéo exigent une détection fiable fondée sur des preuves forensiques explicites. Cependant, les méthodes actuelles reposent principalement sur un raisonnement en chaîne de pensée à passage unique qui néglige les artefacts brefs ou localisés, ou sur des flux de travail multi-agents prédéfinis offrant une flexibilité limitée dans le choix des inspections pertinentes. Même avec une investigation adaptative, l'optimisation de la seule politique d'investigation n'améliore pas directement les capacités forensiques d'experts figés. Nous présentons WeVisForensics, un cadre agentique dans lequel un agent enquêteur principal coordonne une bibliothèque auto-évolutive de sous-agents inspecteurs forensiques couvrant les indices visuels et audio. L'enquêteur apprend quel type d'artefact examiner, sur quelle fenêtre temporelle et à quel moment conclure, et ces interactions fournissent un retour partagé aux deux niveaux d'agents. Notre stratégie d'optimisation de politique à double échelle temporelle, DTPO, referme cette boucle par un apprentissage par renforcement rapide de l'enquêteur et un affinement plus lent des inspecteurs, guidé par des intervalles d'artefacts annotés, afin de signaler des artefacts subtils qu'ils avaient auparavant écartés. En outre, nous constituons ForenCue-30K, qui combine des annotations forensiques spécifiques aux rôles et des trajectoires d'interaction pour superviser les inspections individuelles et leur coordination, avec un ensemble d'apprentissage par renforcement associé. Les expériences montrent que WeVisForensics atteint en moyenne 95,2 % de précision de détection sur des bancs d'essai intra-domaine, hors-domaine et agentiques, soit 9,0 points de plus que le meilleur détecteur antérieur, tout en fondant ses verdicts sur des preuves localisées.
One-sentence Summary
Researchers from Nanjing University, WeChat Vision, and Tencent Inc. propose WeVisForensics, a self-evolving agentic framework for omni AI-generated video detection in which a main investigator coordinates forensic inspector sub-agents across visual and audio cues, optimized via dual-timescale policy optimization (DTPO) on ForenCue-30K to localize subtle artifacts and ground its verdicts in localized evidence, achieving 95.2% detection accuracy across in-domain, out-of-domain and agentic benchmarks, 9.0% above the strongest prior detector.
Key Contributions
- DTPO is a dual-timescale policy optimization algorithm in which the investigator agent is optimized via fast reinforcement learning while inspector sub-agents are refined more slowly from the investigator's issued calls, turning its rollouts into their curriculum.
- WeVisForensics is an agentic video forensics framework in which a main investigator agent adaptively dispatches inspections across a self-evolving library of visual and audio forensic inspectors. It averages 95.2% detection accuracy across in-domain, out-of-domain, and agentic benchmarks, 9.0% above the strongest prior detector.
- ForenCue-30K is a role-structured dataset with per-role forensic annotations, executed agentic trajectories, and a reinforcement-learning pool, and AgenticCue is introduced as a benchmark evaluating both detection verdicts and whether relevant sub-agents are invoked.
Introduction
Advances in video generation have made synthetic content increasingly realistic, so reliable AI-generated video detection is now an essential media-forensics task that depends on subtle visual, textual, and audio artifacts. Prior multimodal large language model detectors either judge a full video in a single holistic pass, which can miss brief or spatially small artifacts and produce unverifiable rationales, or coordinate fixed expert agents that do not improve from the interactions they serve. The authors introduce WeVisForensics, an agentic framework in which a main investigator adaptively dispatches trainable forensic inspector sub-agents and co-evolves with them through Dual-Timescale Policy Optimization (DTPO). They also curate ForenCue-30K, a role-structured forensic dataset with executed agent trajectories, and AgenticCue, a benchmark that evaluates both detection accuracy and inspection precision.
Dataset
The authors build ForenCue-30K from 30,000 curated videos. Its construction follows three principles: broad generator coverage, a role-aligned forensic taxonomy, and temporally grounded evidence annotation. The dataset supplies multi-level supervision: 117,552 role-specific records for sub-agent inspectors, 33,896 interaction trajectories for the main agent investigator, and a 7,500-row RL pool for DTPO. The name ForenCue-30K refers to the core video supervision set, while expanded annotation records, trajectories, and the RL pool are counted separately.
Sources and composition
- Generated videos are primarily curated from public datasets: GenBuster-200K, RewardData, VBench-2.0, and AIGVDBench.
- These are supplemented with videos from AI video-generation platforms and specialized forgery domains, including digital humans and speaker-mismatched audio forgeries.
- Training generators include CogVideoX, HunyuanVideo, LTX-Video, EasyAnimate, Wan 2.1/2.2, Gen-2/3, Pika 1.5, Kling 1.0/1.5, MiniMax-Video-01, Veo 3, and Vidu Q1.
- Real videos come from OpenVid-1M and Koala-36M, plus stock platforms Pexels and Pixabay.
Role-grounded supervision records
- The taxonomy matches six forensic roles: Visual appearance, Physical consistency, Common sense, OCR consistency, ASR analysis, and Audio profile.
- Reported record counts include 26,378 for Visual appearance, 32,348 for Physical consistency, 40,246 for Common sense, and 18,580 for OCR consistency.
- Each record isolates one perspective and stores a verdict of issue, no-issue, or inconclusive, plus confidence, explanation, and the temporal interval of the artifact.
- Real videos contribute records under the same format, so each role sees both positive and negative outcomes.
Processing and filtering
- A rubric-conditioned VLM annotator examines timestamped frames under role-specific guidelines and localizes observable artifacts within temporal intervals.
- Explanations must refer strictly to observable visual evidence. Generated videos with no visible anomaly for a role are recorded as no-issue for that role.
- A cross-family VLM verifier checks and refines or rejects candidate annotations to filter hallucinated explanations, over-generalized commonsense claims, and imprecise intervals.
- Human reviewers inspect sampled annotations and directly annotate audio inconsistencies while watching clips with sound.
- In an audit of 340 video-role annotations, 99.1% are supported by visual evidence.
Inspector and trajectory data
- Inspector records use the same input the sub-agent sees at inference: frames from the requested window at 2 FPS for window-based roles, and the frame at the requested timestamp for OCR.
- Targets are structured reports containing verdict, confidence, and an explanation that cites timestamps within the window.
- Interaction trajectories are built through execution-grounded synthesis. A VLM planner selects roles and temporal scopes, calls the corresponding inspectors, and writes reasoning from the actual returned reports.
- Training-only metadata is stripped from student-facing messages. Trajectories average about two calls under a four-call budget.
- In a human audit of 100 trajectories, 96.0% show appropriate tool selection.
RL pool and quality control
- The RL pool contains 7,500 rows from 15 collections and 56 source subsets, with 3,750 unique real clips and 3,180 unique generated clips repeated with source-dependent weights.
- Each generated clip includes role-specific artifact intervals for selecting inspector calls and a verified set of relevant roles for the tool-coverage reward.
- No global train mixture ratio is explicitly reported; the RL pool balances labels and sources through source-dependent weights.
- Export checks validate record structure, role-specific fields, video duration, and sampled-frame availability.
- Trajectory checks match call identifiers to stored responses, enforce action budgets and protocol rules, and screen for training-only metadata leakage.
- Each trajectory retains video and source identifiers, sampled-frame timestamps, ordered messages, and target verdict.
AgenticCue benchmark
- AgenticCue is separate from ForenCue-30K and shares no videos with it.
- It targets newer commercial generators, including Veo 3.1, Kling 2.5/3.0, Seedance 1.5/2.0, Hailuo 2.3/H3, Wan 2.6, Sora 2, Jimeng, and HappyHorse 1.0, plus held-out digital-human, face-swap, and voice-mismatch audio forgeries.
- It contains five forensic categories: Visual, Physical, OCR, Common sense, and Audio. Each category has 500 generated videos verified to exhibit that category’s artifact and shares the same 500 real videos, forming 1,000 balanced videos per category.
- Some clips belong to multiple categories: 170 clips do so, yielding 2,330 unique generated videos.
How the data is used
- The 117,552 role-specific records train the sub-agent inspectors.
- The 33,896 interaction trajectories train the main agent investigator through execution-grounded tool use.
- The RL pool is used for DTPO, with artifact intervals grounding inspector-call selection and relevant roles defining the tool-coverage reward.
- AgenticCue is used for held-out evaluation, reporting binary detection accuracy and invocation precision rather than final-verdict-only performance.
Method
The authors leverage an agentic framework, WeVisForensics, to treat AI-generated video detection as a structured investigation rather than a single perceptual act. The system comprises a main agent acting as an investigator and a Forensic Sub-agent Library consisting of specialized inspectors. This design enforces a strict separation between planning and perception, ensuring that the investigator decides what to check and where, while each inspector evaluates the presence of specific artifacts within the requested scope.
An investigation begins with a global overview of the video, summarizing sampled frames, duration, and audio presence. At each turn, the investigator emits a structured message containing a reasoning field and an action. The action either invokes a specific sub-agent with a temporal or spatial scope, or terminates the process with a final verdict. The protocol restricts the investigator to a maximum of four tool calls to encourage efficient evidence gathering. The sub-agent library covers six forensic perspectives, including four visual inspectors for appearance, physical consistency, common sense, and on-screen text, alongside two fixed audio tools for voice profiling and ASR analysis. Each visual inspector returns a structured report comprising a verdict, a calibrated confidence level, and a grounded explanation. The OCR inspector operates as a three-stage pipeline, localizing text, routing for genuine text presence, and performing crop-level glyph inspection to identify generation-degraded characters.
To train this multi-agent system, the authors construct a role-grounded supervision pipeline that mirrors the agent architecture. Curated videos are annotated separately for every forensic role, converting these annotations into training records for the inspectors and executing inspector calls to assemble interaction trajectories for the investigator.
This process yields a comprehensive dataset that provides multi-level supervision comprising role-specific artifact records, interaction trajectories, and a reinforcement learning pool. The role-structured taxonomy categorizes artifacts into fine-grained dimensions such as texture stability, kinematic motion, causality, and glyph structure. This ensures that the evidence an inspector learns to report aligns perfectly with the evidence the investigator learns to request and interpret.
The training pipeline proceeds in two stages. In the first stage, supervised fine-tuning initializes the models. All agents share a frozen backbone, while the investigator and the four visual inspectors are trained as separate low-rank adapters. This parameterization allows the system to be served efficiently as a single backbone with lightweight adapters.
The second stage employs Distillation and Trajectory Policy Optimization to jointly improve the investigator and the inspectors through their own interactions. The fast process updates the investigator policy using Group Relative Policy Optimization. To address the sparsity of the terminal reward, the authors introduce Call-Level Utility Estimation, which measures the change in a frozen reference model's support for the correct label after each inspection call. The CLUE score for call j is computed as:
cj=clip(log−ℓj++ϵ−ℓj−+ϵ,−b,b)where ℓj− and ℓj+ represent the log probabilities of the correct label before and after the response, respectively. This provides a per-action credit signal, allowing inspection choices to receive feedback on the informativeness of the evidence produced. A tool-coverage reward further penalizes investigations that reach the correct verdict through irrelevant inspections.
The slow process refines the inspectors using privileged information distillation. Interaction calls are grounded in annotated artifact intervals. An exponential-moving-average teacher receives privileged context, such as the video label or annotated intervals, while the student relies solely on inference-time inputs. The objective combines a distillation loss with a label correction term that contrasts opposing verdicts, ensuring the inspector corrects false alarms and misses without suppressing valid inconclusive responses. Additionally, the system incorporates skill-card evolution, where a frozen reflector model proposes explicit reporting rules to handle specific failure modes, such as distinguishing motion blur from structural defects. These updates are synchronized periodically, allowing the investigator to learn between inspector refreshes and ensuring the forensic sub-agent library adapts to the changing distribution of generators.
Experiment
The work introduces the AgenticCue benchmark, an out-of-domain evaluation covering newer commercial generators and five forensic categories, which scores both final accuracy and whether the agent invokes the relevant specialist tools. Main experiments use GenBuster-Bench as the in-domain benchmark and GenBuster-Bench++ plus MintVid as out-of-domain benchmarks, and show that WeVisForensics with tool-augmented inspection generalizes better than binary detectors and chain-of-thought MLLMs, especially on artifact categories where baselines collapse. Ablations demonstrate that fast main-policy optimization and slower sub-agent evolution are complementary, with the evolving inspectors producing sharper evidence and the trained main agent issuing more targeted calls. Visual and qualitative examples confirm that correct fake decisions often require both an inspector that can report subtle artifacts and a main policy that can appropriately weigh that evidence.
The forensic sub-agent library covers six roles: four visual inspectors and two audio tools. Three visual inspectors examine window frames at 2 FPS and return issue, no issue, or inconclusive, while OCR operates on a timestamped frame with text crops and returns text_ok, text_suspicious, or no_text. Jointly evolving the library and the main agent yields complementary accuracy gains, with evolved inspectors learning to flag artifacts they previously treated as coherent. The appearance, physical consistency, and common sense inspectors share a window scope, frames at 2 FPS input, and issue, no issue, or inconclusive verdicts. OCR consistency is timestamp-scoped and receives frame plus text crops, returning text_ok, text_suspicious, or no_text. Audio analysis splits into a fixed voice profile member and an ASR member that checks whole-track transcript timings for naturalness or timing anomalies. Evolving only the library improves the SFT investigator, but evolving the library together with the DTPO-trained investigator improves accuracy more than the sum of the separate gains. Evolved sub-agents learn to report artifacts they once called coherent, reducing missed anomalies on generated videos and improving evidence F1.
ForenCue-30K is the most complete forensic video training dataset among those compared, adding role-specific records, temporal artifact intervals, OCR/audio annotations, call-observation trajectories, and agentic tool-oriented evaluation. Existing datasets provide only partial supervision, such as explanations alone or explanations with either role or temporal labels. The added trajectory and agentic signals support main-agent and sub-agent training beyond what prior resources offer. ForenCue-30K is the only compared dataset that includes OCR/audio annotations, call-observation trajectories, and agentic tool-oriented evaluation. Temporal artifact intervals appear only in ViF-CoT-4K and ForenCue-30K, while role-level annotations appear only in FaceVid-Forensics-100K and ForenCue-30K. Earlier datasets cover only subsets of these supervision signals, with none combining role, temporal, evidence modality, trajectory, and agentic annotations.
Generated videos are most likely to warrant appearance inspection, followed by physical inspection, while OCR, common sense, and audio roles are progressively rarer. The base rates show what fraction of generated videos call for each role, which is also the precision achieved by invoking a specialist on every eligible video. For less common roles, uniform invocation is therefore increasingly wasteful and targeted invocation becomes more important. Appearance and physical artifacts are the most prevalent roles, making indiscriminate invocation of those specialists relatively efficient. OCR, common sense, and audio cues are much less frequent, so fixed invocation of those specialists would often target videos that do not need them.
WeVisForensics achieves the strongest overall detection performance across in-domain, out-of-domain, and AgenticCue evaluations, outperforming binary, generic MLLM, and chain-of-thought baselines. Binary detectors remain near chance on in-domain data, while generic MLLMs and chain-of-thought approaches struggle with unseen generators and fine-grained artifact categories. WeVisForensics maintains high balanced accuracy across AgenticCue categories, and DTPO yields consistent gains over the supervised fine-tuned agent. WeVisForensics ranks first on out-of-domain benchmarks, including the six-generator MintVid mean and AgenticCue artifact categories. Binary detectors stay near chance on the in-domain split and show limited effectiveness on AI-generated video detection. Generic MLLMs perform well on commonsense violations but drop substantially on fine-grained categories such as audio and OCR. Chain-of-thought detectors degrade sharply across unseen generators and remain near chance on several AgenticCue artifact categories. DTPO improves over the supervised fine-tuned agent on all AgenticCue categories, with the largest gains in visual appearance and OCR.
The DTPO-trained agent matches or improves sub-agent invocation precision across all five roles relative to the SFT agent while issuing fewer tool calls per video. The largest gains occur for OCR and commonsense, and precision on appearance remains unchanged. This suggests DTPO learns to target informative inspectors rather than calling tools more often. DTPO improves OCR and commonsense invocation precision by over 20 percentage points each compared with SFT. DTPO uses 12.4% fewer calls per video than SFT while achieving equal or higher precision on every role.
The experiments evaluate a forensic video detection framework with six sub-agent roles, an agentic dataset called ForenCue-30K, and a training pipeline combining supervised fine-tuning with DTPO. Dataset comparisons show ForenCue-30K is the only resource that adds role, temporal, OCR/audio, trajectory, and agentic tool annotations, while role prevalence analysis finds appearance and physical artifacts most common and targeted invocation most useful for rare roles. WeVisForensics outperforms binary, generic MLLM, and chain-of-thought baselines across in-domain, out-of-domain, and AgenticCue evaluations, with DTPO producing consistent gains. DTPO also improves sub-agent invocation precision across roles while reducing tool calls per video, indicating the agent learns to select informative experts rather than over-call tools.