One-sentence Summary

Researchers from The Hong Kong University of Science and Technology (Guangzhou), Zhejiang University, and other institutions survey agentic artifact creation as stateful construction linking an operational artifact representation, a construction policy, and runtime verification whose feedback can redirect later actions, reviewing 259 works (230 systems and 29 benchmarks) across six artifact families to formulate principles for targeted repair and revalidation.

Key Contributions

• The paper defines agentic artifact creation as stateful construction in which an AI system materially constructs or revises a deliverable and intermediate observations redirect later work, then surveys 230 systems and 29 benchmarks across six artifact families.
• The survey compares six artifact families and analyzes application settings and evaluation practice as separate dimensions, finding that key construction challenges include tightly coupled decisions, failures that appear late or without a clear cause, and edits too coarse for local repair.
• Four design principles are formulated for control and accountability: externalizing commitments, defining control boundaries, linking feedback to feasible repairs, and revalidating affected state after changes, with scoped preference memory and authority-aware delegation as concrete mechanisms.

Introduction

Generative models now produce text, images, code, video, and other content from natural-language prompts, and these capabilities are entering professional and creative workflows where the deliverable must satisfy several interdependent acceptance criteria. Direct generation works well for bounded, easy-to-inspect outputs but is less reliable when decisions affect downstream constraints, failures appear late, and broad edits make local repair difficult. Prior surveys tend to organize work by model family, agent capability, domain, or process rather than by the delivered artifact itself. The authors address this gap by defining Agentic Artifact Creation as stateful construction in which observations of intermediate artifact states redirect later revision decisions. They survey more than 200 systems across six artifact families, propose a functional model based on operational representation, construction policy, and runtime verification, and synthesize principles, challenges, and a research agenda for inspectable construction control.

Dataset

Overview

The authors use a mapped full-publication corpus of artifact-construction systems. The selected excerpts describe a qualitative evidence base organized by artifact families: data visualizations, audio artifacts, spoken audio, and 3D assets. Numeric dataset sizes, training splits, and mixture ratios are not reported in the source text.

Sources and composition

• Data visualizations: drawn from AMACE, PlotGen, CoDA, Data Formulator 2, SVG editing workflows, and DV-World. The data consist of visual artifacts linked to transformation paths, chart specifications, and conversational revision state.
• Audio artifacts: comprises symbolic or rendered sound, with editable state across notation, lyrics, production graphs, stems, and waveforms.
• Spoken audio: no dedicated spoken-audio construction system appears in the mapped full-publication corpus; evidence comes from condition-decomposed text- or video-to-audio generation and podcast-production pipelines.
• 3D assets: split into visual assets and parametric models; systems include ShapeCraft, LL3M, InfiniHuman, SmartAvatar, Seek-CAD, CADDesigner, LLM-driven FreeCAD automation, STEP- and NURBS-based generators, CADIR, and ArtisanCAD.

Subset details and filtering rules

• Data visualizations: filtering and acceptance require effective communication and faithful visual marks to source data and recorded transformations. Processing preserves an executable transformation path, inspects the render, and keeps data transformation, chart specification, and conversational revision synchronized.
• Audio artifacts: no size or filtering details given. Validation checks use musical relations, source-script-speaker links, or scene-level event alignment.
• Spoken audio: no size or filtering details given. Material must keep source content, scripts, speakers, timing, and rendered segments aligned. AudioToolAgent routes audio question-answering and speech-to-text tools but does not maintain an editable audio artifact.
• 3D assets: visual assets require semantic parts, appearance controls, and edit handles to survive revision; parametric models preserve dimensions, features, constraints, and precise boundary geometry. Relevant processing includes procedural histories, progressive mesh construction, and cross-backend construction graphs with dependencies, constraints, topology, and diagnostics.

How the model uses the data

• The corpus is used as an evidence base for mapping artifact families and identifying where targeted repair is possible.
• No model-training split, mixture weighting, or dataset scale is described in these excerpts.
• The authors do not present these passages as a training dataset; they map published systems and their artifact-level processing links.

Cropping and metadata construction

• No image cropping strategy or numeric preprocessing pipeline is reported.
• Metadata is constructed around links between rendered output, encodings, transformations, and source data. These links determine whether revision can be targeted rather than falling back to whole-asset regeneration.

Method

The authors propose the Agentic Artifact Creation paradigm, which treats the delivered artifact as state that can be inspected and changed during production. Unlike direct generation pipelines where intermediate observations cannot redirect later actions, agentic creation couples a construction policy, an operational representation, and runtime verification so that feedback can redirect later decisions before acceptance.

At the functional level, the authors organize Agentic Artifact Creation into three roles that form a recurrent construction process.

The Operational Representation exposes the current artifact-side state $R_t$Rt​ through an Intermediate Form and provides an Edit Interface. The Intermediate Form can be an Artifact Instance, a Structured Model, or an Executable Program, determining what persists across steps. The Edit Interface specifies operations such as Unit Edit, Relation Edit, or Whole-Artifact Edit to modify the state. Applying an action $a_t$at​ yields the updated state $R_{t+1} = U(R_t, a_t)$Rt+1​=U(Rt​,at​).

The Construction Policy interprets the Task Specification $T$T, current representation $R_t$Rt​, and available feedback $f_t$ft​ to select the next action $a_t = \pi(T, R_t, f_t)$at​=π(T,Rt​,ft​). This policy is governed by Decision Control, which can be Workflow-based or Autonomous, and Agent Topology, which ranges from Single-Agent to Centralized or Decentralized Multi-Agent Systems.

Runtime Verification assesses observations $o_{t+1}$ot+1​ of the updated state against acceptance criteria to produce feedback $f_{t+1} = V(T, R_{t+1}, o_{t+1})$ft+1​=V(T,Rt+1​,ot+1​). It draws from Observation Sources such as Artifact State, Runtime Behavior, Evaluation Signals, or External Responses. The Feedback Function then derives Criterion Status, Failure Diagnosis, or Revision Guidance to inform the Construction Policy. If the state is accepted, the artifact is delivered; otherwise, the feedback drives revision.

This functional architecture supports three construction affordances: Composability, which allows steps to be replaced or reordered; Traceability, which links requirements to actions and observations; and Revisability, which provides intervention points for targeted repair.

The authors apply this framework across a landscape of six artifact families: textual, 2D visual, audio, video, spatial, and behavioral. Each family exhibits distinct failure-observation points, ranging from static inspection to temporal playback and runtime consequences.

These families are further organized into 16 analytical profiles, categorizing specific artifact types such as creative writing, data visualizations, music, narrative videos, 3D assets, and software systems.

To guide system design across these diverse contexts, the authors formulate four principles of agentic creation, using decision interdependence, failure observability, and repairability as a qualitative lens.

First, Externalize Commitments requires connecting acceptance criteria to addressable artifact state so that consequential requirements remain available to later actions. Second, Define Control Boundaries specifies delegation and review boundaries at points where artifact dependencies change, ensuring responsibility for cross-boundary consistency. Third, Make Feedback Actionable connects relevant acceptance criteria to timely evidence, diagnosis, and feasible repair actions. Finally, Revalidate Affected State treats acceptance evidence as version-scoped, invalidating dependent evidence when changes occur and selectively revalidating the affected state. Together, these principles define inspectable control relations that support controlled goal realization with proportionate control cost.

Experiment

The reviewed experiments span visual document construction, engineering design, and evaluation benchmarking. Visual-document studies contrast staged, unified, and trajectory-aware systems for posters and presentations, showing that repairing page-level issues can break source grounding or deck-level narrative commitments. Engineering-design experiments distinguish deployable system development, where simulation provides feedback, from simulation modeling, where the transition model or simulator is the delivered artifact, and they highlight alignment across multiple accepted deliverables. Evaluation and benchmark analyses characterize artifact, trajectory, and system-level targets; evidence channels and evaluators; protocol conditions; and benchmark landscapes, finding that single aggregate scores hide task-specific and criterion-specific failures while open-ended artifacts require versioned acceptance specifications and budget-matched comparisons.

Adjacent surveys organize mainly around model families, agent architectures, creative workflows, software agents, tool trajectories, human collaboration, or evaluation methods rather than the delivered artifact. Their artifact scope spans multimodal content, software, general tasks, and digital tasks, while construction state and feedback control are mostly partial or absent. Evaluation emphasis varies from artifact-level and task-level to trajectory-level and interaction-level, with only evaluation-focused work covering multiple levels. The delivered artifact is not the primary unit of analysis in these surveys; their primary lenses include model families, agent architectures, creative workflows, software agents, tool trajectories, human collaboration, and evaluation methods. Construction state and feedback control are mostly partial or absent as organizing axes, and evaluation emphasis varies from artifact-level and task-level to trajectory-level and interaction-level, with only evaluation-focused work spanning multiple levels.

Artifact families are distinguished by which dependency regimes and observation modes dominate their acceptance criteria, with secondary marks indicating subtype-level involvement. The signatures move from reading-oriented textual and rendering-oriented 2D visual artifacts, through playback-oriented audio and video, toward persistent spatial structures and state-dependent behavioral artifacts where dynamic and interaction criteria become more prominent. Evaluation evidence is uneven: dedicated benchmarks are most developed for 2D visual and behavioral artifacts, and coverage is stronger for delivered artifacts and bounded executions than for trajectories or validity beyond tested cases. Textual and 2D visual artifacts are characterized by semantic or perceptual dependencies with reading or rendering as primary observation modes. Audio and video artifacts add temporal and playback-oriented criteria, while spatial and behavioral artifacts emphasize spatial or dynamic state and interaction. Dedicated benchmarks are most developed for 2D visual and behavioral artifacts, whereas audio, video, and spatial work more often relies on protocols from system papers.

The representative systems primarily cover textual artifact creation, with one 2D visual entry. Textual systems use varied maintained intermediate forms and mostly workflow-based construction policies. Runtime verification and agent topology differ more widely across the listed systems. Listed textual systems maintain varied intermediate forms such as outlines, manuscripts, drafts, character graphs, and LaTeX state. Workflow-based construction policy is common across the set, with one textual system also marked as autonomous. Runtime verification details range from unreported to combinations of status, diagnosis, guidance, state, metric, and response observations. The single 2D visual representative uses a data plus specification intermediate form rendered to a chart under a single-agent workflow.

The representative textual and 2D visual protocols consistently evaluate the delivered artifact, while trajectory evidence is uneven and often limited or absent. Among the displayed protocols, only one reports an agentic system property beyond task capability, specifically controllability. LM judges and human reviewers appear more frequently as evaluation channels than rule-based checks or specialist models. All selected protocols explicitly cover the artifact itself, while trajectory coverage ranges from no reported coverage to limited or explicit. Only one protocol in this subset evaluates a system property beyond task capability, reporting controllability. LM judges and human reviewers are more common evaluation channels than rule-based checks or specialist models across these protocols.

The analyses indicate that adjacent surveys mostly organize around model families, agent architectures, workflows, and evaluation methods rather than the delivered artifact, with construction state and feedback control only partially covered. Artifact families differ by whether acceptance relies on semantic or perceptual reading, temporal playback, or spatial and state-dependent interaction, and dedicated benchmarks are most developed for 2D visual and behavioral artifacts while audio, video, and spatial work often depend on system protocols. Representative systems are predominantly textual, maintain varied intermediate forms such as outlines or drafts, and use workflow-based construction, though runtime verification and agent topology vary considerably. Evaluation protocols consistently assess the final artifact, cover trajectory evidence unevenly, rarely report agentic properties beyond task capability, and rely more on LM judges and human reviewers than on rule-based or specialist checks.