One-sentence Summary

Researchers from Fudan University et al. propose Agentopia, a long-term multi-agent life simulation framework in which 100 agents autonomously pursue personal growth, relationships, and goals over 10 simulated years, and by defining a life reward and training LLMs with rejection sampling, the framework elicits emergent social behaviors and improves role-playing benchmarks by 15.6%.

Key Contributions

• Agentopia is a long-term multi-agent life simulation framework in which 100 agents autonomously pursue personal growth, develop social relationships, and fulfill needs and goals over 10 simulated years.
• A life reward signal models agent well-being, and rejection sampling over high-advantage trajectories uses this reward to train the underlying LLM.
• Experiments across three diverse worlds show emergent social behaviors and improved in-simulation well-being; life reward training also improves downstream role-playing, yielding a 15.6% overall improvement on the CoSER Test with the largest gains in anthropomorphism (+23.7%) and character fidelity (+16.4%).

Introduction

The authors address LLM-based persona simulation and role-playing, which matters for AI companions, digital games, and content creation but still struggles to align models with human cognition and behavior. Prior work has largely simulated agent societies for only days or within single conversations, often focusing on low-level physical actions rather than long-term social dynamics, and it depends heavily on costly human data for optimization. To overcome these limits, the authors introduce Agentopia, a multi-agent framework that supports long-term life simulation across simulated years, defines a life reward to model human well-being, and uses that reward for data-free training that improves both in-simulation social outcomes and downstream role-playing ability.

Method

Agentopia: Framework and Optimization Method

The authors introduce Agentopia, a unified framework for long-term life simulation in agent societies. The design organizes simulation around three core concepts: the simulation procedure, the agent, and a separate environment model. Instead of encoding social dynamics through hard-coded rules, Agentopia uses large language models for both character behavior and environmental feedback. The week is the basic simulation unit. A simulated year contains $n_w$nw​ weeks, and each week consists of four stages: Plan, Contact, Activity, and Review. At the end of each simulated year, the framework performs profile updates, position applications, and life reward calculation.

Agent Design

Each agent role-plays a specific persona consisting of a profile, social relationships, and dynamic states. The profile stores relatively stable identity information such as background story, personality traits, talents, initial position, and assets. These attributes are updated once per simulated year. Social relationships are not represented as explicit graph structures or relationship labels. Instead, they are maintained as inter-character memory. Each agent stores free-text memories about other agents, which the agent can update autonomously. This design unifies friends, rivals, strangers, and other relationship types through the same memory mechanism. Dynamic states include vitality, fulfillment, skills, position, and assets. Fulfillment is grounded in Maslow’s hierarchy of needs and captures satisfaction across mood, material, social, and esteem dimensions. It decays naturally each week following hedonic adaptation theory. Changes to fulfillment and vitality are determined by the environment model based on activity content.

Agentopia also includes a context management mechanism with three layers. The roleplay prompt contains the agent’s persona, recent weekly diaries as short-term memory, summaries of long-term memory files, worldview rules, and role-playing principles. The stage prompt adds stage-specific instructions and context. Message history records the current multi-turn messages, function call results, and compact reasoning within the current stage.

For long-term memory, agents maintain a file-system-based store. Each agent manages three categories of files: general.txt for personal notes and plans, characters/<who>.txt for knowledge about specific people, and others/<name>.txt for other topics. Agents use read_file, update_file, and list_files function calls to manage memory. A read-before-write constraint ensures that an update is allowed only after the agent has read the target file in the same invocation.

Simulation Procedure and Scheduling

The simulation is turn-based because LLM agents generate one response at a time. Agentopia abstracts away movement and object manipulation and focuses on social interaction, planning, and decision-making. In the Plan stage, each agent creates a weekly plan based on memory and current state, reviews previous plans through memory file calls, and selects a consumption level for the week.

In the Contact stage, agents communicate over multiple rounds to arrange shared schedules. Communication is pairwise. Each week contains $n_c$nc​ contact rounds. Agents receive newly arrived messages and contact history from the past three weeks, then decide whom to contact and what action to generate. Agent actions are wrapped in <role_action> tags and include contact, propose_joint_activity, respond_invitation, and cancel_joint_activity. After all contact rounds, the system resolves the collected actions to determine which joint activities are successfully created. Public activities are scheduled before Contact, when the environment model creates public events and agents sign up for events matching their interests. Encounter activities are arranged after Contact for agents who remain idle on a given day.

In the Activity stage, agents carry out one activity per active day, with $n_d$nd​ active days per weekly cycle. Activities are joint, solo, encounter, or public. In the Review stage, agents reflect on weekly experiences and summarize them into a weekly diary. They may also update memory files through function calls. Weekly diaries are included in later context so agents can recall past experiences.

Activity Execution

Joint activities model multi-agent, multi-turn interactions. They are created through invitations and negotiation during Contact. The environment model provides environmental descriptions and selects the next speaker each turn. Agents can mark generated content as public, private, or selective, where selective content is visible only to specified persons. Agents can also send gifts or exit early. The environment model applies response filtering to evaluate whether each response respects roleplay principles covering anthropomorphism, character fidelity, and feasibility.

Solo activities are the default when an agent has no other schedule. They follow a single-turn format in which the agent describes an intended action and the environment model evaluates feasibility based on the character’s background and returns an outcome. Solo activities also support spending, allowing agents to purchase goods or services and gain material fulfillment.

Encounter activities are chance meetings created by the environment model for idle agents. They follow the same multi-turn dialogue format as joint activities but do not appear in agent schedules. Encounters can introduce strangers or bring together agents with meaningful plot connections. Public activities are open community events centered on shared interests. They follow a flow similar to solo activities, but at the end participants see what other participants did and may form new social connections.

Environment Model and Supporting Systems

The environment model is a stateless LLM that acts as a generative environment engine. It provides feedback during activities, judges feasibility, evaluates outcomes, selects next speakers in joint activities, generates public and encounter activities, ranks candidates for position applications, updates profiles at year end, and performs response filtering. This approach replaces extensive hard-coded rules with learned, generative environment behavior.

The economy system gives agents income from position-based weekly salaries, character-specific weekly income, and additional work during activities. Expenses come from a weekly living standard selection and from consumption activities. The position system assigns each agent a job or social role that provides income and skill growth. A yearly position application process lets agents apply for new positions, with outcomes determined by the environment model based on ability and position requirements. The location system gives agents grounded environmental perception. The environment model creates locations for each world, and joint and encounter activities must specify a location.

Life Reward

At the end of each simulated year, Agentopia computes a life reward for every agent based on three dimensions: social, subjective, and economic. These rewards are externally determined rather than self-reported.

Social reward measures an agent’s standing based on perception from other agents. Following the Warmth-Competence model, each agent privately scores every person in its social circle on affection and respect using a 0 to 100 scale. Scores are rank-rescaled to eliminate individual scoring scale differences. They form two weighted directed graphs, one for affection and one for respect. Weighted PageRank is applied on each graph to compute initial social standing scores. A mutual affection bonus is then added. The final standing score for agent $i$i is

where $S_j$Sj​ is agent $j$j’s raw PageRank score, $\mathcal N_{in}(i)$Nin​(i) is the set of agents who know $i$i, $w_{ji}$wji​ is the normalized edge weight from $j$j to $i$i, and $\alpha$α is the mutual affection coefficient. This mechanism amplifies the contribution of reciprocated relationships. The social reward is the average of the affection and respect scores:

Subjective reward is computed from an agent’s fulfillment history across four dimensions. A penalty mechanism penalizes agents whose fulfillment or vitality falls below the 25th percentile threshold across all agents in the same week. The subjective reward is

where $D$D is the number of fulfillment dimensions, $f_{w,d}$fw,d​ is the fulfillment value for week $w$w and dimension $d$d, $n_p$np​ is the total number of penalty instances over the year, and $\lambda_p$λp​ is the penalty weight.

Economic reward is the net change in an agent’s deposit over the year:

The total reward applies z-score normalization to each dimension and combines them as

Life Reward Training

The authors propose a rejection-sampling-based training method because the long-horizon nature of life simulation makes end-to-end methods such as PPO impractical. Each agent trajectory involves hundreds of LLM calls per simulated year, and a full simulation spans multiple years. For each agent $i$i at simulated year $t$t, the return is

where $\gamma$γ is the discount factor and $T$T is the final time step. Because each agent produces only one life trajectory, learning a critic or averaging over multiple rollouts is difficult. Agentopia therefore uses each agent’s own previous return as a self-referential baseline. After normalizing returns across time steps, the advantage is defined as

This advantage measures whether an agent’s expected future life reward improves relative to its own past. Within each reward period, the top 25% of agents by advantage are selected, and all trajectories from those agents within the period are included as training data. Because selection is based on self-improvement rather than absolute performance, the method avoids favoring agents with inherently better initial conditions.

Response filtering removes agent responses with malformed actions or invalid parameters. Responses in joint and encounter activities are also checked against roleplay principles, and violating responses are excluded. To mitigate catastrophic forgetting, the approach uses self-distillation. Model responses to general-purpose instructions from the Tulu 3 dataset are mixed with Agentopia trajectories at a 50:50 ratio measured by output tokens.

Experiment

The experiments simulate 100 agents across three distinct social worlds for ten years using Qwen3.5-397B-A17B as the primary model. Long-term analysis shows that subjective and economy rewards improve on average while social reward remains stable, and the reward dimensions align with expected behavioral drivers. Fine-tuning through life rewards improves economy and subjective rewards, social recognition, several fulfillment dimensions, and general role-playing ability, though it also shifts agents toward rewarded behaviors while deprioritizing unrewarded ones. Computational cost analysis reveals that growing memory and input tokens become the main bottleneck in long-horizon social simulation.

Agentopia extends prior agent society systems by supporting simulations that run for years, whereas earlier listed systems operate on day or week scales. It pairs free-form actions and LLM-generated environmental feedback with file-based memory, skill growth, and economic mechanisms. Most prior systems instead rely on predefined actions, rule-based feedback, and retrieval memory. Agentopia is the only listed system with a years-long time scale, while others are limited to days or weeks. Agentopia and BookWorld are the only systems with free-form action spaces and LLM-generated environmental feedback. Agentopia combines file-based memory, skill growth, and an economic system, a combination not shared by the earlier systems listed.

The heatmap indicates that token usage and contact-related interaction metrics have the strongest positive associations with particular reward dimensions, while function call counts remain weakly related across settings. These associations vary by environment, with token and contact correlations generally stronger in The Campus than in Arcane Academy. In combined averages, token volume and contact counts align most clearly with the same reward dimension. In combined averages, input and output token volumes and active and passive contact counts all show their strongest positive correlation with the same reward dimension. Function call count remains near zero across all reward dimensions and worlds, while token and contact correlations are generally stronger in The Campus than in Arcane Academy.

Life reward training improved economy and subjective well-being in cross-world social simulations, and agents gained broader peer recognition. Fulfillment improved for mood, social, and esteem needs, but material fulfillment declined and activity patterns shifted toward public participation and away from solo activities and skill advancement. Economy and subjective rewards rose, while material fulfillment dropped as saving increased relative to spending. Agents were respected and liked by more peers, with a larger relative gain in respect than liking. Public activity participation increased, whereas solo activities and skill advances declined substantially.

Qwen3.5-397B-Agentopia shows clear gains over its Qwen3.5-397B base model, with the largest relative improvements in anthropomorphism and character fidelity. Its average role-playing score also exceeds Claude-4.5-Sonnet, though it still trails Claude-4.5-Opus and Gemini-3-Pro. The results indicate that life reward training transfers to stronger general anthropomorphism and role-playing ability. Claude-4.5-Opus records the highest average role-playing score, with Gemini-3-Pro close behind. Qwen3.5-397B-Agentopia surpasses Claude-4.5-Sonnet on average, led by higher anthropomorphism and storyline quality. Compared with the base Qwen3.5-397B, the trained model improves most in anthropomorphism and character fidelity.

Simulating 100 agents for 10 years across three worlds consumes substantial computational resources, with input tokens dominating overall usage. Memory and persona context drive high input demand, and runtime grows steadily as agents accumulate memory over time. This makes context and memory management a central bottleneck for long-horizon social simulation. Input tokens dominate token consumption by a wide margin across all three worlds. The Campus world has the highest overall token cost and longest runtime, while The Apartment has the highest number of LLM calls. Per-week runtime increases substantially over the simulated decade as agent memory and context grow.

The experiments evaluate Agentopia as a long-horizon social simulation platform, comparing it with prior systems and examining behavioral correlations, life reward training effects, role-playing quality, and computational costs. Token usage and contact interactions are most strongly associated with reward outcomes, especially in The Campus, while function calls remain weakly related. Life reward training improves economy, subjective well-being, peer recognition, and public participation but lowers material fulfillment and reduces solo or skill-focused activity. The trained Qwen model gains notably in anthropomorphism and character fidelity, and long simulations show that memory and persona context make input tokens and growing runtime the main scalability bottleneck.