One-sentence Summary

Alibaba Group presents Wan-Streamer v0.2, a latency-preserving upgrade of their end-to-end audio-visual interaction model that increases the interactive stream resolution from $192 \times 336$192×336 to $640 \times 368$640×368 while maintaining $\sim 200\ \text{ms}$∼200 ms model-side signal-to-signal latency, achieved through a thinker-performer split where the performer employs Ulysses-style context parallelism for high-resolution latent video generation, keeping the compact boundary and enabling real-time, scene-grounded mid-shot agents with legible posture, gaze, hands, and local scene layout without additional delay.

Key Contributions

• Wan-Streamer v0.2 increases the interactive video output from 192×336 to 640×368 at 25 FPS while preserving approximately 200 ms model-side streaming latency, keeping the same native full-duplex formulation.
• A serving topology separates the model into a single-GPU thinker for latency-critical perception and language/state computation and a Ulysses-style context-parallel multi-GPU performer that absorbs the higher-resolution video generation cost through pre-sharded K/V caches and sequence parallelism for latent video denoising.
• The resolution upgrade expands usable visual interaction from close-up video calls to higher-fidelity close-ups and scene-grounded mid-shot agents, where posture, gaze, hands, nearby objects, and local scene layout remain legible during real-time conversation.

Introduction

Real-time audio-visual interaction demands systems that combine full-duplex speech, multimodal perception, and streaming video generation without breaking the natural dialogue cadence. Wan-Streamer v0.1 introduced a single-transformer causal timeline that jointly models user and agent text, audio, and video, but its 192p output stream forced a close-up framing restricted to facial responses, leaving body posture, objects, and scene context illegible for wider compositions. The authors address this by upgrading the visual output to 640×368 at 25 FPS while preserving approximately 200 ms model-side latency. They introduce a serving topology in which a single-GPU thinker handles the latency-critical perception and state update, while a Ulysses-style context-parallel multi-GPU performer absorbs the added high-resolution latent video denoising cost through pre-sharded K/V caches and sequence parallelism, expanding the interaction scope to scene-grounded mid-shot agents that remain visibly connected to their surroundings.

Method

The authors leverage a stable native-streaming formulation where the model operates as an end-to-end causal stream. User text, audio, and video observations continuously update a shared history that conditions the agent's text, speech, and video responses. Generated audio-video latents are committed back into the history after each unit, allowing subsequent responses to depend on both user behavior and the agent's previous expressions. To accommodate the increased computational cost of generating higher-resolution 640x368 video without degrading interactive latency, the deployed model is split into two distinct roles: a Thinker and a Performer.

The Thinker resides on a single GPU and is responsible for the compact, low-latency interaction path. It hosts the causal audio and video encoders, the token-causal Transformer path for language and state updates, KV-cache construction, and the causal decoders that transform returned latents into output audio and video. The language and state computations are reflected in the K/V cache that conditions the generation process.

The Performer consists of a Ulysses-style context-parallel GPU group dedicated to the computationally expensive flow-matching latent generation path. Performer ranks maintain pre-sharded K/V caches and split the long latent video sequence across ranks for denoising, communicating through Ulysses all-to-all and gather collectives around attention. Because audio latents are relatively short, sequence sharding would introduce unnecessary overhead, so they are generated without sequence sharding.

As illustrated in the framework diagram, at each streaming unit $k$k, the Thinker consumes current user observations and produces a new Performer-compatible K/V slice. Simultaneously, it receives the generated latents from the previous unit, decodes them, and emits the response. The Performer ranks receive the current K/V slice, update their local shards of the full-history cache, and execute Ulysses context-parallel denoising for the next unit. This scheduling effectively separates throughput from response latency. Real-time throughput is achieved by ensuring the Performer group processing time, K/V and latent transfer, and intra-Performer Ulysses communication fit within a 160 ms unit, while the model-side response latency remains approximately 200 ms. By concentrating the additional high-resolution workload in the context-parallel Performer, the Thinker remains a streamlined path for rapid interaction.

Experiment

The evaluation isolates the model-side latency and visual quality of the v0.2 640×368 video output under the same response boundary used in v0.1. Model-side signal-to-signal latency remains approximately 200 ms, keeping total remote interaction latency at roughly 550 ms with a fixed network budget. Qualitative inspection shows clearer close-up calls and more stable, scene-grounded mid-shot agents, with improved facial detail, gaze, and motion, all while maintaining low-latency streaming.

Wan-Streamer v0.2 raises output resolution from 192×336 to 640×368 while preserving the same 25 FPS frame rate, approximately 200 ms model-side latency, and roughly 550 ms total interaction latency under the unchanged 350 ms network budget. The performer component transitions from single-GPU latent generation to multi-GPU Ulysses-style context parallelism, with a revised communication protocol that broadcasts K/V slices and keeps the all-to-all/gather within the performer group. Qualitative observations indicate clearer close-up calls and more scene-grounded mid-shot agents in the higher-resolution streaming output. Resolution increases from 192×336 to 640×368 with no increase in model-side latency (~200 ms) or total interaction latency (~550 ms) at the same network budget. The performer adopts multi-GPU Ulysses-style context-parallel generation, and the communication flow is updated to broadcast performer-compatible K/V slices while the latent sequence exchange stays inside the performer group.

Wan-Streamer v0.2 evaluates a higher-resolution streaming pipeline while preserving the same interactive latency by shifting the performer from single-GPU to multi-GPU Ulysses-style context parallelism. The updated communication flow broadcasts performer-compatible K/V slices and confines the all-to-all latent exchange to the performer group. Qualitative observations indicate clearer close-up calls and more scene-grounded mid-shot agents in the enhanced output.