One-sentence Summary

The Qwen Team presents Qwen-Music, a music generation model that integrates a 25 Hz single‑codebook tokenizer, a Melody‑CoT‑augmented autoregressive LLM, and a generative stereo renderer, trained on more than 5 million hours of multilingual data with quality‑aware pre‑training and progressive post‑training, achieving state‑of‑the‑art results on 13 of 16 objective metrics and outperforming commercial systems such as Suno V5.5, Suno V5, and MiniMax Cover in both text‑to‑music and cover song generation while preserving reference melodies more accurately.

Key Contributions

• Qwen-Music is a unified music generation system that combines a compact 25 Hz single-codebook Music Semantic Tokenizer, an autoregressive LLM with a melody-aware chain-of-thought (Melody-CoT) mechanism, and a generative stereo renderer, enabling both text-to-music and cover song generation.
• The training pipeline pre-trains on over 5 million hours of multilingual music data using a quality-graded curriculum, then applies a progressive post-training stage with supervised initialization, offline DPO, and online GSPO to improve musicality, controllability, and instruction following.
• Extensive evaluations show that Qwen-Music achieves state-of-the-art results on 13 of 16 objective text-to-music metrics, wins professional human preference tests against leading proprietary systems (e.g., 59.1% vs. MiniMax Music 2.5+), and preserves reference melodies more accurately than Suno V5.5, Suno V5, and MiniMax Cover in cover song generation.

Introduction

The authors address the challenge of generating full songs that are musically coherent, lyrically intelligible, and acoustically realistic, a task that requires joint modeling of lyrics, melody, vocal performance, and instrumentation over long time spans. Prior work struggles with the mismatch between high-level semantic planning and low-level waveform rendering: discrete-token language models can handle long-range composition but lose fine acoustic detail, while cover generation adds the extra constraint of preserving a reference melody under new stylistic conditions. The authors’ main contribution is Qwen-Music, a unified system that decouples song generation into a semantic composition stage and a generative acoustic rendering stage. It compresses music into a compact 25 Hz single-codebook token stream, uses an explicit melody-planning mechanism with a Melody-CoT approach, and employs a quality-graded training curriculum with multi-stage preference alignment to enable controllable text-to-music and reference-melody-based cover song generation.

Dataset

The authors use a large-scale collection of raw music tracks from diverse sources to train Qwen-Music-Render. Before training, every file passes through an acoustic quality pipeline that identifies and removes low-fidelity audio. The pipeline is detailed in the paper and summarized below.

• Raw collection: The initial dataset consists of music files in lossless containers, but many are transcoded from MP3, duplicated mono, clipped, or upsampled with empty high-frequency content. No specific source list or total size is given; the pipeline is applied to the entire raw pool.

• Quality filtering pipeline: Noise-floor estimation: The first and last 10% of the mono waveform are used as edge segments. Hann-windowed 8192-point RFFTs are computed over non-overlapping windows, peak-normalized, and averaged. The empirical noise floor η̂ is the 20 log₁₀ of the median magnitude of the top 10% bins. Cutoff detection: Five equally spaced interior windows (skipping near-silent regions) are processed similarly. The cutoff frequency f_c is the highest FFT bin whose mean magnitude stays above −80 dB relative to the peak. Drop measurement: A local energy drop Δ_dB is measured across ±2 kHz around f_c. If no reliable content exists above f_c, the post-cutoff maximum is replaced by η̂. Labels: Files are labeled hard_cutoff when f_c/f_Nyquist < 0.85 and Δ_dB ≥ 40 dB (or ≥ 25 dB near η̂). They are labeled suspicious when the ratio is < 0.90 with Δ_dB ≥ 20 dB. All others are natural.

• Training usage: Only files that pass the quality check (i.e., are labeled natural) are kept for training the Qwen-Music-Render model. The filtering ensures that the renderer learns from genuine high-fidelity music, avoiding artifacts from lossy transcodes, upsampling, or clipping. The paper does not mention further cropping, metadata construction, or mixture ratios for this subset.

Method

The authors leverage an inference pipeline that transforms a user request into a complete high-fidelity stereo song. Given a natural-language description, the system first rewrites it into a structured textual condition containing musical tags and generated lyrics. Conditioned on this rewritten text, Qwen-Music-LLM autoregressively predicts Music Semantic Tokens, optionally incorporating reference melody tokens via Melody-CoT conditioning for cover song generation. Qwen-Music-Render then takes both the rewritten textual condition and the generated Music Semantic Tokens as input and performs generative rendering to produce high-fidelity 48 kHz stereo waveforms.

As shown in the figure below:

This pipeline is supported by three core components: Qwen-Music-Tokenizer, Qwen-Music-LLM, and Qwen-Music-Render.

Qwen-Music-Tokenizer is a low-bitrate discrete tokenizer that maps a raw music waveform into a single stream of 25 Hz Music Semantic Tokens. It uses a single backbone consisting of a convolutional frontend followed by a Conformer encoder. The authors train the backbone in four stages: bidirectional self-supervised pretraining with BestRQ, causal adaptation of the pretrained encoder, multi-task supervised fine-tuning with lyric and spectral targets, and VQ tokenizer training.

As illustrated in the figure below:

In the first stage, the encoder is trained bidirectionally with a masked-prediction objective. The loss is defined as: $\mathcal{L}_{\mathrm{BestRQ}} = \frac{1}{|\mathcal{M}|} \sum_{t \in \mathcal{M}} \mathrm{CE}(p(h_t), q(x_t))$LBestRQ​=∣M∣1​∑t∈M​CE(p(ht​),q(xt​)) where $\mathcal{M}$M is the set of masked frames, $h_t$ht​ is the encoder output at frame $t$t, $q(\cdot)$q(⋅) is the frozen random-projection quantizer, and $p(\cdot)$p(⋅) is the prediction head. In the third stage, multi-task supervision is applied using a combined objective: $\mathcal{L}_{\mathrm{SFT}} = \lambda_{\mathrm{ctc}} \mathcal{L}_{\mathrm{CTC}} + \lambda_{\mathrm{mel}} \mathcal{L}_{\mathrm{spec}}^{\mathrm{mel}} + \lambda_{\mathrm{chr}} \mathcal{L}_{\mathrm{spec}}^{\text{chroma}}$LSFT​=λctc​LCTC​+λmel​Lspecmel​+λchr​Lspecchroma​ The final stage inserts a single VQ codebook of 32768 entries under a cosine metric, trained with a straight-through estimator and a commitment loss.

Qwen-Music-LLM adopts an autoregressive backbone initialized from a 3B dense variant of Qwen3.5-Omni. To address the challenge of text-to-music generation where text prompts typically specify style and lyrics only at a high level, the authors introduce Melody-CoT, a melody-token-based chain-of-thought for explicit melody planning. Before generating full-mixture Music Semantic Tokens, the model is trained to produce an intermediate sequence of melody tokens that describes a coarse-grained vocal melody contour.

The Melody Tokenizer converts a reference vocal pitch contour into a compact sequence of discrete melody tokens. The pitch curve is downsampled by a factor of 8 with median pooling: $\tilde{\mathbf{f}} = \text{MedianPool}_8(\mathbf{f})$f~=MedianPool8​(f) To prevent the melody condition from leaking absolute pitch range or singer-dependent timbre information, the authors use a relative MIDI representation. The melody token $z_i$zi​ is computed as: $\bar{m} = \text{median}\{\text{round}(\text{hz2midi}(\tilde{f}_i)) \mid \tilde{f}_i > 0\}$mˉ=median{round(hz2midi(f~​i​))∣f~​i​>0} $z_i = \begin{cases} \operatorname{clip}(\text{round}(\mathrm{hz2midi}(\tilde{f}_i)) - \bar{m}, -127, 127) + 127, & \tilde{f}_i > 0, \\ 255, & \tilde{f}_i = 0, \end{cases}$zi​={clip(round(hz2midi(f~​i​))−mˉ,−127,127)+127,255,​f~​i​>0,f~​i​=0,​

Qwen-Music-Render is a three-stage neural render that turns discrete output into a high-fidelity waveform. The first stage is a Diffusion Transformer that produces a continuous latent through conditional flow matching. The second stage is Spec-VAE, whose Spec Decoder inverts the latent sequence back to a coarse complex STFT. The third stage is a Band-Mode Refiner that predicts frequency-adaptive magnitude and phase residuals.

To improve reconstruction quality, the authors introduce Spec-SnakeBeta, a frequency-aware extension of the SnakeBeta activation. While the original SnakeBeta uses channel-wise learnable parameters, Spec-SnakeBeta parameterizes $\alpha$α and $\beta$β along the frequency axis: $\text{Spec-SnakeBeta}(\mathbf{x})_{b,c,t,f} = x_{b,c,t,f} + \frac{1}{\exp(\beta_f) + \epsilon} \sin^2(x_{b,c,t,f} \cdot \exp(\alpha_f))$Spec-SnakeBeta(x)b,c,t,f​=xb,c,t,f​+exp(βf​)+ϵ1​sin2(xb,c,t,f​⋅exp(αf​)) The parameter $\alpha_f$αf​ is initialized with a log-frequency prior: $\alpha_f = \log\left(\frac{\mathrm{freq}_f}{\bar{f}} + \epsilon\right), \quad \bar{f} = \frac{1}{F} \sum_f \mathrm{freq}_f$αf​=log(fˉ​freqf​​+ϵ),fˉ​=F1​∑f​freqf​

As shown in the figure below:

This frequency-dependent parameterization provides the activation prior used by the Spec-VAE decoder.

The authors train Qwen-Music-Render only on files whose acoustic evidence matches a genuine high-fidelity music target. They apply an automated pipeline to filter out lossy-transcoded or upsampled files. The key idea is to estimate the noise floor from track edges and detect the spectral cutoff from interior segments. The empirical noise floor is estimated as: $\hat{\eta} = 20 \log_{10}(\text{median}(\bar{\mathbf{M}}_{\text{edge}}[\text{top } 10\% \text{ bins}]))$η^​=20log10​(median(Mˉedge​[top 10% bins])) The local energy drop across a $\pm 2$±2 kHz band around the cutoff $f_c$fc​ is measured as: $\Delta_{\mathrm{dB}} = \max \bar{\mathbf{M}}_{\mathrm{int}}[f_c - 2\mathrm{kHz}: f_c] - \max \bar{\mathbf{M}}_{\mathrm{int}}[f_c: f_c + 2\mathrm{kHz}]$ΔdB​=maxMˉint​[fc​−2kHz:fc​]−maxMˉint​[fc​:fc​+2kHz]

As shown in the figure below:

The detector flags files as hard cutoff or suspicious based on the cutoff ratio and the drop magnitude, ensuring only high-quality data is used for renderer training.

The authors adopt a quality-graded pre-training curriculum for the backbone LLM. They organize the corpus into quality levels and schedule them progressively. Stage 1 trains on lower-quality data to learn a general mapping from text conditions to music semantic tokens. Stage 2 trains on mid-quality data with a learning-rate decay schedule for quality consolidation. Stage 3 focuses on high-quality data for refinement, improving musicality, controllability, and instruction following.

After pre-training, the authors further align Qwen-Music-LLM with human musical preferences through a multi-stage post-training pipeline. Phase I performs supervised fine-tuning on curated high-quality data to establish a clean generation prior. Phase II performs iterative offline alignment with Direct Preference Optimization, progressively improving musicality and controllability. Phase III applies on-policy Group Sequence Policy Optimization for fine-grained musicality and audio-quality improvement.

Experiment

The evaluation setup covers text-to-music and cover song generation, assessed through blind A/B human preference tests, objective metrics, and genre-wise Bradley-Terry analysis, with Qwen-Music outperforming or matching leading commercial systems like Suno V5.5 across diverse musical styles. It validates the model's strong balance of musicality, controllability, lyric intelligibility, and melody preservation, while rendering ablations confirm that rewritten textual conditioning with text-drop classifier-free guidance and a Band-Mode Refiner improve audio fidelity and stereo reconstruction. Overall, Qwen-Music combines robust generation quality with high-fidelity rendering, showing competitive performance across a wide range of tasks and genres.

On the Artificial Analysis Music with Vocals Leaderboard, Qwen-Music (entered as JazzCat) achieved third place, trailing only Suno V5.5 and Mureka V8. It outperformed several strong competitors including Suno V5 and multiple MiniMax models, despite having fewer evaluation samples. JazzCat ranked third with an Elo score just behind Mureka V8 and Suno V5.5. JazzCat's Elo exceeds that of Suno V5 and MiniMax Music 2.5+ and 2.6, placing it among the top English vocal music generation systems.

The four-stage recipe progressively builds a discrete music tokenizer by first learning continuous representations via bidirectional masked prediction on 30-second crops, then adapting to causal attention, followed by multi-task supervised fine-tuning on full songs, and finally training a vector-quantized bottleneck. All stages use the same 0.6B-parameter Conformer backbone and are initialized from the previous checkpoint, except the first stage which starts randomly. Training advances from bidirectional BestRQ pretraining to causal adaptation, then to supervised lyric and spectral fine-tuning, and finally to VQ tokenizer training. Audio length increases from 30-second crops in the first two stages to full songs up to 300 seconds in the final two stages. The backbone is a 24-layer Conformer with 1024 width, 16 attention heads, and a 25 Hz latent rate, shared across all stages.

The training configuration evolves across three stages: reconstruction-only pretraining, adversarial fine-tuning with a waveform discriminator, and refiner training with frozen encoder-decoder using separate waveform and spectral discriminators. Optimizer choice shifts from Muon for the generator in early stages to AdamW for the refiner, while generator learning rate increases and batch size drops to 1. Discriminator learning rates are low, and only the spectral discriminator receives a warmup period. Stage 3 freezes the encoder-decoder and trains only the refiner, switching the generator optimizer to AdamW and using separate discriminators for waveform and spectral domains. The waveform discriminator learning rate is set to 10^-6, two orders of magnitude lower than the generator’s 1.5×10^-4, while the spectral discriminator uses 10^-5 and includes a 15k-step warmup. Generator learning rate is raised from 10^-4 in stage 1 to 1.5×10^-4 in stages 2 and 3, and per-GPU batch size is reduced from 4 to 1 in the later stages.

An automated quality gate inspects music files for encoder metadata, bitrate sufficiency, loudness compliance, genuine stereo separation, and genuine high-frequency content. The frequency cutoff module estimates noise floor from track edges independently of the spectral cutoff from interior segments, enabling reliable detection of upsampled or lossy-transcoded files. Only tracks that pass all rule-engine checks are admitted for renderer training. Metadata and encoder checks flag files carrying conversion tool signatures such as lavf, lavc, ffmpeg, or lame, or explicit transcode/convert tags. Bitrate quality is assessed through multiple criteria, including monotonic presentation timestamps, realized-to-nominal bitrate ratio, complex-segment mean bitrate, and coefficient of variation, with a tiered quality label from HiRes to Trash. Loudness is constrained to broadcast-standard ranges: integrated loudness between -25 and -5 LUFS, true peak between -1 and +2 dBFS, loudness range 3–15 LU, and peak-to-loudness ratio at least 6 LU. Fake stereo duplicates are rejected when the fraction of zero absolute difference between left and right channels exceeds 0.99. Frequency cutoff detection uses noise-floor estimates from the first and last 10% of the waveform to avoid silent-edge bias, then requires a hard energy gap of at least 40 dB around the cutoff, with suspicious cases flagged at 20 dB, and demands at least one valid interior segment where the cutoff ratio relative to Nyquist ranks above 0.5.

Across seven genres, Qwen-Music achieves the highest Bradley–Terry rating in four genres (Electronic/EDM, Jazz & Blues, Punk & Hardcore, and R&B/Soul), indicating strong listener preference in diverse styles. It remains competitive with Suno V5.5, ranking first or second in five genres, while Mureka V8 leads in Metal & Hard Rock and Suno V5.5 leads in Hip-Hop/Rap. These within-genre comparisons show that Qwen-Music is broadly preferred but not universally dominant. Qwen-Music is rated best in Electronic/EDM, Jazz & Blues, Punk & Hardcore, and R&B/Soul. In Hip-Hop/Rap, Suno V5.5 leads, and in Metal & Hard Rock, Mureka V8 leads, while Qwen-Music remains competitive (second in Pop and Hip-Hop/Rap).

Qwen-Music is evaluated as a top English vocal music generation system, ranking third on the Artificial Analysis leaderboard despite fewer evaluation samples. The model's tokenizer is built through a four-stage recipe that progressively moves from bidirectional pretraining to vector quantization on a Conformer backbone, while the renderer training advances through reconstruction, adversarial, and refiner stages with carefully tuned optimizers and discriminators. A rigorous automated quality gate filters training data by rejecting tracks with metadata issues, low bitrate, mono duplication, or missing high frequencies, ensuring only genuine high-fidelity stereo files are used. Across seven genres, listener preference tests show Qwen-Music achieves the highest rating in four genres and remains competitive overall, though it is not universally dominant.