Online Tutorial | CSM Is Here, Get out of the Way! More Vivid Speech Generation, Say Goodbye to Delayed, Dull and Mechanical Sounds
When chatting with AI voice assistants, I always feel something is wrong. They answer questions freely, but they lack a bit of "human touch". The tone is dull, the pauses are abrupt, and they occasionally freeze in inexplicable places. This sense of disharmony between human and non-human is actually the "uncanny valley effect". When the AI voice is highly similar to human voice but not perfectly consistent, users will feel uncomfortable.
Recently, the speech generation model CSM (Conversational Speech Model) launched by the Sesame team has stood out among many speech models.The model uses the Llama backbone architecture and lightweight audio decoder, combined with an end-to-end Transformer framework, to generate RVQ audio codes based on text and audio inputs, and then output fluent, natural, and emotional speech.Create a voice assistant that can meet users’ emotional needs.
Compared to traditional AI speech generation models, CSM does much more than simply generate audio:
*Stronger emotional understanding:Able to deeply analyze the context and flexibly adjust the tone and intonation.
*More natural conversation rhythm:Fine-tune details such as pauses, emphasis, interruptions, etc. to make conversations smoother.
*Almost zero-delay experience:The efficient inference architecture makes speech generation closer to real time and improves interaction efficiency.
The "CSM Conversational Speech Generation Model Demo" tutorial is now available on the HyperAI official website. Come and check it out!
Tutorial address:
Demo Run
1. Log in to hyper.ai, on the Tutorial page, select CSM Conversational Speech Generation Model Demo, and click Run this tutorial online.
2. After the page jumps, click "Clone" in the upper right corner to clone the tutorial into your own container.
3. Select "NVIDIA RTX 4090" and "PyTorch" images. The OpenBayes platform has launched a new billing method. You can choose "pay as you go" or "daily/weekly/monthly" according to your needs. Click "Continue". New users can register using the invitation link below to get 4 hours of RTX 4090 + 5 hours of CPU free time!
HyperAI exclusive invitation link (copy and open in browser):
https://go.openbayes.com/9S6Dr
4. Wait for resources to be allocated. The first clone will take about 2 minutes. When the status changes to "Running", click the jump arrow next to "API Address" to jump to the Demo page. Due to the large model, it will take about 3 minutes to display the WebUI interface, otherwise "Bad Gateway" will be displayed. Please note that users must complete real-name authentication before using the API address access function.
Effect display
Select or upload personal audio, enter the conversation content, and click "Generate conversation" to generate the conversation.
*By default, Speaker A will start the first round of speaking, followed by Speaker A and Speaker B taking turns to communicate (currently only supports English content generation).
