Block's New AI Model Dessa Trained Exclusively on Human Audio

Block has announced an artificial intelligence model named Dessa that was trained entirely on audio data, excluding any text. The company states the system was built "from scratch on nothing but human conversation." This approach distinguishes the model from standard large language models that typically process written words before generating spoken responses.

The development team, led by Ragavan Thurairatnam, published several audio recordings to demonstrate the model's capabilities. In one clip shared by Thurairatnam, the interaction appears largely natural, though the model initially misidentifies the speaker as someone named Ronaldo. Other samples on the project website show varying levels of coherence. In one exchange about Christopher Nolan's films, the model diverts into a discussion of Bryan Singer's Superman movies. In another, a user describes a surprise trip to Paris, prompting what the site describes as a response of "teary gratitude." A separate test pairing Dessa with OpenAI's GPT-Live-1 voice model produced results that sounded more fluid to listeners.

The team emphasizes that Dessa is a "true speech-to-speech model," meaning it processes and outputs audio without intermediate text translation. They note the system can operate locally on a MacBook Pro M4 Max or a more powerful machine. This local processing capability suggests the model is designed for lower latency and reduced reliance on cloud servers compared to many existing voice assistants.

Three primary motivations drive the project: curiosity, improved human-AI understanding, and safety. The developers compare the training method to how infants learn language. They note that babies acquire speech without reading written words, and they sought to replicate that biological process in a machine. The team acknowledges the technical difficulty, stating, "It can. It's just very hard." They argue that current voice interfaces flatten spoken language into text, causing a loss of timing and nuance inherent in human dialogue.

The safety argument focuses on emotional perception. The team observes that as AI systems gain autonomy, they often remain "nearly deaf to the emotion in our voices." They cite a scenario where a person in severe pain speaks to a voice assistant that lacks the ability to detect distress. Dessa is positioned as an initial step toward resolving this gap by processing emotional cues directly from the audio stream.

Block intends to provide early access to a small group of AI safety researchers. The announcement did not specify a timeline for a broader public release or detailed technical specifications beyond the hardware requirements and training methodology.