en.Wedoany.com Reported - US-based voice artificial intelligence company Rime is expanding its voice data collection, model development, and industry delivery system, with a focus on advancing low-latency speech-to-speech models. The company has built its own recording studio in San Francisco, collecting training data through real-person dialogue recordings to reduce reliance on publicly available online audio, and optimizing voice interaction capabilities for scenarios such as enterprise customer service, healthcare, aviation, and fintech.
Founded in 2022, Rime's founding team members have backgrounds in research at Stanford University, development of Amazon's Alexa voice system, and engineering R&D in the US. Unlike methods that directly scrape internet audio for model training, the company collects dialogue data in its recording studio with clear speaker, context, and pronunciation information, which is then used for training speech synthesis, pronunciation control, and interaction models.
The proprietary recording system enables more targeted model training. Enterprise voice systems often need to handle brand names, product models, drug names, and industry-specific terminology in real-world use, where general-purpose voice models are prone to accent errors, missing syllables, or inconsistent pronunciation. Rime adopts a phoneme-based model architecture, breaking words down into more fundamental pronunciation units and adjusting pronunciation methods based on the usage environment of different enterprises and industries.
This technical approach does not require customers to retrain the entire model. Enterprises can add brand vocabulary, specialized terms, and specific pronunciations to existing voice systems, enabling AI voices to maintain high accuracy when dealing with professional content in healthcare, aviation, and finance. The company's R&D focus has shifted from simply generating natural-sounding voices to addressing pronunciation consistency and voice interaction stability in real business scenarios.
Previously, Rime used a multi-model pipeline consisting of speech-to-text, text processing, large language models, and text-to-speech. After a user speaks, the system first recognizes the text, then generates a response via the large language model, and finally converts the text into speech. Running multiple models sequentially can increase latency and may lead to unnatural tone, pauses, and turn-taking.
The company is now transitioning to developing speech-to-speech models, allowing the system to directly understand voice input and generate voice output, reducing the orchestration steps between multiple models. This new R&D direction focuses on addressing issues such as response latency, turn-taking in multi-party conversations, background noise interference, and user interruptions, enabling AI voice agents to more closely mimic the conversational rhythm of human customer service representatives.
Speech-to-speech models also need to integrate semantic understanding, voice generation, and real-time interaction into a unified system. The model must not only determine what the user said but also recognize speaking speed, pause positions, and whether the conversation has ended, before deciding when to respond. Reducing the number of intermediate models shortens the system's operational path and facilitates unified control over voice style, emotion, and pronunciation.
Rime's voice models are already deployed in food services, healthcare, aviation, and fintech, with clients including Mayo Clinic, Dialpad, Upstart, and Asurion in the US. Different applications have varying requirements for voice systems: healthcare scenarios require accurate handling of disease and drug names, aviation scenarios need to recognize flight and airport information, while customer service systems prioritize response speed and stability during long calls.
The company also plans to expand its model development, engineering implementation, and partner delivery teams to strengthen the transition of voice models from R&D environments to enterprise systems. Rime recently brought on a chief scientist with experience in audio understanding and deep learning R&D, and will continue to refine its processes for data collection, model training, system deployment, and customer adaptation.
The core of this business adjustment is not simply increasing the number of voice models, but building a complete system from real-person voice collection, phoneme data processing, model development, to enterprise delivery. Subsequent progress will primarily be reflected in the launch of speech-to-speech models, reduced interaction latency, improved noise environment recognition, and integration with more industry systems.
