Researchers reported a 5.76% macro-average error score for Confucius4-R2T2 on 160-millisecond speech chunks, offering a benchmark for a model designed to keep live transcripts stable. The paper describing the tests was posted Nov. 4.
The score compared with 21.01% for the model’s base, Qwen3-ASR-1.7B, under the same settings. Both systems were evaluated with one token withheld before it appeared in the released transcript.
The researchers combined results from 13 English and Chinese datasets, assigning the same weight to each. They measured English errors by words and Chinese errors by characters, so the combined figure is a macro-average across the two types of measures.
Researchers started with Qwen3-ASR-1.7B and post-trained it on 1,500 hours of audio in English and Chinese. The researchers used a Longest Stable Prefix method, which advances a transcript as more speech arrives while keeping text that has already been released fixed. The adaptation did not change the base model’s architecture.
The model also has a “Wait” action that can request more audio before committing to text. Researchers said fewer revisions could benefit live captions, simultaneous translation and language-processing systems that rely on recognized speech.
That stability comes with a timing trade-off. The paper says withholding a token improves recognition but adds a few tenths of a second to mean emission lag. Its retrospective emission lag measure excludes processing, network, queueing and rendering time, so the 160-millisecond chunk size should not be read as the wait a viewer experiences. The study did not measure end-user wait in a deployed system.
The researchers have released the model’s weights and code.
