Qwen-Audio-3.0-ASR Improves Entity Recall With Hierarchical Hotwords

Speech recognizers can post low error rates on clean benchmarks while still failing on the details that determine whether a production transcript is usable: regional dialects, changing entities, conversation history, disfluencies, and streaming latency. Qwen-Audio-3.0-ASR addresses those cases with a Qwen-based mixture-of-experts system that treats transcription behavior as an instruction-controlled decoding task rather than a fixed acoustic-to-text pass. The report covers 30 languages and 16 Chinese dialectal varieties, alongside a streaming variant for latency-sensitive deployment.

Core Contribution

The central contribution is not a new benchmark score in isolation, but a collection of ASR controls placed inside one decoding framework: dialect-aware transcription, domain-entity recognition, two-tier hotwords, long-audio context, and optional native polishing. The authors argue that integrating these controls into the recognizer avoids the operational cost and error propagation of separate adaptation and rewriting stages.

Figure 1 organizes those production features around four system properties: instruction-controlled decoding, context awareness, single-pass generation, and low latency.

The most concrete addition is hierarchical hotword conditioning. Retrieved candidates are split into a high-priority set, P0, and a broader P1 set, with P0 assigned greater conditioning weight. That division is intended to make an explicitly supplied critical term more likely to survive decoding without discarding useful lower-confidence candidates.

Technical Approach

The report describes a Qwen-backbone MoE model trained on tens of millions of hours of speech. Its audio encoder is paired with text instructions so that output mode can be specified at inference time. The same recognition pass can emit either a raw transcript or a polished transcript; the latter removes fillers and stutter-like repetitions, resolves self-corrections toward the speaker's final wording, and normalizes punctuation and formatting.

Long-audio context is handled by conditioning a current audio segment on recently recognized text from the same session. The examples show why this differs from frame-local acoustic decoding: a prior mention can disambiguate a technical filename, a repeated abbreviation, or a person's name when the current audio alone supports a plausible but incorrect homophone. This is useful for meetings, messages, and call transcripts, where local acoustics often do not contain enough evidence.

The streaming workflow retains configurable right context and refreshes the displayed hypothesis at utterance end using all available context. Figure 5 makes the deployment trade-off explicit: partial transcripts are produced incrementally, while longer context can improve the finalized text.

Results and Analysis

On the multilingual public benchmarks in Table 2, Qwen-Audio-3.0-ASR reports the lowest macro average among the listed systems on GigaSpeechBench and Common Voice 15: 22.50 versus 22.76 for Doubao-ASR on GigaSpeechBench, and 4.57 versus 5.01 for Azure on Common Voice 15. The comparison is less uniform on FLEURS, where its 4.94 macro average trails GPT-4o Transcribe at 3.95. That mixed result matters: the system is competitive across several public multilingual sets, but the report does not establish a universal lead over every commercial baseline.

The production-oriented tests are more favorable. Across 15 internal industry domains, Figure 8 reports the highest entity recall for Qwen-Audio-3.0-ASR in every displayed domain. In Table 3, hotword conditioning raises recall for P0 person names from 62.12% to 99.43%, for P0 subject terms from 69.36% to 99.42%, and for P0 trending buzzwords from 63.08% to 99.46%. The result supports the narrower claim that high-priority supplied entities can be recovered reliably under this evaluation setup; it does not by itself show how well retrieval finds those candidates in an unconstrained application.

Native polishing improves the report's internal readability score from 2.53 to 3.44 on a 1–5 scale, while retaining a 3.44 faithfulness score. Its faithfulness is close to the 3.47 reported for a Qwen3.6-Plus cascade, but requires one inference pass rather than two. That is a credible systems advantage when end-of-utterance latency and pipeline simplicity matter, although the rating scale and evaluators are not independently described in the shown results.