Audio LLM Feedback Improves Text-to-Audio Instruction Following

Text-to-audio systems can produce convincing short clips while still missing the instruction. The failure mode is most visible in prompts with several sound events and an order constraint: a clip may contain rain, footsteps, and a car, but omit one event or place them in the wrong sequence. The paper argues that global audio-text similarity and perceptual quality metrics are too coarse for this setting, then introduces an instruction-level training signal built from audio-aware large language models.

Core Contribution

The central idea is to use an audio-aware LLM as a structured judge rather than as a generic caption scorer. For each generated candidate, the judge checks two explicit properties: whether every requested sound event is present, and whether the events occur in the requested temporal order. Those judgments are converted into preferred and rejected samples for direct preference optimization, so the text-to-audio model is trained against the same errors the authors want to reduce.

Figure 1 shows the full ALLM-Judged Preference Optimization loop: a reference text-to-audio model samples multiple clips for one instruction, an audio-aware LLM scores event existence and ordering, and the resulting preference pairs fine-tune the generator.

Technical Approach

The method, called ALLM-Judged Preference Optimization, first validates whether current audio-aware LLMs can serve as instruction-level evaluators. The authors test Qwen2.5-Omni-7B, Qwen2.5-Omni-3B, and Audio Flamingo 3 on event-existence and temporal-order benchmarks, including AudioCaps-test, CompA, AudioTime, and synthetic MultiEvent-Temporal-2/3/4 sets built by concatenating ESC-50 clips. The judge outputs binary existence decisions for target events and a relative ordering over events, summarized through exact match, micro accuracy, pairwise accuracy, Kendall’s tau, and joint accuracy.

Preference construction is deliberately fine-grained. A preferred sample must satisfy full event coverage and full temporal-order agreement. Rejected samples are split into two failure types: clips with all events present but wrong order, and clips with missing events. This separation matters because it lets the training signal distinguish semantic incompleteness from temporal mistakes, which CLAP-style ranking tends to collapse.

The authors also introduce Sound Scene Story Benchmark, or S3Bench, for evaluation rather than training. It contains 1,200 narrative-style instructions: 300 with two events, 500 with three, 200 with four, and 200 involving overlapping events. The prompts are designed to stress temporal progression and multi-event composition more than typical short caption benchmarks.

Results and Analysis

The judge validation is the first support point. Qwen2.5-Omni-7B is the strongest evaluator in Table 1, with 94.4% joint accuracy on AudioTime and 89.1% on MultiEvent-Temporal-2, though it drops to 57.6% on MultiEvent-Temporal-4. Human verification on 150 generated clips reports 89.8% agreement for event existence and 93.5% for temporal order, with Pearson 0.85 and Spearman 0.83 for existence and Kendall’s tau 0.93 for temporal ordering. That is enough to make the judge useful as a training signal, but it is not error-free, especially as event count rises.

On AudioCaps-test, the final ALLM-DPO model reaches 87.4% event-existence exact match, 94.7% micro accuracy, 89.1% temporal exact match, 92.8% pairwise temporal accuracy, Kendall’s tau 0.86, and 71.0% joint accuracy. The closest training baseline, TangoFlux-CRPO, reaches 67.4% joint accuracy with 87.4% event-existence exact match and 85.7% temporal exact match. The gain is therefore concentrated in temporal ordering and joint instruction satisfaction, not in a broad increase across every metric.

The standard audio metrics are more mixed but mostly stable. ALLM-DPO reports FAD 3.31, FD 20.07, KL 1.16, IS 11.54, and CLAP 0.375 on AudioCaps-test. TangoFlux-CRPO has better FAD at 2.34 and higher CLAP at 0.397, while the proposed model improves joint instruction accuracy. That trade-off supports the paper’s main claim: optimizing for fine-grained instruction following can improve correctness without a large collapse in audio quality, but it does not dominate every distributional or similarity metric.

The harder benchmarks show why the method is useful. On MultiEvent-Temporal-3, ALLM-DPO reaches 55.0% joint accuracy versus 40.0% for TangoFlux-CRPO and 32.8% for TangoFlux-base. On MultiEvent-Temporal-4, it reaches 18.4% versus 11.6% and 7.6%. On S3Bench, it reaches 49.9% joint accuracy versus 45.4% for TangoFlux-CRPO and 35.3% for TangoFlux-base. These are still far from solved, but the relative gains are largest where event structure is most demanding.

Figure 2 captures the online DPO result: rebuilding preference pairs over successive rounds improves micro accuracy, Kendall’s tau, and joint accuracy, while the static one-round variant loses quality after later iterations.

Limits and Significance

The evidence is strongest for preference training around the tested TangoFlux-style models and Qwen-based judges. The authors do not show that the judge is unbiased, that it transfers equally to all audio domains, or that four-event narrative instructions are close to practical long-horizon audio generation. Still, the paper makes a useful shift: it treats instruction following as a verifiable structure in the audio, not as a side effect of better caption similarity.