StorytellingSerious

29th International Conference on Computer and Information Technology (ICCIT 2026), Bangladesh ·

Controlled Evaluation of Cross-Lingual TTS Backends for Indic-to-English Dubbing

Jandhyala Dhanya, Laasya Reddy Dendi, Dandu Ashish Varma, BHVSP Subrahmanyam, Neeraj Choudhary

Accepted · Conference · Speech synthesis

CLF5 better preserves the speaker’s voice, while XTTS-v2 dubs the same clips faster.

Abstract

Dubbing systems trade intelligibility, speaker similarity, timing, and compute against each other. We compare Cross-Lingual F5-TTS (CLF5) and XTTS-v2 inside one Indic-to-English pipeline, holding separation, recognition, diarization, translation, reference selection, duration fitting, and remixing fixed, and we measure rather than assume that the shared stages held. The evaluation uses 18 public Hindi, Telugu, and Tamil clips totalling 79.2 minutes on one NVIDIA T4. Two of five metrics show no backend effect: mixed-dub WERtts is 0.187 for CLF5 and 0.181 for XTTS-v2 with a zero-spanning interval, and Speech Coverage sits at ceiling (99.94%) for both under the no-drop policy. CLF5 attains higher similarity to its conditioning reference (0.867 versus 0.822, p = 0.0003; generic-voice floor 0.555) and higher ratings from 17 listeners for naturalness (3.98 versus 3.73) and perceived speaker similarity (3.94 versus 3.69). XTTS-v2 reduces end-to-end RTF from 1.55 to 1.25, a 19% reduction. Disabling silence borrowing and the no-drop policy reduces coverage to 95.6% and 81.9% on a six-clip subset, indicating that the shared fitting stage, not the backend, sets speech completeness. Backend choice therefore trades reference similarity against throughput while leaving intelligibility and completeness unchanged.

Findings

  • The controlled comparison used 18 Hindi, Telugu, and Tamil clips totaling 79.2 minutes.
  • Both systems reached 99.94% speech coverage under the shared no-drop policy. The confidence interval for their word error rate difference included zero.
  • In ratings from 17 listeners, CLF5 scored higher for naturalness and perceived speaker similarity.

Sources and citation

This paper has been accepted. A public proceedings record is pending.

View BibTeX citation
@inproceedings{indic-dubbing-2026,
  title = {{Controlled Evaluation of Cross-Lingual TTS Backends for Indic-to-English Dubbing}},
  author = {Jandhyala Dhanya and Laasya Reddy Dendi and Dandu Ashish Varma and BHVSP Subrahmanyam and Neeraj Choudhary},
  booktitle = {29th International Conference on Computer and Information Technology (ICCIT 2026), Bangladesh},
  year = {2026},
  note = {Accepted}
}

What are you in the mood for?