Publication Info
- Authors: Qibing Bai, Shuai Wang, Yuhan Du, Bohan Li, Yannan Wang, Haizhou Li
- Venue: arXiv preprint
- Year: 2026
- Paper: https://arxiv.org/abs/2607.03928
- Project Page: https://p1ping.github.io/TokAN-Samples
- Code: https://github.com/P1ping/TokAN
- BibTeX: /bib/tokan.bib
Abstract
Accent normalization (AN) seeks to convert non-native (L2) accented speech into standard (L1) speech while preserving speaker identity. Current techniques either require naturally recorded parallel L1–L2 speech for training, or suffer from quality degradation when supervised by synthesized targets.
We present TokAN, a token-based accent normalization framework that operates on self-supervised discrete speech tokens extracted from a jointly trained vector-quantization (VQ) tokenizer. An autoregressive encoder-decoder model performs token-to-token conversion, translating L2-accented token sequences into standard speech tokens. The system further uses GRPO-based reinforcement learning post-training with word error rate and accent classifier confidence as complementary rewards.
TokAN reconstructs speech through a non-autoregressive flow-matching synthesizer conditioned on the source speaker embedding, and includes a flow-matching duration predictor for total-duration-aware synthesis. Experiments on seven English accents show that TokAN reduces WER from 12.40% to 9.89% after supervised fine-tuning, and further to 9.23% after RL post-training.
Highlights
- Token-level accent normalization without synthetic supervisory speech.
- Jointly trained SSL VQ tokenizer optimized for speech reconstruction and phonetic content.
- Accent-universal autoregressive token converter with RoPE and self-attention-only decoding.
- GRPO post-training directly optimizes intelligibility and accent reduction.
- Duration-aware synthesis supports duration-critical use cases such as dubbing and live casting.
