Joint training framework for text-to-speech and voice conversion using multi-source Tacotron and WaveNet

Voice Conversion

Categories: Fourier Transform | Mean Opinion Score | Mel-spectrogram | Recurrent Neural Network | WaveNet

Tags: 2019 | Fuming Fang | Haizhou Li | Junichi Yamagishi | Mingyang Zhang | Xin Wang

We investigated the training of a shared model for both text-to-speech (TTS) and voice conversion (VC) tasks. We propose using an extended model architecture of Tacotron, that is a multi-source sequence-to-sequence model with a dual attention mechanism as the shared model for both the TTS and VC tasks. This model can accomplish these two different tasks respectively according to the type of input. An end-to-end speech synthesis task is conducted when the model is given text as the input while a sequence-to-sequence voice conversion task is conducted when it is given the speech of a source spea...

High-quality nonparallel voice conversion based on cycle-consistent adversarial network

Voice Conversion

Categories: Adversarial Training | Deep Learning | Dynamic Time Warping | Fourier Transform | Generative Adversarial Network | Mean Opinion Score | Mel-spectrogram

Tags: 2018 | Fuming Fang | Isao Echizen | Jaime Lorenzo-Trueba | Junichi Yamagishi

Although voice conversion (VC) algorithms have achieved remarkable success along with the development of machine learning, superior performance is still difficult to achieve when using nonparallel data. In this paper, we propose using a cycle-consistent adversarial network (CycleGAN) for nonparallel data-based VC training. A CycleGAN is a generative adversarial network (GAN) originally developed for unpaired image-to-image translation. A subjective evaluation of inter-gender conversion demonstrated that the proposed method significantly outperformed a method based on the Merlin open source neu...