Traditional voice conversion methods rely on parallel recordings of multiple speakers pronouncing the same sentences. For real-world applications however, parallel data is rarely available. We propose MelGAN-VC, a voice conversion method that relies on non-parallel speech data and is able to convert audio signals of arbitrary length from a source voice to a target voice. We firstly compute spectrograms from waveform data and then perform a domain translation using a Generative Adversarial Network (GAN) architecture. An additional siamese network helps preserving speech information in the translation process, without sacrificing the ability to flexibly model the style of the target speaker. We test our framework with a dataset of clean speech recordings, as well as with a collection of noisy real-world speech examples. Finally, we apply the same method to perform music style transfer, translating arbitrarily long music samples from one genre to another, and showing that our framework is flexible and can be used for audio manipulation applications different from voice conversion.
Conclusions
We proposed a method to perform voice translation and other kinds of audio style transfer that doesn’t rely on parallel data and is able to translate samples of arbitrary length. The generator-discriminator architecture and the adversarial constraint result in highly realistic generated samples, while the TraVeL loss shows to be an effective constraint to preserve content in the translation, while not relying on cycle-consistency. We conducted experiments and showed the flexibility of our method with substantially different tasks. We believe it is important to discuss the possibility of misuse of our technique, especially given the level of realism achievable by our technique as well as other methods. While applications such as music genre conversion don’t appear to present dangerous uses, voice conversion can be easily misused to create fake audio data for political or personal reasons. It is crucial to also invest resources into developing methods to recognize fake audio data.