Creating new voices using normalizing flows

Piotr Biliński; Tom Merritt; Abdelhamid Ezzerg; Kamil Pokora; Sebastian Cygert; Kayoko Yanagisawa; Roberto Barra-Chicote; Daniel Korzekwa

doi:10.21437/interspeech.2022-10195

Creating new voices using normalizing flows

Abstract

Creating realistic and natural-sounding synthetic speech remains a big challenge for voice identities unseen during training. As there is growing interest in synthesizing voices of new speakers, here we investigate the ability of normalizing flows in text-to-speech (TTS) and voice conversion (VC) modes to extrapolate from speakers observed during training to create unseen speaker identities. Firstly, we create an approach for TTS and VC, and then we comprehensively evaluate our methods and baselines in terms of intelligibility, naturalness, speaker similarity, and ability to create new voices. We use both objective and subjective metrics to benchmark our techniques on 2 evaluation tasks: zero-shot and new voice speech synthesis. The goal of the former task is to measure the precision of the conversion to an unseen voice. The goal of the latter is to measure the ability to create new voices. Extensive evaluations demonstrate that the proposed approach systematically allows to obtain state-of-the-art performance in zero-shot speech synthesis and creates various new voices, unobserved in the training set. We consider this work to be the first attempt to synthesize new voices based on mel-spectrograms and normalizing flows, along with a comprehensive analysis and comparison of the TTS and VC modes.

Citations

8

CrossRef
0

Web of Science
1 2

Scopus

Authors (8)

Piotr Biliński dr inż/
Tom Merritt
Abdelhamid Ezzerg
Kamil Pokora
Sebastian Cygert dr inż.
Kayoko Yanagisawa
Roberto Barra-Chicote prof.
Daniel Korzekwa mgr inż.

Cite as

Full text

download paper

downloaded 37 times

Publication version: Accepted or Published Version
DOI:: Digital Object Identifier (open in new tab) 10.21437/Interspeech.2022-10195
License: Copyright (2022 International Speech Communication Association (ISCA))

full content of the article see on external site open in new tab

Keywords

SPEECH

Details

Category:

Conference activity

Type:

publikacja w wydawnictwie zbiorowym recenzowanym (także w materiałach konferencyjnych)

Language:

English

Publication year:

2022

Bibliographic description:

Biliński P., Merritt T., Ezzerg A., Pokora K., Cygert S., Yanagisawa K., Barra-Chicote R., Korzekwa D.: Creating new voices using normalizing flows// / : , 2022,

DOI: