We warmly invite you to participate in the “VoiceMOS Challenge”, a machine learning challenge which has been preliminarily accepted as a special session in INTERSPEECH 2022.

The purpose of this challenge is to compare different machine learning based systems and approaches on the task of predicting the human-rated mean opinion score (MOS) of synthetic speech. With recent interest in data-driven approaches for MOS prediction using machine learning, a challenge for this task to encourage research in this area is timely and important. We recently collected a large-scale dataset of MOS ratings for a large variety of text-to-speech and voice conversion systems spanning many years.  This machine learning challenge releases this data to the public for the first time.

The challenge is divided into a main track and an “out-of-domain” (OOD) sub-track. Participants of both tracks are provided with a shared training and validation set curated from the above-mentioned dataset. The main track uses the test set from the same corpus, thus amounting to in-domain evaluation. The  OOD sub-track, on the other hand, uses out-of-domain (OOD) datasets for evaluation from a separate listening test on different audio samples. We are using the CodaLab (https://competitions.codalab.org/) as the competition platform, and we provide a few baseline scripts for training MOS prediction systems and generating sample submissions.

The challenge schedule is as follows:
        • Release of main track and out-of-domain training data: current
        • Release of evaluation data: February 21, 2022
        • Results submission deadline: February 28, 2022
        • Results sent to participant: March 7, 2022
        • INTERSPEECH Paper submission: March 21, 2022

Likewise with many challenges, there is no participation fee.
If you are interested in participating in the challenge, please do the following two steps to register:

1. Register for the challenge on CodaLab:  https://codalab.lisn.upsaclay.fr/competitions/695

2. Fill out this Google Form with information about your team: https://docs.google.com/forms/d/e/1FAIpQLSfvXt5hLRmiICN2SwmInWAAStgCtwC6a8XyzITZ6bt-2gt1HQ/viewform

Please only register one CodaLab account for your team.  Please also be patient since we need to manually approve your account on CodaLab; we typically approve accounts within one day.  Once your account is approved, you will be able to see more information on the CodaLab page about downloading the challenge data, etc.

Please freely contact us if you have questions at voicemos2022@nii.ac.jp.

Looking forward to hearing from you.
Regards,
The VoiceMOS Challenge organizing committee
Erica Cooper, Wen-Chin Huang, Yu Tsao, Hsin-Min Wang, Junichi Yamagishi, and Tomoki Toda