How you can help evaluate speech synthesis systems
« previous post | next post »
The University of Edinburgh is co-ordinating the Blizzard Challenge, the only regular speech synthesis evaluation campaign in the world. And you can help. The systems are evaluated by having people listen to them in operation and reporting on what they hear. The listening test has now started, and if you have a computer with sound capability, and maybe a nice pair of headphones, you can be one of the expert evaluators. It's a chance to hear synthesis samples from over 20 leading academic and commercial research groups (and some lesser-known ones). Hundreds of listeners are needed for this test, so the organizers are turning to Language Log. Do think about taking part.
There are in fact have two parallel tests running this year: one on English, accessible if you click here, and if you have the necessary language skills, one on Mandarin Chinese, accessible if you click here. You can do both, if you speak both languages. Each one should take under an hour and can be done in several sessions if you prefer (the system will remember where you got to and pick up from there next time). If you know other people who might be interested in evaluating the speech quality of machines that talk, please encourage your students to take part.
Don't be jealous if you're not a student here at Edinburgh, but I should mention that undergraduate students at the University of Edinburgh who speak UK English can even get paid to do the evaluation test. They should look for the advert on the student employment service web pages later next week.
For everyone else, it's just a way of serving science and society by contributing to the quality of the voice synthesis systems of the future. You can help make sure they won't all sound like daleks ("Exterminate! Exterminate!") by evaluating them now and giving the best ones the highest grades. One day we will be able to say to a machine, "Open the pod bay doors, HAL", and be fully and immediately understood, and we'll hear a perfectly modulated synthetic voice come back saying "I'm afraid I can't do that, Dave." If our name is Dave, that is, and we're trapped outside in space and a rogue computer has taken control of the ship and is determined to make us die in the icy emptiness of the space between Mars and Jupiter. Or it might be better than that: you might actually interact with some voice-driven systems on the phone that have machine-synthesized voices that don't make your teeth itch. It's a worthy cause. Take the test. Evaluate a speech synthesis system for the betterment of science, technology, and the society of the future. Thank you. Again, it's this one for English, and this one for Mandarin. Basic details about the project here.