Acting on the evaluation result

[Natasha H]

It sounds like it would be difficult to trace back to the problem because we are using a pipeline system. How can we actually improve our system by using the data collected?

[Simon]

You are correct in saying that it’s very hard to trace back a problem in the output speech to a specific component in the synthesiser, especially so when the text-processing has the typical pipeline architecture.

Where possible, we will perform component-level testing and, if we are lucky, that can be done objectively (without humans) by comparing output to a gold-standard reference.

Otherwise, what we have to do is make a hypothesis, then create an experiment to test (or refute) that hypothesis. In general, this is going to require us to create two or more systems that differ in a specific way (e.g., the pronunciation dictionary is or is not carefully tuned to the speaker), then compare them in a listening test.

[Author]

Posts