Preparation
Revise the Module 5 class about evaluation to remind yourself of what was covered there.
You should already have in mind some hypotheses that you want to test, since those would have influenced the additional speech that you have recorded in the studio. Now, you need to formalise those hypotheses, then test them.
It is essential to commit to each hypothesis in writing before proceeding. Spend a little time getting the wording just right, since that will be your guide when designing an experiment that tests the hypothesis (and, for example, doesn’t accidentally test something else).
Remember that you will get a higher mark for demonstrating understanding in your report, than for simply doing more work. Likewise, there are more marks to be obtained from a good evaluation of whatever voice(s) you mange to build, than for making those voices better or training an excessive number of models.
Listening test
You should have at least one (and perhaps two, but not many more) hypotheses that need to be tested using a listening test with multiple listeners. You will be asking a group of listeners to confirm something that you already believe to be true (i.e., to confirm what you have found through your own listening). This is most likely to be that there is some perceptual difference between two experimental conditions, but could also be to confirm that there is no difference.
Don’t use an excessive number of listeners – you won’t get a higher mark for that. Something around 10 listeners should be sufficient for the purposes of this exercise, depending on how many responses you gather per listener.
Other ways to test a hypothesis
You should also have a small number of other hypotheses that do not need a formal listening test, but can be tested in some other way. For example: expert listening (you are the expert!), or an objective measure. You might use qualitative and/or quantitive evaluations here.
Don’t test an excessive number of hypotheses: you won’t be able to fit them all in your report. It’s better to keep the number smaller, so that you can test each one rigorously, and write it up well.
Implementation
Use the forum to find out more about tools for running an evaluation.
Advice
You should run all your ideas past us in a lab session, before actually executing any evaluation. We will help you sharpen your hypotheses, and check that your proposed evaluation method will actually test the hypothesis (and only the hypothesis).