Quiz

To reveal an answer, click a flashcard.

Simple neural networks, as described in this module, cannot model sequences. Just like regression trees, they only process one data point at a time. What additional machinery could be used to create a sequence-to-sequence (linguistic contexts to acoustic features) model?

We need to answer that separately for training and synthesis. For training, we need an external mechanism to align the input and output sequences; this can be borrowed from an HMM-based speech synthesis or automatic speech recognition system. For synthesis, we need a duration model. That could be also borrowed from an HMM synthesiser (i.e., it would be a regression tree), or it could be a neural network.

There are many different timescales in text-to-speech: utterances, words, syllables, phones, etc. How are these reconciled when using a neural network?

Rather naively! We simply pick the smallest timescale ("the clock that ticks the fastest"), which in this module was the frame. Typically, that clock ticks at 200Hz. All slower-changing timescales are upsampled to this rate. If we were directly generating a waveform, as is done in some recent approaches, the clock would have to run at the waveform sampling rate (e.g., 16kHz or higher).

Neural networks are trained to minimise the error between predicted and reference acoustic features, on a training set. Other than the neural network itself, what other design choices might influence what error is being minimised?

The two things I can think of are
(i) the choice of acoustic features - minimising error in the cepstral domain is not the same as minimising error in the spectral domain, for example;
(ii) the error measure (sometimes called the loss function), which is very often the mean squared error, although other choices are possible.