Merlin is a toolkit for building Deep Neural Network models for statistical parametric speech synthesis. It is a typical frame-by-frame approach, pre-dating sequence-to-sequence models.

A systematic investigation of the benefits of moving from frame-by-frame models to sequence-to-sequence models.

Somewhat old, but might be helpful in getting some of the basic concepts clear, if you find Nielsen’s “Neural Networks and Deep Learning” too difficult to start with.

A great introduction. Relatively light on maths, and with some interactive explanations.

A review article that makes some useful connections between HMM-based speech synthesis and unit selection.

Written with a traditional “starting from automatic speech recognition” viewpoint, you will need to make the connections for yourself to the more general concept of text-to-speech as a regression problem.

The key paper about the STRAIGHT vocoder, which was originally intended for manipulating recorded natural speech.

Explains the types of statistical tests that are employed in the Blizzard Challenge. These are deliberately quite conservative. For example, MOS data is correctly treated as ordinal. Also includes a Multi-Dimensional Scaling (MDS) section that is not as widely used as the other types of analysis.

A method for evaluating the intelligibility of synthetic speech, which avoids the ceiling effect.

An extension and practical application of Wells’ keyvowels idea, which enables efficient generation of a pronunciation dictionary tailored to a specific accent or speaker.

Discusses some of the design choices made when writing Festival’s unit selection engine (Multisyn) and the tools for building new voices.