Module 5 – speech synthesis – waveform generation

First, remind yourself about the architecture of a text-to-speech system: a front-end linguistic processor, followed by a waveform generator.

CUI 2024 slides available

Pipeline architecture for TTS

The first method we’ll use for waveform generation is Time-Domain Pitch Synchronous Overlap and Add (TD-PSOLA), so let’s see first how that works in practice.

CUI 2024 slides available

TD-PSOLA …the hard way

If we wish to modify the spectral envelope, then we need a more powerful technique than TD-PSOLA, and so we’ll then look at linear prediction, which can manipulate source and filter separately.

Download the slides for the module 3, 4 and 5 videos

Download the additional slides for the class on 2019-10-24 : Module 5

Total video to watch in this module: 63 minutes

Diphones

Using our knowledge of speech production, we predict that making joins at mid-phone positions will sound better than at phone boundaries.

Using our knowledge of speech production, we predict that making joins at mid-phone positions will sound better than at phone boundaries.

In this video, I occasionally refer to a second screen displaying a waveform; that screen was not recorded. I think most of the content of this video still makes sense.

slownormalfast

Log in if you want to mark this as completed

TD-PSOLA

Time-domain pitch-synchronous overlap-and-add is a remarkably simple but effective way to independently modify the duration and F0 of speech.

Time-domain pitch-synchronous overlap-and-add is a remarkably simple but effective way to independently modify the duration and F0 of speech.

slownormalfast

Log in if you want to mark this as completed

Join smoothing

To understand the power of linear predictive waveform coding, we'll consider the problem of smoothing the joins in concatenative synthesis.

To understand the power of linear predictive waveform coding, we'll consider the problem of smoothing the joins in concatenative synthesis.

slownormalfast

Log in if you want to mark this as completed

The filter

It's just a simple equation, but for this course we don't need to get too deep into the maths.

It's just a simple equation, but for this course we don't need to get too deep into the maths.

The notation used in the linear prediction equation is slightly different from that used in class. It’s only a change of notation: the equation is otherwise exactly the same.

slownormalfast

Log in if you want to mark this as completed

Performing join smoothing

Now we understand how linear predictive coding works, we can use it to smooth the spectral envelope across joins.

Now we understand how linear predictive coding works, we can use it to smooth the spectral envelope across joins.

slownormalfast

Log in if you want to mark this as completed

Filter parameter update

Since the filter is time-varying, we need to decide how frequently (and at what moments in time) to update its co-efficients.

Since the filter is time-varying, we need to decide how frequently (and at what moments in time) to update its co-efficients.

slownormalfast

Log in if you want to mark this as completed

Inverse filtering

Exciting the filter with a simple pulse train doesn't produce good quality. Fortunately, there is an almost-perfect excitation signal: the residual.

Exciting the filter with a simple pulse train doesn't produce good quality. Fortunately, there is an almost-perfect excitation signal: the residual.

slownormalfast

Log in if you want to mark this as completed

This concludes the speech synthesis material for this course. You should have been doing all of the essential readings as you went along. Now is a good time to go back and complete as many of the recommended readings as you can, and perhaps some of the extra readings on topics that are of particular interest to you.

CUI 2024 slides available

Interactive unit selection