Handbook of phonetic sciences – Ch 20 – Intro to Signal Processing for Speech

The inner product between two signals is calculated by multiplying the corresponding samples (one from each signal) and summing up those values.

Intuitively, think of this as a measure of how similar the two signals are. If they are similar, then the inner product will have a high value. If they are very different, it will have a low value. So, we can understand the Fourier transform in this intuitive way:

Take the signal we want to analyse.

Create a sine wave of a particular frequency, and take the inner product between this and our signal. The resulting value is “how much of that frequency is present in our signal”. Plot that result, as a dot on a chart with frequency along the horizontal axis, and “how much” (i.e., magnitude) on the vertical axis.

Repeat for a range of frequencies. Join the dots. The final plot is the spectrum.

Fourier theory will tell us exactly what frequencies of sine waves we need to use, in order to perfectly characterise the signal (i.e., for the spectrum and the signal to contain exactly the same information, and thus to be able to make one from the other, in either direction).

Now on to phase: this is the relative offset (i.e., shift in time) between the sine wave and the signal, before we take the inner product. You correctly state that phase is important. Luckily, Fourier analysis not only computes the magnitudes of frequency components present in our signal, it also computes the phase that each sine wave needs to be at, so that when we sum those sine waves together we reconstruct our signal exactly.

Poles and zeros are properties of a filter. They correspond to the physical properties of resonance and anti-resonance.

It is common to model the vocal tract as an all-pole filter: something with only resonances. The most common all-pole filter used is Linear Prediction.

The relationship between our model (i.e., filter) parameters and the vocal tract shape is not trivial, because our model is such a simplistic approximation of the true vocal tract. So, for example, we wouldn’t normally use pole frequencies as features for Automatic Speech Recognition (although in the early days, features like that were widely used).

But, for conceptual understanding, we can say that the poles of a Linear Prediction filter correspond to resonant frequencies of the vocal tract, which we call formants. (Poles occur in pairs, and there will be two poles per formant).

To do formant tracking, we could fit an all-pole filter to a speech signal, and use the poles to identify the formants.

[This level of detail is beyond the scope of Speech Processing. These concepts are still important, and will become more relevant in Speech Synthesis.]

Linear Prediction refers to a specific form of filter being used in a source-filter model. A linear predictive filter is very simple: it predicts each speech sample (the filter’s output) as a weighted sum of the previous few speech (i.e., filter output) samples. The weights are called the filter co-efficients.

Such a filter has only resonances (technically called “poles”), and no anti-resonances (“zeros”). It can be used as a simple model of the vocal tract. We need to excite the filter with an input signal, such as a pulse train. The output generated will be a synthetic speech waveform.

The frequency response of the filter corresponds to the spectral envelope of the generated speech.

It’s important to realise that, when we model speech with a simple source-filter model, such as linear prediction, we are only really modelling properties of the signal. We are not directly modelling the vocal tract in any realistic sense.

The two forms of energy exchanged are the same as in the swing.

A swing has maximum potential energy at its highest point, when it is momentarily stationary (= no kinetic energy). It has maximum kinetic energy at its lowest point, when it is moving fastest.

In air resonating in a tube, the same two forms of energy are exchanged. Potential energy is air under increased pressure, and kinetic energy is air moving at maximum velocity.

Look at these air molecules – they alternate between being “all bunched together and stationary” (maximum potential energy) and being “evenly spaced and moving quickly” (maximum kinetic energy).

To understand why pressure is potential energy, imagine a cylinder storing gas: it contains high pressure gas. Open the valve and this is converted into kinetic energy as the gas rushes out at high speed. Potential energy has been converted to (= exchanged for) kinetic energy. The total amount of energy is conserved.