Whilst the video is playing, click on a line in the transcript to play the video from that point. Let's move on from discrete things and colored balls to continuous values because that's what our features for speech recognition are They're extracted from frames of speech And so far where we've got with that is filter bank features A vector of numbers Each number is the energy in a filter in a band of frequencies for that frame of speech So we need a model of continuous values And the model we're going to choose is the Gaussian So I'm going to assume you know about Gaussians from last week's tutorial But let's just have a very quick reminder Let's do this in two dimensions So I'll just pick two of the filters in the filter bank and draw that two-dimensional space Perhaps I'll pick the third filter and the fourth filter in the filter bank And each of the points I'm going to draw is the pair of filter bank energies It's a little feature vector So each point is a little two-dimensional feature vector containing the energy in the third filter and the energy in the fourth filter So lots of data points If I would like to describe the distribution of this data with a Gaussian It's going to be a multivariate Gaussian This means going to be a vector of two dimensions And its covariance matrix is going to be a two by two matrix I'm going to have here a full covariance matrix Which means I could draw a Gaussian that is this shape on the data We've made the assumption here that the data are distributed normally And so that this parametric probability density function is a good representation of this data So I can use the Gaussian to describe data But how would we use the Gaussian as a generative model? Let's do that But let's just do it in one dimension to make things a bit easier to draw So here I've got my three models again And by some means yet to be determined I've learned these models They've come from somewhere And these models are now Gaussians So this is really what the models look like Model A is this Gaussian It has a particular mean and a particular standard deviation Along comes an observation So these are univariate Gaussians Our feature vectors are one-dimensional feature vectors So along comes a one-dimensional feature vector It's just a number And the question is Which of these models is most likely to have generated that number? Here's the number 2.1 Now remember that a Gaussian can't compute a probability That would involve integrating the area between two values So for 2.1 all we can say is what's the probability density at 2.1 So off we go 2.1 This value 2.1 This value 2.1 This value Compare those three Clearly this one is the highest And so we'll say this is an A And that's how we'd use these three Gaussians as generative models We'd ask each of them in turn Can you generate the value 2.1? Now for a Gaussian the answer is always yes Because all values have non-zero probability So of course we can generate a 2.1 What's the probability density at 2.1? We just read that off the curve Because it's a parametric distribution And compare those three probability densities So we've done classification with a Gaussian Let's just draw the three models on top of each other On top of each other to make it even clearer What's the probability of 2.1 being an A or a B or a C?
Gaussian distributions in generative models
Using Gaussian distributions to describe data and as generative models
Log in if you want to mark this as completed
|
| ||||||||||||||||||||||||||||||||||||||||||