Creating independent features by rotating the feature space

[Ahmed Abdellah]

When I first saw the 2D data points that are tilted along a diagonal line in the plot (hence covariance exists), the first thing that came into my mind is to apply rotation to all of those data point so that the diagonal line becomes vertical (or maybe horizontal, whichever closest). And that way we, solve the covariance problem in one step. (picture included)
Is my solution possible?

Attachments:

You must be logged in to view attached files.

[Simon]

Yes … but you would need to introduce an extra parameter to model that rotation. Since this is just for one Gaussian (e.g., for one HMM state), that extra parameter would only correct the rotation for its distribution. You would need to introduce an extra “rotation” parameter for every Gaussian, because it would (in general) be different in each case.

In fact, that’s exactly what covariance does. For a 2-dimensional distribution, there is just one covariance parameter (remember that the covariance matrix is symmetric), and you can think of it as that rotation parameter.

[Author]

Posts