Category questions

[Anonymous Student]

When it comes to the questions(features/predictors), I’m not quite clear about whether category questions could or should be used. For example

“Is predictor 1 in {a,e,i,o,u} ?”

[Simon]

These questions can be very useful, because a single split can give a large reduction in entropy, and we end up with a smaller tree than if we had to ask each individual question in sequence.

Including category questions is very common. It’s a kind of feature engineering because it’s exactly equivalent to adding a new 2-valued predictor to every data point.

This is a good way to include domain knowledge or our own intuitions about the problem.

Related example: In HMM-based automatic speech recognition, regression trees are used to cluster the parameters of context-dependent models. Category questions are used as standard.

[Author]

Posts