Finding best Entropy

[Maureen D]

Could you please clarify the way the best possible questions for the entropy in CART are computed? That is, do someone has to manually enter the different features which could possibly be useful in a regression tree or is there a way to find all the possible features automatically?

[Simon]

Features must be specified by the system designer, using knowledge of the problem. If in doubt, every possible feature that the designer can think of is extracted – the tree building algorithm will select the most useful ones.

Questions about features can be hand-designed, but can often be automatically created from the features (e.g., enumerate all possible yes/no questions about every possible value of every feature).

At each iteration of tree building (i.e., splitting a node), all available questions are tried, and the one that most reduces the entropy (in the case of a classification task) is placed into the tree at that node.

[Author]

Posts