understand cart in a geometic way

[Siyu D]

To simplify the problem, I assume there are only two predictors used to predict the predictee and then I generate a tree either to do classification or to do regression. If I take one predictor as X-coordinate and the other as y-coordinate, thus node splitings can be viewed as a kind of partition of the xy-plane and within a particular partitioned area, there is a corresponding value. This value would be an interval if I do regression or class K if I do classification. My question is whether CART can only partition the xy-plane into pieces of rectangulars using verticle or horizontal lines and cannot generate areas with their boundaries curves or oblique lines

[E Hermann]

You can transform your features in any way, so if you ask “is y < f(x)”, you can divide the space with an arbitrary function.

[Simon]

Siyu, your understanding is exactly right: CART partitions the feature (predictor) space in a binary fashion. Each node lower down the tree subdivides the partition created by the answer (“Yes” or “No”) of its parent node.

See Figure 1 on this page, for a regression tree example.

Enno correctly points out that you can transform the features in any way you wish. But, it’s important to recognise that this would be “feature engineering” and would be done before starting to build the CART. The CART training algorithm can only select amongst the questions provided about the predictors; it cannot invent new predictors, or new questions, or learn feature transforms.

[Author]

Posts