weighted and unweighted sums

[Jason T]

At the end of “Trying the first split of the root node” video in the CART worked example, I do not understand what the simplification referring to weighted and unweighted sums of the entropy is? What do these terms mean?

[Jonas R]

In the last step of calculating the entropy from the split, you need to add the entropy of the “yes” node to the entropy of the “no” node. This is what takes place during the original video (at 6:31), where Simon adds the entropy values to get 0 + 0.972 = 0.972 bits. The final note at the end of the video says that entropy is actually not added like that though. Remember that two of the seven entries went to the “no” node, and the other five went to “yes”. That means that we need to multiply each of the entropies by the fraction of the entries for the respective node. Thus, we multiply the “no” entropy by 2/7 (0 * 2/7 = 0) and multiply the “yes” entropy by 5/7 (0.972 * 5/7 = 0.694). The answer in this case will be 0 + 0.694 = 0.694 bits.

[Simon]

Jonas is correct.

[Author]

Posts