Weighted sum of the two entropies

[Ross C]

In TTS notebooks, it says the denominator for the weighted sum is the ‘left’ side, but isn’t it completely arbitrary which side is left and which is right?

[Simon]

Which cell of which notebook are you referring to?

[Ross C]

It’s solved now. I hadn’t noticed the formula spread beyond the visible end of the line (i.e. ” + sum(right_counts)) “). I should have counted my brackets.

The rest of this post is the original problem:

I want to achieve this for my second level (having asked one question): “Now we need to calculate the entropy of each of those two distributions, and compute the weighted sum. This will give the total entropy of the partitioned data.”

The above instruction is in tts-m4-2-decision-tree-pencil-and-paper (search for “weighted”)

I think I should use the following calculation: “total_entropy = ( sum(left_counts)*entropy(left_counts) + sum(right_counts)*entropy(right_counts) ) / (sum(left_counts)”

The above instruction is in tts-m4-1-entropy (search for “weighted”).

[Simon]

total_entropy = ( sum(left_counts)*entropy(left_counts) + sum(right_counts)*entropy(right_counts) ) / (sum(left_counts) + sum(right_counts))

[Author]

Posts