DNNs vs Decision trees

[Pilar O]

In the Zen’s reading there is a part where they compare the advantages of DNNs vs decision trees, and they say:

“Decision trees are inefficient to express complicated functions of input features, such as XOR, d-bit parity function, or multiplex problems (p. 2)”

I don’t understand this! could you explain this in simple words?
Thank you!

[Simon]

I think Zen has been “borrowing” text from Wikipedia !

XOR (which means “exclusive OR”) is a logic function and is often used as an example of something that is non-trivial to learn. For a decision tree to compute XOR, the tree will have duplicated parts, which is inefficient. Here’s a video that explains:

To compute XOR with a neural network, at least two layers are needed.

More generally, the divide-and-conquor approach of decision trees is inefficient for considering combinations of predictors that “behave like XOR”: the tree gets deep, and the lower parts will not be well-trained because only a subset of the data is used.

It’s hard to say what XOR, d-bit parity functions, or multiplex problems have got to do with speech synthesis though (we should ask Zen!), other than that they are also non-trivial to compute.

So, all that Zen is really saying is that neural networks are more powerful models than decision trees. Whether neural networks actually work better than decision trees for speech synthesis remains a purely empirical question though: try them both and see which sounds best!

[Author]

Posts