Unit or component testing

[Marianne DHK]

I don’t think I completely understand the point of unit/component testing yet as discussed by Taylor in 17.2.5. Are unit tests always used alongside system tests? What would be an example where the results from a unit test could provide more useful information than the system test? My impression is that the system should work as a whole and that might not be accomplished by just testing/improving all parts of it seperately even if you could.

[Simon]

Unit tests (actually, component tests – as discussed in this thread) are used during system development. For example, we might iteratively test and improve the letter-to-sound module in the front end until we can no longer improve its accuracy on a test set of out-of-vocabulary words. This component testing provides more useful information than system testing, because we know precisely which component is causing the errors we observe, so we know where improvements are needed.

Once our system is complete, and we think we’ve got decent performance in each component, we then perform end-to-end system testing.

Your insight that testing and improving individual components might not lead to best end-to-end performance certainly has some truth in it. Some components contribute far more than others to overall system performance. Putting that another way, end-to-end testing might sometimes help us identify (using our engineering intuition) which component needs most improvement: where a given amount of work would have the largest effect.

I think you are also suggesting that components should be optimised jointly. This is another good insight. For example, if errors in one component can be effectively corrected later in the pipeline, then there is no need to improve that earlier component. Unfortunately, the machine-learning techniques used in modules in a typical system (e.g., Festival’s text processing front end) do not lend themselves to this kind of joint optimisation.

[Author]

Posts