Explaining expansion of NSWs

[Clem A]

I understand that normalization of NSWs involves classification & then expansion.

I have used disambiguators (RegEx + CART) to explain how NSWs are classified, but now I’m not sure how to explain the expansion process.

I feel like Finite State Transducers (which can recognise & expand) could be an explanation, but since the ‘recognition’ part has already been covered by RegEx + CART, i don’t know if it is?

Does Festival use FST? is an FST a general machine learning idea or is it a specific thing.
Basically I don’t know what to call the type-specific rules that are used to expand NSWs…

[Simon]

The expansion method depends on the category of NSW. Some are trivial and can be done with very simple rules (e.g., LSEQ) or no further processing (ASWD), whilst others require something more sophisticated, e.g., time or money expressions where there is context-dependency and possibly re-ordering between the characters and the words.

FSTs would be a sensible formalism for some categories, and these would generally be written by hand (possibly expressed via another formalism such as a grammar). The precise choice per NSW category will depend on the particular system, so don’t get hung up on that too much.

For the purposes of the assignment, you can assume Festival uses a variety of methods including both simple rules and FSTs.

You are right that an FST is both an acceptor (for recognising a pattern of characters in an NSW and thus classifying the token as being of that NSW type) and an emitter (for outputting the words). However, you are also right that classification takes places before expansion, so we only use the FST to transduce (“translate”) the token’s characters to words.

[Author]

Posts