How to enable phone joins in multisyn

[Korin Richmond]

Festival has a few strategies it uses to cope with diphones that are required to synthesise a sentence but which are missing from its voice database. Specifically, it does the following things, in this order:

1) If the missing diphone is across a word boundary, then a short silence is inserted between the words instead.

2) Otherwise, the “diphone backoff rules” for the voice are searched to see whether any allowable substitutions will give diphones that *are* present in the voice database.

3) Otherwise, the diphone is omitted and a phone join is made instead – i.e. the units to the left and right of the missing diphone are extended to fill in, joined at a phone instead of diphone boundary

With that in mind, if you want to make Festival do phone joins to cope with missing diphones, you just provide an empty table of allowable back off options:

festival> (du_voice.setDiphoneBackoff currentMultiSynVoice nil)

This will set the backoff table to “nil”. Once you’ve done that, the next time you synthesise a sentence which requires a diphone that is missing, you’ll notice the following message:

requested diphone missing:
-=-=-=-=-=- EST Warning -=-=-=-=-=-
Making phone joins to compensate…
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

[Morten H]

I’m having some success with improving a voice with phone joins instead of back off, but I still have some bad sounding synthesis where inter word silence is inserted, and it would be intersting to experiment with that as well. Is there a way to avoid or control silence being inserted between words?

[Korin Richmond]

To change the default behaviour of inserting silence when a missing diphone occurs across a word boundary, you would need to edit the C++ code and recompile festival (i.e. IIRC, this behaviour is hard-coded in C++ and not accessible in Scheme).

[Author]

Posts