Issues with model name (label dimension)

[Etienne D]

Good evening,

I am currently trying to generate test utterances using a durational model.

I have trained the model, and have generated new labels instead of the oracle durations, I’ve moved those new labels in the data/label_state_align folder.

I then attempted to generate the output features of the acoustic model, but I get an error :

2016-07-25 01:47:48,617 INFO main : saved 187 vectors to /mnt/courses.homes/s1571333/Documents/dna_synth/ss_dnn/data/label_norm_HTS_187.dat
2016-07-25 01:47:48,632 DEBUG labels : HTS-derived input feature dimension is 187 + 0 = 187
2016-07-25 01:47:48,632 INFO main : label dimension is 187
2016-07-25 01:47:48,633 INFO main : generating from DNN
2016-07-25 01:47:48,634 DEBUG dnn_generation: Starting dnn_generation
Traceback (most recent call last):
File “../dnn_tts/run_lstm.py”, line 1091, in <module>
main_function(cfg)
File “../dnn_tts/run_lstm.py”, line 885, in main_function
dnn_generation(test_x_file_list, nnets_file_name, lab_dim, cfg.cmp_dim, gen_file_list)
File “../dnn_tts/run_lstm.py”, line 355, in dnn_generation
dnn_model = cPickle.load(open(nnets_file_name, ‘rb’))
IOError: [Errno 2] No such file or directory: ‘/mnt/courses.homes/s1571333/Documents/dna_synth/ss_dnn/nnets_model/DNN_TANH_TANH_TANH_TANH_TANH_TANH_LINEAR__mgc_lf0_vuv_bap_0_6_1024_1024_1024_1024_1024_1024_187.199.train.1750.0.004000.rnn.model’

Checking the models in my nnets_model folder, I can see that I do not have this model, but another one that differs only in the dimensionality of the labels (lab_dim in run_lstm.py).

I am assuming that the issue is coming from the 9 positional features added during the NORMLAB step, but I am not sure how to solve that.

Thanks for your time, and enjoy Crete (if you’re there)!

Edit: Fixed the error message

[Simon]

You need to have exactly the same kind (including dimensionality) of input features at training time and at synthesis time. It sounds like you have trained a network that uses the 9 positional features, but are trying to synthesise from it without providing those features. Either

– add them to the inputs you are synthesising from

or

– remove them from the inputs for the training data

In general, the input text for synthesis should go through exactly the same processing pipeline as the text for the training data. The key difference is the use of a duration model at synthesis time, in place of forced alignment for the training data.

[Etienne D]

Hi,

I’ve managed to get the right label dimensionality by retraining the durational model with the subphone_feats option set to full and the add_frame_features option set to True (as they are with the acoustic model), and also retrained the acoustic model after spotting an unrelated error.
I am now working with labels of dimension 456 with both durational and acoustic model.
However, I am getting an error at the TRAINDNN stage of the durational model, complaining about the different number of frames in labels and acoustic features:

2016-07-25 19:09:57,495 INFO acoustic_comp: processing file 2001 of 2002 : /mnt/courses.homes/s1571333/Documents/dna_synth/ss_dnn/data/nn_dur_5/herald_998.cmp exists
2016-07-25 19:09:57,495 INFO acoustic_comp: processing file 2002 of 2002 : /mnt/courses.homes/s1571333/Documents/dna_synth/ss_dnn/data/nn_dur_5/herald_999.cmp exists
2016-07-25 19:10:06,086 INFO main : normalising acoustic (output) features using method MVN
2016-07-25 19:10:11,819 INFO main : saved MVN vectors to /mnt/courses.homes/s1571333/Documents/dna_synth/ss_dnn/data/norm_info_dur_5_MVN.dat
2016-07-25 19:10:11,824 INFO main : saved dur variance vector to /mnt/courses.homes/s1571333/Documents/dna_synth/ss_dnn/data/var/dur_5
2016-07-25 19:10:11,851 DEBUG labels : HTS-derived input feature dimension is 447 + 9 = 456
2016-07-25 19:10:11,851 INFO main : label dimension is 456
2016-07-25 19:10:11,852 INFO main : training DNN
2016-07-25 19:10:11,854 DEBUG main.train_DNN: Starting train_DNN
2016-07-25 19:10:11,854 DEBUG main.train_DNN: Creating training data provider
2016-07-25 19:10:11,856 DEBUG main.train_DNN: Creating validation data provider
2016-07-25 19:10:11,858 CRITICALListDataProvider: the number of frames in label and acoustic features are different: 378 vs 0
378
0
2016-07-25 19:10:11,858 CRITICAL main : train_DNN threw an exception
Traceback (most recent call last):
File “/mnt/courses.homes/s1571333/Documents/dna_synth/dnn_tts/run_lstm.py”, line 1091, in <module>
main_function(cfg)
File “/mnt/courses.homes/s1571333/Documents/dna_synth/dnn_tts/run_lstm.py”, line 808, in main_function
cmp_mean_vector = cmp_mean_vector, cmp_std_vector = cmp_std_vector)
File “/mnt/courses.homes/s1571333/Documents/dna_synth/dnn_tts/run_lstm.py”, line 201, in train_DNN
shared_train_set_xy, temp_train_set_x, temp_train_set_y = train_data_reader.load_one_partition()
File “/mnt/courses.homes/s1571333/Documents/dna_synth/dnn_tts/utils/providers.py”, line 155, in load_one_partition
shared_set_xy, temp_set_x, temp_set_y = self.load_next_partition()
File “/mnt/courses.homes/s1571333/Documents/dna_synth/dnn_tts/run_lstm.py”, line 792, in main_function
os.makedirs(model_dir)
File “/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py”, line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 17] File exists: ‘/mnt/courses.homes/s1571333/Documents/dna_synth/ss_dnn/nnets_model’

Would you have any pointers as to where I have messed up?
Thank you for your time

[Simon]

This is the error

2016-07-25 19:10:11,858 CRITICALListDataProvider: the number of frames in label and acoustic features are different: 378 vs 0

Having 0 frames is obviously a silly bug somewhere – inspect your files carefully. Reduce the validation set to a single sentence, reproduce the error, then inspect all the relevant files for that sentence.

[Srikanth R]

You can’t use same configuration file for both acoustic and duration model.

Please check below file for duration model configuration:
https://svn.ecdf.ed.ac.uk/repo/inf/dnn_tts/configuration/duration_configfile.conf

Please check below file for acoustic model configuration:
https://svn.ecdf.ed.ac.uk/repo/inf/dnn_tts/configuration/acoustic_configfile.conf

[Author]

Posts