Bottleneck features as additional input to DNN

[Giorgia M]

Hello,

By adding the variable GENBNFEA: True in the config file, we have managed to train the bottleneck.py and extracted the .cmp features from the bottleneck layer. Now we have a .cmp file for each input and we want to combine it with the normal input to the DNN.

Question 1: We believe that we need to split the bottleneck .cmp files, as we did for normal training. Is this correct? If yes, how do we define the widths dimensions? We believe that bap and lf0 will remain the same as normal training. Nevertheless, although in our config file we have specified mgc to be 60, we are getting the assertion error from line 71 in split_cmp.py, supposedly due to the width of mgc,

Question 2: How do we combine the bottleneck input with the normal input? In the config file, do we need to specify another input folder for the bottleneck features? We cannot have them in the same folder as the normal input because they would have the same name.

Thanks

[Srikanth R]

Answer to Q1: run_lstm.py has a well-defined variable for additional input (appended_input_dim) but run_dnn.py is used to be hard-coded for the combined input dimension. The output remains same: mgc with 60 and dmgc with 180.

Answer to Q2: At the moment, the bottleneck code is not completely implemented within Merlin. Yes, we need to have different folders: one for the label input and one for the bottleneck. The stacking of features is done outside Merlin, using independent scripts.

[Author]

Posts