Running Merlin on Eddie3

[Zhiyao L]

I kept running into an error (the output file and error file as attached) attempting to train the models with merlin on eddie3. The python scripts and bash scripts, as well as submission has been all right. I have changed the merlin script called by

05_train_duration_model.sh

from run_merlin.py to other backends like Keras and Tensorflow:

./scripts/submit.sh ${MerlinDir}/src/run_keras_with_merlin_io.py $duration_conf_file

None worked. It seems that Eddie3 and python scripts were both all right. And there is no disambiguous error message on the .e27xxxxx error file. The process quitted with 0 (meaning no error), but the training process was never run (.o27xxxxx file showed that the process ended in 0min 0sec). This is strange.

[Simon King]

The attached files didn’t upload (you need to add a .txt ending to get around the security on the forum). To debug on Eddie, do not submit jobs to the queue, but instead use an interactive session (using qlogin with appropriate flags to get on to a suitable GPU node).

But, talk to classmates first – several people are attempting to use Eddie and you should share knowledge and effort. Also, look at the Informatics MLP cluster, where people have got Merlin working – see

https://www.wiki.ed.ac.uk/display/CSTR/CSTR+computing+resources

[Zhiyao L]

Another upload attempt

Attachments:

You must be logged in to view attached files.

[Simon King]

Looks like you have all processes set to “False” which means nothing will be done (other than loading the config files and writing some log output).

[Simon King]

You should get everything working on a DICE machine, using just the CPU and a small data set, before attempting to use Eddie. Have you done that?

[Author]

Posts