-
Notifications
You must be signed in to change notification settings - Fork 11
Open
Description
Issue: Supervised.sh
Hi!
I'm trying to use the program for the UKBB dataset but was encountering numerous issues due to specific/ old versions of programs/ modules used.
I ended up Dockerizing
the program with identical versions of all the tools listed using CONDA and PIP, but I still ran into trouble that seems out of the purview of pure code but rather the model generation method (?)
Error log:
/miniconda/envs/ukbb/lib/python3.6/site-packages/torch/nn/modules/rnn.py:38: UserWarning: dropout option adds dropout after all but last recurrent layer, so non-zero dropout expects num_layers greater than 1, but got dropout=0.1 and num_layers=1
"num_layers={}".format(dropout, num_layers))
Traceback (most recent call last):
File "ukb/train.py", line 368, in <module>
main(args)
File "ukb/train.py", line 246, in main
verbose=args.verbose)
File "/software/ukb/utils/utils.py", line 65, in timed
result = method(*args, **kw)
File "/software/ukb/models/trainer.py", line 426, in fit
checkpoint_freq=checkpoint_freq, checkpoint_dir=checkpoint_dir, **params
File "/software/ukb/utils/utils.py", line 65, in timed
result = method(*args, **kw)
File "/software/ukb/models/trainer.py", line 125, in fit
model = self.model_class(**params) if not self.use_cuda else self.model_class(**params).cuda()
File "/miniconda/envs/ukbb/lib/python3.6/site-packages/torch/nn/modules/module.py", line 249, in cuda
return self._apply(lambda t: t.cuda(device))
File "/miniconda/envs/ukbb/lib/python3.6/site-packages/torch/nn/modules/module.py", line 176, in _apply
module._apply(fn)
File "/miniconda/envs/ukbb/lib/python3.6/site-packages/torch/nn/modules/module.py", line 176, in _apply
module._apply(fn)
File "/miniconda/envs/ukbb/lib/python3.6/site-packages/torch/nn/modules/rnn.py", line 112, in _apply
self.flatten_parameters()
File "/miniconda/envs/ukbb/lib/python3.6/site-packages/torch/nn/modules/rnn.py", line 105, in flatten_parameters
self.batch_first, bool(self.bidirectional))
RuntimeError: CUDNN_STATUS_EXECUTION_FAILED
Main takeaway from the error:
dropout option adds dropout after all but last recurrent layer, so non-zero dropout expects num_layers greater than 1, but got dropout=0.1 and num_layers=1
Using NVIDIA Cuda with specified versions on an Arch Linux host running Ubuntu 20.04 Docker image.
Any pointers?
Thank you!
Metadata
Metadata
Assignees
Labels
No labels