"No such file or directory" when creating Jupyter notebook from scratch

[ProgramItUp]

I'm running into a problem with a notebook I've created from scratch in Jupyter that I'm not getting with a with the included example hyperas/examples/simple_notebook.ipynb.

It appears that hyperas is looking for a temperary file that does not exist.

Any thoughts about what is going wrong?

Steps to recreate the error:

1. Create new Jupyter notebook
2. Copy the sample code from the hyperas README and pasted it into the notebook created in 1)
3. Selected "Cell->Run All"
4. Error is thrown: See below
5. Selected "Kernel -> Restart and Run All"
6. Error is thrown: same as 4)
7. "File -> Save and Checkpoint", "Kernel -> Restart and Run All"
8. Error is thrown: same as 4) and 6)

Using TensorFlow backend.
/usr/local/lib/python2.7/dist-packages/hyperas/optim.py
/usr/local/lib/python2.7/dist-packages/hyperas/optim.py
/usr/local/lib/python2.7/dist-packages/hyperas/optim.py
<ipython-input-1-43ea71f3309d>
/usr/local/lib/python2.7/dist-packages/IPython/core/interactiveshell.py
/usr/local/lib/python2.7/dist-packages/IPython/core/interactiveshell.py
/usr/local/lib/python2.7/dist-packages/IPython/core/interactiveshell.py
/usr/local/lib/python2.7/dist-packages/ipykernel/zmqshell.py
/usr/local/lib/python2.7/dist-packages/ipykernel/ipkernel.py
/usr/local/lib/python2.7/dist-packages/ipykernel/kernelbase.py
/usr/local/lib/python2.7/dist-packages/ipykernel/kernelbase.py
/usr/local/lib/python2.7/dist-packages/ipykernel/kernelbase.py
/usr/local/lib/python2.7/dist-packages/tornado/stack_context.py
/usr/local/lib/python2.7/dist-packages/zmq/eventloop/zmqstream.py
/usr/local/lib/python2.7/dist-packages/zmq/eventloop/zmqstream.py
/usr/local/lib/python2.7/dist-packages/zmq/eventloop/zmqstream.py
/usr/local/lib/python2.7/dist-packages/tornado/stack_context.py
/usr/local/lib/python2.7/dist-packages/tornado/ioloop.py
/usr/local/lib/python2.7/dist-packages/zmq/eventloop/ioloop.py
/usr/local/lib/python2.7/dist-packages/ipykernel/kernelapp.py
/usr/local/lib/python2.7/dist-packages/traitlets/config/application.py
/usr/local/lib/python2.7/dist-packages/ipykernel_launcher.py
/usr/lib/python2.7/runpy.py
/usr/lib/python2.7/runpy.py
---------------------------------------------------------------------------
IOError                                   Traceback (most recent call last)
<ipython-input-1-43ea71f3309d> in <module>()
     80                                           algo=tpe.suggest,
     81                                           max_evals=5,
---> 82                                           trials=Trials())
     83     X_train, Y_train, X_test, Y_test = data()
     84     print("Evalutation of best performing model:")

/usr/local/lib/python2.7/dist-packages/hyperas/optim.pyc in minimize(model, data, algo, max_evals, trials, rseed, notebook_name, verbose)
     40     best_run = base_minimizer(model=model, data=data, algo=algo, max_evals=max_evals,
     41                               trials=trials, rseed=rseed, full_model_string=None,
---> 42                               notebook_name=notebook_name, verbose=verbose)
     43 
     44     best_model = None

/usr/local/lib/python2.7/dist-packages/hyperas/optim.pyc in base_minimizer(model, data, algo, max_evals, trials, rseed, full_model_string, notebook_name, verbose, stack)
     60         model_str = full_model_string
     61     else:
---> 62         model_str = get_hyperopt_model_string(model, data, notebook_name, verbose, stack)
     63     temp_file = './temp_model.py'
     64     write_temp_files(model_str, temp_file)

/usr/local/lib/python2.7/dist-packages/hyperas/optim.pyc in get_hyperopt_model_string(model, data, notebook_name, verbose, stack)
    126             print(i[1])
    127         calling_script_file = os.path.abspath(inspect.stack()[stack][1])
--> 128         with open(calling_script_file, 'r') as f:
    129             source = f.read()
    130 

IOError: [Errno 2] No such file or directory: '/work/models/regression/Keras-Nonlinear-Regression/<ipython-input-1-43ea71f3309d>'

[yakouyang]

If you run your code in jupyter, function "optim.minimize" must contain a parameter "notebook_name".
As the example "simple_notebook.ipynb":

best_run, best_model = optim.minimize(model=model,
                                      data=data,
                                      algo=tpe.suggest,
                                      max_evals=5,
                                      trials=Trials(),
                                      notebook_name='simple_notebook')

[mahacoder]

This fix does not work in Google colab. Has anybody figured out how to run it in colab yet?

[maxpumperla]

@mahacoder I don't think this will work (although I'm not sure). you'd be better off using plain hyperopt in this case.

[Naren-Sridharan]

@mahacoder I think it works after I give the path to the .ipynb file. I am getting a different error after this though.

[lurenyi233]

@mahacoder It also works.

[pengzhao001]

The above solutions do not work for me. Here is my solution. The working directory assigned by colab is /content/. You just need to download the current notebook from google drive and upload it to /content/ and it should work.

[kenfus]

Any updates on this? I tried all the possibilities but it just doesn't work. It can't find the notebook, even though the path is right. I'm also loading CSV Data from the exact same folder which works flawlessly.

[charlesreid1]

Google Drive is no longer automatically mounted/available to the instance running the CoLab notebook. Before you can access files in your Google Drive, you have to mount your Google Drive in the CoLab instance.

The following code will do that. You'll be given an OAuth url to visit. You log in with your Google account, then copy and paste the authorization code into the notebook cell. Then your Google Drive will be mounted at /gdrive/My Drive/.

from google.colab import drive
drive.mount('/gdrive')
%ls /gdrive

hyperas automatically appends the current working directory to the notebook_name parameter, so you'll need to specify the notebook_name parameter as follows:

notebook_name=os.path.join('..','gdrive','My Drive','Colab Notebooks','example_hyperas')

Here is a link to the example from the hyperas readme in a working Google Colab notebook: https://gist.github.com/charlesreid1/3ea4561566f6e9277c29ad242cee9811

[tgautam03]

I used the exact syntax as mentioned in the link and I get the error :

File "<unknown>", line 387
    o teaches developers how to get results with modern machine learning methods via han
            ^
SyntaxError: invalid syntax

[qemtek]

I get this issue when using hyperas in my PyCharm REPL session.. Is there any way around it? Its a shame as I usually develop in the live session..