Crash when running jupyter notebook with a user with id != 1000

[csala]

Apparently, jupyter notebook command attempts to write something on folder /run/user/1000 which belongs to user with id 1000.
If jupyter notebook is run with a different user, the following stacktrace appears:

Traceback (most recent call last):
  File "/opt/jupyter/bin/jupyter-notebook", line 11, in <module>
    sys.exit(main())
  File "/opt/jupyter/local/lib/python2.7/site-packages/jupyter_core/application.py", line 267, in launch_instance
    return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
  File "/opt/jupyter/local/lib/python2.7/site-packages/traitlets/config/application.py", line 595, in launch_instance
    app.initialize(argv)
  File "<decorator-gen-7>", line 2, in initialize
  File "/opt/jupyter/local/lib/python2.7/site-packages/traitlets/config/application.py", line 74, in catch_config_error
    return method(app, *args, **kwargs)
  File "/opt/jupyter/local/lib/python2.7/site-packages/notebook/notebookapp.py", line 1021, in initialize
    self.init_configurables()
  File "/opt/jupyter/local/lib/python2.7/site-packages/notebook/notebookapp.py", line 815, in init_configurables
    connection_dir=self.runtime_dir,
  File "/opt/jupyter/local/lib/python2.7/site-packages/traitlets/traitlets.py", line 529, in __get__
    return self.get(obj, cls)
  File "/opt/jupyter/local/lib/python2.7/site-packages/traitlets/traitlets.py", line 508, in get
    value = self._validate(obj, dynamic_default())
  File "/opt/jupyter/local/lib/python2.7/site-packages/jupyter_core/application.py", line 99, in _runtime_dir_default
    ensure_dir_exists(rd, mode=0o700)
  File "/opt/jupyter/local/lib/python2.7/site-packages/ipython_genutils/path.py", line 167, in ensure_dir_exists
    os.makedirs(path, mode=mode)
  File "/opt/jupyter/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/run/user/1000/jupyter'

[csala]

Actually, I found out something still more interesting: The id which jupyter uses to decide which folder it should write in (this /run/user/{id}), seems to be the id of the user which the session was initially open with.

So, if I ssh into a host using a user which has id X and then I sudo su - anotheruser, when I run jupyter from user anotheruser the folder it uses is /run/user/X

[takluyver]

This is coming from the environment variable $XDG_RUNTIME_DIR. Normally, when you log in, that is set to a newly created directory which only your user has access to. I'm guessing that it doesn't get set again when you use sudo to change user.

I think we have a fallback if it's not set, so the easiest thing might be to unset the environment variable after sudo-ing. In fact, I thought sudo dropped environment variables by default. :-/

[jesseHFG]

I had the same problem. export XDG_RUNTIME_DIR="" eliminated the problem.

[takluyver]

You can also (at least in bash) unset XDG_RUNTIME_DIR for the same effect.

[GISDev01]

unset XDG_RUNTIME_DIR

Worked for me as well. Need to run that before starting juypterhub after sudo-ing into our jupyterapp-specific account.

[lpryszcz]

I have also encountered similar problem. Thanks for the hint.
Note, you can add it to jupyter user ~/.bashrc, then it will load on every login echo 'unset XDG_RUNTIME_DIR' >> ~/.bashrc
https://bioinfoexpert.com/2016/06/24/running-jupyter-as-public-service/

[georgeha]

worked for me too. Great!

[matrs]

Why is this closed if it's still a problem?
I just installed anaconda in arch linux as a regular user, there is no sudo involved and i got this error. no error for su user. Fixed by unsettling $XDG_RUNTIME_DIR

[takluyver]

What is XDG_RUNTIME_DIR set to in that case?

I believe that if XDG_RUNTIME_DIR is set to a location that is not writable, that's a problem with your environment. It's supposed to be set at login to a directory which is r+w for your user and your user only.

[matrs]

The problem is that, the folder set by jupyter notebook is only writable by SU, but that shouldn't happen because i'm starting the notebook as a regular user and also i'm logged in as regular user. This didn't happen when i used ipython/jupyter from my distro repo (i used that for about two years).

The folder is the same as is in the OP, /run/user/1000/jupyter

[matrs]

So I was installing anaconda in a new partition owned by the root user, even though everybody was able to write in it. I changed the owner to my regular user and now i don't have the problem anymore. Previously, the folder /run/user/1000/jupyter was owned by root and by definition that folder is only writable by the target user. Now is owned by my regular user.

[acharles7]

unset XDG_RUNTIME_DIR
type above in your terminal and you are good to go.

[flgomezc]

Export XDG_RUNTIME_DIR worked for me!.

I followed the remote jupyter tutorial from AlexanderLabWHOI (https://alexanderlabwhoi.github.io/post/2019-03-08_jpn-slurm/)

I run jupyter in a Surm cluster. I wrote there a function in ~/.bash_profile to call Jupyter with the specified port:

function jpt(){
mkdir run
export XDG_RUNTIME_DIR=/home/my_username/run/
jupyter notebook --no-browser --port=$1
}
In my local machine I have another function in my ~/.bashrc file:

function jptnode(){
ssh -t -t my_username@hpc.address.edu -L 8888:localhost:8888 ssh node1 -L 8888:localhost:8888
}

[FilipeMaia]

I think the issue is that XDG_RUNTIME_DIR requires a level of desktop packages to be installed (not sure exactly which ones yet), which will not be the case on computational nodes in a cluster for example.

[FilipeMaia]

And the XDG_RUNTIME_DIR can be set on a login node which does have the required packages and that environment variable is then exported by slurm to when using batchspawner.

[takluyver]

Jupyter no longer uses XDG_RUNTIME_DIR since jupyter-core version 4.5, released in June 2019. This kind of problem was part of the rationale for that. See jupyter/jupyter_core#143 for details.