Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

In bundled python package installation is performed in user location

See original GitHub issue


When the download and install .deb package and try run pip install using pip from bundled python there is information that the package will be installed in user mode because site-package is not writable.

Packages installed in user mode will be available in all python3.8 environments and may introduce unexpected side effects.

pip bundled in app should use different location to install a new packages.

Zrzut ekranu z 2021-09-23 12-26-04

Proposed Solution

set alternative install location in user directory when run in non administrator mode and add this location to PYTHONPATH

Issue Analytics

  • State:open
  • Created 2 years ago
  • Comments:10 (1 by maintainers)

github_iconTop GitHub Comments

jpivarskicommented, Sep 25, 2021

I’m watching this issue, too, because it’s the one capability that’s necessary to make jupyterlab_app useful. (What’s Python without packages?)

As I understand the current status,

  • running pip install X in the JupyterLab terminal or in a notebook cell starting with ! runs pip in some global mode, putting the installed package in ~/.local, rather than in the conda environment
  • running conda install X in the JupyterLab terminal or in a notebook cell starting with ! doesn’t work at all because the install area is not writable.

Oddly, considering this error, the pip and conda commands are coming from the right place (this output came from the JupyterLab terminal; the output from a notebook cell is the same):

$ which pip
$ which conda

Aha! The problem is that the site-packages is not writable, and so that’s why pip is deferring to ~/.local:

$ pip install uproot
Defaulting to user installation because normal site-packages is not writeable
Collecting uproot
  Using cached uproot-4.1.2-py2.py3-none-any.whl (290 kB)
Requirement already satisfied: numpy in /opt/JupyterLab/resources/jlab_server/lib/python3.8/site-packages (from uproot) (1.21.2)
Requirement already satisfied: setuptools in /opt/JupyterLab/resources/jlab_server/lib/python3.8/site-packages (from uproot) (58.0.4)
Installing collected packages: uproot
Successfully installed uproot-4.1.2

We see where it is when uninstalling the package:

$ pip uninstall uproot
Found existing installation: uproot 4.1.2
Uninstalling uproot-4.1.2:
  Would remove:
Proceed (Y/n)? y
  Successfully uninstalled uproot-4.1.2

The error message from conda seems to be from the same underlying cause:

$ conda install uproot
Collecting package metadata (current_repodata.json): failed

NotWritableError: The current user does not have write permissions to a required path.
  path: /opt/JupyterLab/resources/jlab_server/pkgs/urls.txt
  uid: 1000
  gid: 1000

If you feel that permissions on this path are set incorrectly, you can manually
change them by executing

  $ sudo chown 1000:1000 /opt/JupyterLab/resources/jlab_server/pkgs/urls.txt

In general, it's not advisable to use 'sudo conda'.

Interestingly, the conda in JupyterLab_app sees my normal conda environments:

$ conda env info
usage: conda-env [-h] {create,export,list,remove,update,config} ...
conda-env: error: invalid choice: 'info' (choose from 'create', 'export', 'list', 'remove', 'update', 'config')
$ conda env list
# conda environments:
base                  *  /opt/JupyterLab/resources/jlab_server

But the conda I use outside of JuptyerLab_app, in a normal terminal, for instance, does not see the JupyterLab_app one:

% conda env list
# conda environments:
base                  *  /home/jpivarski/miniconda3
np13                     /home/jpivarski/miniconda3/envs/np13
np21                     /home/jpivarski/miniconda3/envs/np21
numba-py38               /home/jpivarski/miniconda3/envs/numba-py38
pandas-0.22              /home/jpivarski/miniconda3/envs/pandas-0.22
py27                     /home/jpivarski/miniconda3/envs/py27
py39                     /home/jpivarski/miniconda3/envs/py39
rapids                   /home/jpivarski/miniconda3/envs/rapids
test-fastjet             /home/jpivarski/miniconda3/envs/test-fastjet

If there’s some way now to pip and conda into /opt/JupyterLab/resources/jlab_server, I’d love to know it! If not, this is a bug-fix/feature-request. Thanks!

(Or do I just have to install it not from the .deb? That seemed like the appropriate way to install it, as I’m an end-user, not a developer.)

Czakicommented, Sep 23, 2021

@Czaki Ah… seems it is there but I had to restart the kernel before I could see the newly installed packages, whereas typically (for me at least…) packages seem to be available without the need to restart (although a warning that a kernel restart may be required is typically given.)

If I good remember it was caused by installation outside site-packages. I meet this when performing installation in developer mode (pip install -e).

Read more comments on GitHub >

github_iconTop Results From Across the Web

Installing Packages - Python Packaging User Guide -
Python “Virtual Environments” allow Python packages to be installed in an isolated location for a particular application, rather than being installed globally.
Read more >
Python Package Installation on Windows - ActiveState
To manually install packages in Python with, do the following: Download the package and extract it into a local directory. Navigate to ......
Read more >
How to bundle a Python application including dependencies?
Python is installed in the standard location; the package and its dependencies are installed in a separate directory (possibly site-packages); the installation ...
Read more >
Packaging applications to install on other machines with Python
setuptools solves the problem of packaging your application in a nice directory structure for making scripts and libraries available to users.
Read more >
Using Python's pip to Manage Your Projects' Dependencies
By chaining the packages rptree and codetiming in the pip install command, you install both packages at once. You can add as many...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found