!pip not installing packages as expected

I recently installed Python 3.9.6 from python.org and tried to run notebook against this interpreter but ran into a couple of issues.

Problem

• Install Python 3.9.6, which was not added to the PATH
• Open a notebook, Python 3.9.6 is listed in kernel list, select this kernel
• Run import pandas as pd, it prompts to ask to install ipykernel
• After ipykernel is installed, rerun the cell, we get error “pandas is missing”
• Insert a new cell, run !pip install pandas, it works but it installs the pandas package to python 2.7 as that the interpreter in the PATH

Proposal

The underling problem is the disconnect between Python active interpreter, Jupyter kernel, and the python interpreter in the PATH. If users are using pyenv, venv, or conda, the extension will activate the environment, which will set up the environment properly, however we don’t have such thing for the vanilla Python interpreter.

We should probably

• Show a warning for !pip if the pip in the PATH is not the one in the interpreter behind the selected kernel
• Automatically add the pip from the interpreter of the selected kernel to the PATH temporarily

[rchiodo]

Personally I think we should put up an error if the user has !pip or !conda as they don’t do what the user expects.

[DonJayamanne]

Suggestion:

• Add a diagnostic error for usage of !pip and recommend using %pip or %conda if its a conda environment.