Details on the A/B experiment removing `python.pythonPath` support

Summary of the situation

For security reasons we need to not automatically trust the python.pythonPath setting when it originates from a .vscode/settings.json file in a workspace. We also had a long-standing request to not write a user’s environment path to .vscode/settings.json to allow for it to be committed to version control without be specific to any one user’s machine.

We set up an A/B experiment to measure impact on users in dropping support for python.pythonPath. This issue is a response from users who were negatively impacted. We are working with various teams within Microsoft to try and come up with a situation where we can bring back some semblance of the old semantics while still providing the level of security and safety we need to.

To stay updated on our work to resolve the situation, please feel free to subscribe to this issue – there’s a “Subscribe” button in the right column – and we will post updates to this issue as things develop.

Related issues:

• https://github.com/microsoft/vscode-python/issues/7805
• https://github.com/microsoft/vscode-python/issues/2125

Wiki: https://github.com/microsoft/vscode-python/wiki/AB-Experiments

Opting out of the experimental semantics

To opt out of the experiment, you can set “python.experiments.optOutFrom” to either the specific experiment or to ["All"] to opt out of all experiments. See https://github.com/microsoft/vscode-python/wiki/AB-Experiments#faq for more details.

Original post

_Originally posted by @zor-el in https://github.com/microsoft/vscode-python/pull/12002#issuecomment-643187987_

I’m posting this coming from #2125 because that issue is referenced from the announcing blog post.

Complete removal of python.pythonPath from the settings is a massively breaking change that IMO isn’t warranted by the #2125 issue that has been used to justify it in the above blog post. (This change has certainly been breaking things left and right for me.) The explanation in that blog post’s comment that “it can be useful” doesn’t invalidate my argument. A breaking change is harmful, so it should be significantly outweighed by a benefit that cannot by obtained otherwise. This is not the case here though, IMO.

Although it is true that pythonPath would likely be different for different people, I’d argue that is true for the entire (or most of) .vscode/settings.json. For example, extensions can and do store settings in .vscode/settings.json that may just as well conflict with other users.

Settings are not generally transferable from user to user and from machine to machine, so there is little point in sharing them. In those cases that one would want to share them, one would certainly want to separate transferable from non-transferable settings.

Note the OP’s issue wasn’t the existence of python.pythonPath, but the unsolicited modification of .vscode/settings.json, which sneaks in non-transferable content into (in their case) shared settings.

Apparently the underlying issue is the lack of means for separation of transferable from non-transferable settings. That separation is inadequately addressed by removal of pythonPath (it is a general issue, not one specific to Python). The concrete issue #2125 is not about pythonPath removal either but about unsolicited modification of .vscode/settings.json. That is just as inadequately addressed by mere removal of pythonPath (IMO it could have been solved without breaking everybody’s settings).

Question: Why can we not keep python.pythonPath for backward compatibility and simply ask the user for permission to overwrite it (and hint at the fact that the setting may not be transferable)?

Perhaps even the experiment’s pythonPath caching behavior could remain intact, but an explicit pythonPath setting could take precedence?

[fredvd]

And on topic: I use VS code to work on projects where we use zc.buildout. It’s common practice to have a virtualenv inside every project folder to have an isolated environment. We have even some tooling which can generate/modify the settings.json.

I was never able (since 2019) to get VS code to pick up the correct virtualenv / venv Python automatically. That was until I started setting my pythonPath explicitly in the settings.json using:

"python.pythonPath": "${workspaceFolder}/bin/python3.7"

The settings.json can be shared with other team members or put in VCS as we all have our Python inside the project and everything is relative/inside the workspace. zc.buildout also generates a list with paths of the used Python modules, which are written to an .env file. actually this is also called PYTHONPATH . In my settings.json I use python.envFile for that:

"python.envFile": "${workspaceFolder}/.vscode/.env",

I agree with most other peoples feedback here: if you want to improve the automatic detection (which worked very poorly so far in my experience), do this by all means. But why remove a setting which is used actively by a significant group of users who scaffold the IDE settings using automation tools? Without any fallback?

[FlyingRedDonkey]

I don’t like this change.

It would have been preferable if a better workaround to the original issue was released first before breaking python.pythonPath. Or perhaps just leave it be; it ain’t hurtin’ nobody by existing.