Kernelspecs with non-default arguments don't work.

Environment data

• VS Code version: 1.59.0
• Jupyter Extension version (available under the Extensions sidebar): 2021.8.11
• Python Extension version (available under the Extensions sidebar): 2021.8.1
• OS (Windows | Mac | Linux distro) and version: Ubuntu 20.04.02 LTS (WSL2)
• Python version: 3.7.11
• Type of virtual environment used (N/A | venv | virtualenv | conda | …): poetry, same issue with system python environment
• Jupyter server running: Remote

Expected behaviour

According to jupyter documentation kernel.json must contain certain keys and values.

argv: A list of command line arguments used to start the kernel. The text {connection_file} in any argument will be replaced with the path to the connection file.

I expect that my custom kernel.json (language=python) can also be launched in Interactive Window mode of this extension with the following argv.

python main.py --something --something-else -f {connection_file}

Note that {connection_file} argument exists as described in kernel spec.

Actual behaviour

Extension fails with the following notification:

Unsupported KernelSpec file. args must be [<pythonPath>, '-m', <moduleName>, arg1, arg2, ..]. 
Provied python /path/to/main.py --something --something-else --ip=127.0.0.1 --stdin=9058 
--control=9056 --hb=9055 --Session.signature_scheme="hmac-sha256" 
--Session.key=b"d6623136-342f-460e-816c-edc0fcdb5712" --shell=9057 --transport="tcp" 
--iopub=9059 --f=/tmp/tmp-32343hby0MHCz6WSc.json

The extension expects that a python kernel should always be started using python -m module_name which as far as I understand does not adhere to the generic nature of kernelspec definitions.

---

• main.py is just an example to show that I want to use python script rather than a python module
• main.py accepts -f argument as well as --ip, --iopub etc
• main.py script handles --something and --something-else arguments first and then starts an ipython kernel by calling IPython.embed_kernel(cfg).
  • --something* does some network and logging customization I need, and that is the reason why I want to start up the kernel as a script
  • Where cfg is an instance of traitlets.config.loader.Config that contains ip, ports, transport, key etc that the script received from {connection_file} or from command line arguments directly.

I know the embed_kernel() works with this extension because when I symbolically link this main.py file to a python file in venv/lib/python3.7/site-packages/main_link.py directory and use python -m main_link in kernel.json, the extension and interactive window works just as expected.

However, I think it would be a correct implementation if vscode-jupyter extension handles argv as it was defined in a kernel.json instead of assuming all python kernels would be started with python -m module_name.

Steps to reproduce

Assuming jupyter package is installed in a virtual environment.

1. Create a new directory called custom_kernel at ./venv/share/jupyter/kernels
2. Copy the contents of ./venv/share/jupyter/kernels/python3 to ./venv/share/jupyter/kernels/custom_kernel (copies kernel.json and icon*.png)
3. Edit ./venv/share/jupyter/kernels/custom_kernel/kernel.json to have the following { "argv": [ "python", "/path/to/main.py", "--something", "--something-else", "--f", "{connection_file}" ], "display_name": "custom-kernel", "language": "python", "metadata": { "debugger": true } }
4. Restart VS Code, create new interactive window and change kernel to custom-kernel

Logs

mainThreadExtensionService.ts:63

[[object Object]]Unsupported KernelSpec file. args must be [<pythonPath>, '-m', <moduleName>, arg1, arg2, ..]. Provied python main.py --something --something_else --ip=127.0.0.1 --stdin=9071 --control=9064 --hb=9063 --Session.signature_scheme="hmac-sha256" --Session.key=b"3a80d972-d49b-4064-b0b2-a3a3750cd411" --shell=9065 --transport="tcp" --iopub=9072 --f=/tmp/tmp-323431mTrva1GuxNz.json
$onExtensionRuntimeError @ mainThreadExtensionService.ts:63
_doInvokeHandler @ rpcProtocol.ts:418
_invokeHandler @ rpcProtocol.ts:403
_receiveRequest @ rpcProtocol.ts:319
_receiveOneMessage @ rpcProtocol.ts:246
(anonymous) @ rpcProtocol.ts:111
fire @ event.ts:577
fire @ ipc.net.ts:513
_receiveMessage @ ipc.net.ts:866
(anonymous) @ ipc.net.ts:705
fire @ event.ts:577
acceptChunk @ ipc.net.ts:286
(anonymous) @ ipc.net.ts:247
(anonymous) @ browserSocketFactory.ts:197
fire @ event.ts:577
_fileReader.onload @ browserSocketFactory.ts:81

mainThreadExtensionService.ts:64

Error: Unsupported KernelSpec file. args must be [<pythonPath>, '-m', <moduleName>, arg1, arg2, ..]. Provied python main.py --something --something_else --ip=127.0.0.1 --stdin=9071 --control=9064 --hb=9063 --Session.signature_scheme="hmac-sha256" --Session.key=b"3a80d972-d49b-4064-b0b2-a3a3750cd411" --shell=9065 --transport="tcp" --iopub=9072 --f=/tmp/tmp-323431mTrva1GuxNz.json
	at f.launch (vscode-file://vscode-app/home/tars/.vscode-server/extensions/ms-toolsai.jupyter-2021.8.1195043623/out/client/extension.js:52)
$onExtensionRuntimeError @ mainThreadExtensionService.ts:64
_doInvokeHandler @ rpcProtocol.ts:418
_invokeHandler @ rpcProtocol.ts:403
_receiveRequest @ rpcProtocol.ts:319
_receiveOneMessage @ rpcProtocol.ts:246
(anonymous) @ rpcProtocol.ts:111
fire @ event.ts:577
fire @ ipc.net.ts:513
_receiveMessage @ ipc.net.ts:866
(anonymous) @ ipc.net.ts:705
fire @ event.ts:577
acceptChunk @ ipc.net.ts:286
(anonymous) @ ipc.net.ts:247
(anonymous) @ browserSocketFactory.ts:197
fire @ event.ts:577
_fileReader.onload @ browserSocketFactory.ts:81

[DonJayamanne]

I’m sorry for some reason the extesnion was’t published, and should be in the marketplace now. Please update the Jupyter extesnion installed to the latest (currently latest is 2021.10.1001338521)

Appreciate your patience in this matter and do let me know if this fixes it.

[rchiodo]

This is a result of our kernel daemon we use to speedup kernel launching (we have a process already running that we pass the arguments to, and it starts ipykernel for us). It’s not handling this case and should probably be skipped in this scenario.

The line throwing the exception is here: https://github.com/microsoft/vscode-jupyter/blob/bd63750c91e6e23dacdfb99c24d88086e0d1acd4/src/client/datascience/kernel-launcher/kernelLauncherDaemon.ts#L60