Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Data viewer crashes when inspecting jupyter data frame variable

See original GitHub issue

Applies To

Notebooks (.ipynb files)
Interactive Window and/or Cell Scripts (.py files with #%% markers)

What happened?

Data viewer crashes on any spark data frame variable, regardless of whether it contains data or was created out of an empty list. I use databricks-connect 9.1.13 for spark.

Does not apply to

normal .py files (without # %% markers)
spark data frames, that have been converted to pandas data frames
my other windows machine (this is a mac)

Urls and pathes have been anonymized

VS Code Version

1.66.2

Jupyter Extension Version

v2022.3.1000901801

Jupyter logs

Info 14:46:55: Kernel Output: NOTE: When using the `ipython kernel` entry point, Ctrl-C will not work.

To exit, you will have to explicitly quit this process, by either sending
"quit" from a client, or using Ctrl-\ in UNIX-like environments.

To read more about this, see https://github.com/ipython/ipython/issues/2049


To connect another client to this kernel, use:
    --existing /var/folders/lt/gnj5lvrn7f926knh9ltmyhhc0000gn/T/tmp-18075p906p9AwDRD1.json
View job details at https://westeurope.azuredatabricks.net/?o=xxx#/setting/clusters/xxx-xxx-xxx/sparkUi
View job details at https://westeurope.azuredatabricks.net/?o=xxx#/setting/clusters/xxx-xxx-xxx/sparkUi
View job details at https://westeurope.azuredatabricks.net/?o=xxx#/setting/clusters/xxx-xxx-xxx/sparkUi
View job details at https://westeurope.azuredatabricks.net/?o=xxx#/setting/clusters/xxx-xxx-xxx/sparkUi
View job details at https://westeurope.azuredatabricks.net/?o=xxx#/setting/clusters/xxx-xxx-xxx/sparkUi

Info 14:46:55: Executing silently Code (completed) = import builtins\nbuiltins.print(_VSCODE_getDataFrameInfo(df))
Error 14:46:55: [SyntaxError: Unexpected token V in JSON at position 0
	at JSON.parse (<anonymous>)
	at g.deserializeJupyterResult (/Users/xxx/.vscode/extensions/ms-toolsai.jupyter-2022.3.1000901801/out/extension.js:2:576269)
	at getDataFrameInfo (/Users/xxx/.vscode/extensions/ms-toolsai.jupyter-2022.3.1000901801/out/extension.js:2:572372)
	at async r.ensureInitialized (/Users/xxx/.vscode/extensions/ms-toolsai.jupyter-2022.3.1000901801/out/extension.js:2:963574)
	at async r.getDataFrameInfo (/Users/xxx/.vscode/extensions/ms-toolsai.jupyter-2022.3.1000901801/out/extension.js:2:962377)
	at async x.prepDataFrameInfo (/Users/xxx/.vscode/extensions/ms-toolsai.jupyter-2022.3.1000901801/out/extension.js:2:953270)
	at async x.showData (/Users/xxx/.vscode/extensions/ms-toolsai.jupyter-2022.3.1000901801/out/extension.js:2:951044)
	at async f.create (/Users/xxx/.vscode/extensions/ms-toolsai.jupyter-2022.3.1000901801/out/extension.js:2:960055)]
Info 14:46:55: Executing silently Code (idle) = import builtins\n_rwho_ls = %who_ls\nbuiltins.print(_VSCODE_getVariableTypes(_rwho_ls))
Info 14:46:55: Executing silently Code (completed) = import builtins\n_rwho_ls = %who_ls\nbuiltins.print(_VSCODE_getVariableTypes(_rwho_ls))
Info 14:46:55: Executing silently Code (idle) = import builtins\n_rwho_ls = %who_ls\nbuiltins.print(_VSCODE_getVariableTypes(_rwho_ls))
Info 14:46:55: Executing silently Code (idle) = import builtins\n_rwho_ls = %who_ls\nbuiltins.print(_VSCODE_getVariableTypes(_rwho_ls))
Info 14:46:55: Executing silently Code (completed) = import builtins\n_rwho_ls = %who_ls\nbuiltins.print(_VSCODE_getVariableTypes(_rwho_ls))
Info 14:46:55: Executing silently Code (completed) = import builtins\n_rwho_ls = %who_ls\nbuiltins.print(_VSCODE_getVariableTypes(_rwho_ls))