Broken diagnostics after editing / reordering cells in polyglot notebooks

Environment data

• Language Server version: 2021.6.3
• OS and version: linux x64
• Python version (and distribution if applicable, e.g. Anaconda):
• python.analysis.indexing: undefined
• python.analysis.typeCheckingMode: off

Expected behaviour

Correct analysis!

Actual behaviour

Many false positives.

I am seeing many error reports, mostly this one but others too. It’s reporting this error in comments at the start of the file as well.

Python language is selected.

I have <N> for a new file, selected python and pasted in my 39 lines of program and errors are not reported. I put an test-error into the new file and that was reported successfully, and removed when I removed my test-error .

So, it’s not the actual syntax etc. I’m relatively new at this. What assumptions are you making that I wouldn’t have predicted? I guess there’s all sorts of things about my setup that I could discover and add but this seems a bit fundamental so I’d appreciate suggestions / guidance.

python --version Python 3.8.10

lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 20.04.2 LTS Release: 20.04 Codename: focal

Logs

Python Language Server Log

XXX 'Please paste the output from your clipboard'

Code Snippet / Additional information

XXX

[jakebailey]

When the non-python cell is present, then we get that single didChange with no contents on the swap:

[Trace - 11:58:18 AM] Sending notification 'textDocument/didChange'.
Params: {
    "textDocument": {
        "uri": "file:///e%3A/work/demo-apps/ds-1/_NotebookConcat_b0021a48b0e74c0cb97107a8c1ddadde.py",
        "version": 16
    },
    "contentChanges": []
}

Then, any changes after that come in duplicated, where the first is empty (should not be sent), and the second one looks right (but won’t work because we didn’t get the change for the swap):

[Trace - 11:58:44 AM] Sending notification 'textDocument/didChange'.
Params: {
    "textDocument": {
        "uri": "file:///e%3A/work/demo-apps/ds-1/_NotebookConcat_b0021a48b0e74c0cb97107a8c1ddadde.py",
        "version": 18
    },
    "contentChanges": []
}


[Trace - 11:58:44 AM] Sending notification 'textDocument/didChange'.
Params: {
    "textDocument": {
        "uri": "file:///e%3A/work/demo-apps/ds-1/_NotebookConcat_b0021a48b0e74c0cb97107a8c1ddadde.py",
        "version": 18
    },
    "contentChanges": [
        {
            "range": {
                "start": {
                    "line": 4,
                    "character": 6
                },
                "end": {
                    "line": 4,
                    "character": 6
                }
            },
            "rangeLength": 0,
            "text": "a"
        }
    ]
}

[erictraut]

Thanks for the trace logs. Those were very helpful. I spent some time walking through each message to reconstruct the document one edit at a time.

The “textDocument/publishDiagnostics” input messages look correct to me. They’re in the right order, and when I apply the deltas, I reconstruct the text that you show in your screen shot above.

The “textDocument/publishDiagnostics” output messages also look correct to me. They include the diagnostics I would expect to see while you’re coding — various syntax errors due to incomplete code. The text ranges for these diagnostics also correspond to the line and column numbers that I would expect from the reconstructed contents.

Subsequent “textDocument/publishDiagnostics” output messages should replace the previously-reported list of diagnostics within the document. That doesn’t seem to be happening. For example, the “textDocument/publishDiagnostics” message sent at timestamp 2:52:12 contains the message Expected ":". The subsequent “textDocument/publishDiagnostics” message at timestamp 2:52:13 no longer contains a diagnostic with this message because the intervening edits have eliminated that syntax error condition. The client is supposed to replace older diagnostic lists with an updated list when it receives a newer “textDocument/publishDiagnostics” message.

Based on the evidence, it looks like the problem is in the code that provides support for jupyter notebooks — the code that combines the cells in the notebook and writes the combined Python code to a file for pylance to analyze. I’m guessing that this code is not properly handling the “textDocument/publishDiagnostics” messages that it receives. It should process them in the order they’re received, and newer diagnostic lists should replace older lists.

@jakebaily, one thing that I noticed is that pylance is not including a version in the “textDocument/publishDiagnostics” message. The LSP protocol indicates that such a version is not required, but I wonder if the absence of this version is triggering a bug in the jupyter notebook client code.