IPyWidget tests randomly fail

Environment data

• VS Code version: XXX
• Extension version (available under the Extensions sidebar): XXX
• Python Extension version (available under the Extensions sidebar): XXX
• OS (Windows | Mac | Linux distro) and version: XXX
• Python and/or Anaconda version: XXX
• Type of virtual environment used (N/A | venv | virtualenv | conda | …): XXX
• Jupyter server running: Local | Remote | N/A

Expected behaviour

XXX

Actual behaviour

XXX

Steps to reproduce:

[NOTE: Self-contained, minimal reproducing code samples are extremely helpful and will expedite addressing your issue]

1. XXX

Logs

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

XXX

[joyceerhl]

Was just looking at this as part of investigating https://github.com/microsoft/vscode-jupyter/issues/3955. If I manually run the ipyvolume notebook, the first cell actually does take a fair bit of time to run, so I’m not surprised that we sometimes timeout waiting for the cell to execute. I noticed that at least for the ipyvolume test, we have retry logic for the part where we verify the outputs, but we don’t retry execution if that times out. I tried moving the cell execution into the retry for the ipyvolume test since that seems to be the one that times out the most often, if that helps and sounds like a good idea we can maybe consider doing this for the other tests.

[joyceerhl]

Update: there are three failure types I have seen based on past GHA run logs:

1. Timeout after 3min: this is associated with a ‘Failed to load resource net::ERR_CONTENT_LENGTH_MISMATCH’ error. Not sure what the root cause of this is. Don had submitted a PR that we thought would fix this but it’s still cropping up. This is the most common failure case.

2. rendered_execution timeout. I cannot repro this yet. Just need to add more logging.

3. AttributeErrors in the output. That one as far as I can tell is the CDN not responding with the script source. This is the least common failure and also the one we can do the least about.