[BUG] Playwright test runner always show test result pass when working with websocket (ws or sockiet io client)

Context:

• Playwright Version:“^1.27.1”
• Operating System: Linux (Ubuntu 22.04)
• Node.js version: v18.2.1
• Browser: request (API testing)
• Extra: “socket.io”: “^4.5.4”,“socket.io-client”: “^4.5.4”, “ws”: “^8.11.0”

Code Snippet

Help us help you! Put down a short code snippet that illustrates your bug and that we can run and debug locally. For example:

test(`check websocket available`, async({}) => {

    const ws = new WebSocket(`${endpoint.WEBSOCKET_HOST}/notification/inapp/register`,{
        headers: {
          Cookie: `token=1${accessToken}`
        }
    });
    ws.on('open', function (){
        ws.send("say Donnnnnnnnnnn") 
    });

})

Describe the bug

When running the test, even inputting an invalid value into the cookie field, Playwright still shows the test result as pass. I checked with socket io client also, but still the result is pass. I guess there’s something wrong with Playwright test runner when working with WebSocket client.

[dgozman]

@cuongld2 That’s probably because Jest does not exit fast enough for the websocket error to be thrown. Most likely Jest will wait for the open socket connection to close before exiting, but Playwright won’t do that. I’d recommend to await things, for example:

  test('websocket connection', async () => {
    await new Promise(done => {
      const ws = new WebSocket('wss://domain/inapp/register', {
        headers: {
          Cookie: 'token=11',
        }
      });

      ws.on('open', function open() {
          ws.send("say something")
          ws.terminate()
          done()
      });
    });
  });

[cuongld2]

Thank you guys, Works like a charm.