CodeceptJS 3.1.0: Playwright: Option "restart: false" stopped working, browser is restarted between scenarios

It’s a regression in CodeceptJS 3.1.0. It works well in 3.0.7. Playwright version doesn’t matter, I tested it with 1.11, 1.13 and 1.14, with the same result.

What are you trying to achieve?

The option restart: false should work - browser should keep open between scenarios

What do you get instead?

The option restart: false doesn’t work - browser is restarted when a scenario finishes and is open with new window for a next scenario.

CodeceptJS 3.1.1 (where the test fails)

$ codeceptjs run --verbose
CodeceptJS v3.1.1
Using test root "/home/mirao/workspace/codeceptjs/tests/my"
Helpers: Playwright
Plugins: screenshotOnFail, retryFailedStep, tryTo

My --
    [1]  Starting recording promises
 › [Session] Starting singleton browser session
  Open CodeceptJS site
    I am on page "/"
    › [Browser:Info] 
    › [Browser:Info] 
    › [Browser:Log] runPoptinNow
    › [Browser:Log] initiatePullPoptinsRequest()
  ✔ OK in 3371ms

  Open Quickstart
    I click {xpath: .//a[@href = '/quickstart']}
    [1] Retrying... Attempt #2
    [1] Retrying... Attempt #3
    [1] Retrying... Attempt #4
    [1] Retrying... Attempt #5
    [1] Retrying... Attempt #6
    [1] Error | Error: Clickable element "{"type":"xpath","output":null,"strict":true,"locator":{"xpath":".//a[@href = '/quickstart']"},"value":".//a[@href = '/quickstart']"}" was not found by text|CSS|XPath
    [1] Error | Error: Clickable element "{"type":"xpath","output":null,"strict":true,"locator":{"xpath":".//a[@href = '/quickstart']"},"value":".//a[@href = '/quickstart']"}" was not found by text|CSS|XPath
    [1] Starting <teardown> session
    [1] <teardown> Stopping recording promises
 › <screenshotOnFail> Test failed, try to save a screenshot
 › Screenshot is saving to /home/mirao/workspace/codeceptjs/tests/my/output/Open_Quickstart.failed.png
  ✖ FAILED in 4903ms

    [2]  Starting recording promises

-- FAILURES:

  1) My
       Open Quickstart:
     Clickable element "{"type":"xpath","output":null,"strict":true,"locator":{"xpath":".//a[@href = '/quickstart']"},"value":".//a[@href = '/quickstart']"}" was not found by text|CSS|XPath
      at new ElementNotFound (/home/mirao/workspace/codeceptjs/node_modules/codeceptjs/lib/helper/errors/ElementNotFound.js:14:11)
      at assertElementExists (/home/mirao/workspace/codeceptjs/node_modules/codeceptjs/lib/helper/Playwright.js:2620:11)
      at Playwright.proceedClick (/home/mirao/workspace/codeceptjs/node_modules/codeceptjs/lib/helper/Playwright.js:2367:5)
  
  Scenario Steps:
  - I.click({xpath: .//a[@href = '/quickstart']}) at Test.<anonymous> (./my_test.js:8:7)
  
  Artifacts:
  - screenshot: /home/mirao/workspace/codeceptjs/tests/my/output/Open_Quickstart.failed.png


  FAIL  | 1 passed, 1 failed   // 9s

CodeceptJS 3.0.7 (where the test passes):

$ codeceptjs run --verbose
CodeceptJS v3.0.7
Using test root "/home/mirao/workspace/codeceptjs/tests/my"
Helpers: Playwright
Plugins: screenshotOnFail, retryFailedStep, tryTo

My --
    [1]  Starting recording promises
 › [Session] Starting singleton browser session
  Open CodeceptJS site
    I am on page "/"
    › [Browser:Log] runPoptinNow
    › [Browser:Log] initiatePullPoptinsRequest()
  ✔ OK in 2149ms

 › [Session] cleaning cookies and localStorage
  Open Quickstart
    I click {xpath: .//a[@href = '/quickstart']}
  ✔ OK in 213ms

 › [Session] cleaning cookies and localStorage

  OK  | 2 passed   // 3s

Provide test source code if related

Feature("My");

Scenario("Open CodeceptJS site", ({ I }) => {
    I.amOnPage("/");
});

Scenario("Open Quickstart", ({ I }) => {
    I.click(locate("a").withAttr({ href: "/quickstart" }));
});

Details

• CodeceptJS version: 3.1.1
• NodeJS Version: v12.22.5
• Operating System: Ubuntu 21.04
• Playwright 1.14
• Configuration file:

exports.config = {
    tests: "./*_test.js",
    output: "./output",
    helpers: {
        Playwright: {
            url: "https://codecept.io",
            show: true,
            browser: "chromium",
            restart: false
        }
    },
    include: {
        I: "./steps_file.js"
    },
    bootstrap: null,
    mocha: {},
    name: "my",
    plugins: {
        pauseOnFail: {},
        retryFailedStep: {
            enabled: true
        },
        tryTo: {
            enabled: true
        },
        screenshotOnFail: {
            enabled: true
        }
    }
};

[DavertMik]

I had a discussion with Playwright’s team and we agreed that both flows can be used. So currently I recommend you to downgrade to 3.0.x version I will work on 3.2.0 where both oprtions will be enabled.

Thanks for the feedback

[shadinka]

@mirao thanks; configured the key restart: “keep”, I checked it locally and it fixed the issue; now running on TC within docker image, 10 threads, and it is also keeps the browser session between scenarios. Unfortunately, I have other issue i’m trying to debug; one of the chunks in multiple run get stuck forever.