[Feature] Can we add feature for test.describe.serial if any dependent test case failed it should not stop entire execution

Feature needed: Playwright should not stop entire execution if failing any one of the dependent test cases running in serial mode using test.describe.serial feature.

We have large End to End scenarios where we can not write independent test cases, so we have to write dependent test cases which depends on previous test block execution result. But in Playwright executing test blocks in sequentially using test.describe.serial and if failing any one of the test case it is actually stopping entire execution which is not feasible. If you can add this feature like it is available in independent test cases execution if failing any test block it should be continue till the number of failures encountered by declared –max-failures or –bail flag count defined in config file.

Yes definitely it will be helpful for the automation guys who has dependent scenarios where waiting for previous block to be execute first as in end to end scenarios there is a number of step are there need to be run in sequentially so we divided those steps in chunk of test blocks.

We are planning to migrate our framework from Protractor to Playwright but since this feature is not available in Playwright and we have large number of dependent scenarios it makes us in decidable mode to go for this tool or not.

Here in the below example like I am running 3 tests blocks one after another in a sequentially manner but failing first test block it stops entire execution which remains untested the other scenarios and functionalities in the script and as well as in another scripts which are yet to be run. Could you please add this feature to make sure we can convert our framework from Protractor to Playwright.

test.describe.serial("End-to-End test case scenarios running sequentially", ()=>{

    test("Test 1", async()=>{
       await expect(true).toBe(false);
    });

    //test-2 depends on test 1
    test("Test 2", async()=>{
       await expect(true).toBe(false);
    });

    //test -3 depends on test 2
    test("Test 3", async()=>{
        await expect(true).toBe(false);
    });
    
});

[Darkensses]

I support @BeBestOf … what about SPAs? So, basically you’re telling us that if the app has a bug, you can’t find any bug in other screens until you fix it first.

So, let’s say we need to automate pixect flow. We want to test the camera button displayed in the screen where the webcam is enabled. To get to that screen, you need to navigate through 2 previous screens/slides. The second slide has a text that needs to be validated, but the test fails… so how can we get to the next screen?

const { test, expect } = require('@playwright/test');

test.describe.serial("Serial test", () => {
    let page;

    test.beforeAll(async ({ browser }) => {
        const context = await browser.newContext();
        page = await context.newPage();
        await page.goto('https://www.pixect.com/es/');
    });

    test('click start button', async() => {
        await page.locator('#btnStart').click();
    });

    test('verify text below camera button', async() => {
        let element = await page.locator('//*[@id="firstMode"]/div[1]');      
        await expect(element).toHaveText(/Usa la camara/); // expected: Usa la cámara
    });

    test('click camera button', async() => {
        await page.locator('#btnCamFirst').click();
    });

    test('click capture button', async() => {
        await page.locator('#btnCapture').click();
    });

})

[nikhilgupta1789]

@rwoll @mxschmitt Fare Enough!! but there are millions of web applications where it is not always the case first ‘Add Item To Cart’ and then ‘Pay For Item’, applications behaves differently having different domains. For an example If I talk about my scenarios I am clicking on ‘Get Started’ button on Playwright site and then clicking on Writing Tests, Running Tests, Test Generator, Trace Viewer etc links and verifying title of each landed page, if anyone of the page title is misspell doesn’t mean other subsequent test cases has not to be tested or skipped. Why not this feature should be added while in case of test.describe.serial where any earlier failure should not skip subsequent test blocks and give control on failures with flags as --max-failure or --bail counts encountered and then stop execution when expected failure counts meet. I am eagerly waiting for this feature to be introduce so we can freely write our some of the end to end test scenarios in dependent manner which we had written in Protractor where page state maintains and shared between test blocks.

Let’s talk about your scenarios If we are writing independent scenarios I am 100% sure there is a code repetition for every test block to reach to the state where last test state was exists. In every independent test blocks we have to repeat the same line of codes I know this can be done with hooks but in end to end scenarios it is very tough to design scenarios like in this manner.

Also many people raised this queries for dependent test cases atleast it has to be given a control to user in test.describe.serial. I request to you @rwoll please think on this. This is a very valid scenarios used in many applications where earlier of test should continue with the subsequent test blocks because applications has different scenarios. Than you very much!! 😃