[BUG] Unknown test(s) in worker

Context:

• Playwright Version: 1.22.2
• Operating System: Windows 10
• Node.js version: 14.18.1
• Browser: All

Code Snippet

import { test } from '@playwright/test'
import * as faker from 'faker';

  test.only(`test with name ${faker.random.alpha({ count: 5 })}`, async ({ }) => {
    console.log('not working test');
  });

Describe the bug When running parameterized tests with randomly generated names using faker library tests are failing with the error “Unknown test(s) in worker:” It was working fine on version 1.20.2.

[patrickromo1]

We have a similar use case but instead of random values we are sampling/iterating values such as states. For heavy data driven applications this is really useful because iterating over all the values without sampling would equate to thousands of tests and test suites running multiple times a day mitigates the risk of sampling because it slowly fills in the gaps. If playwright does not want to support this going forward it can be orchestrated at a higher level before test execution but it probably would not be as valuable.

import { _ } from 'lodash';

const states = _.sample(['AL', 'GA', 'IN', 'MO', 'NE', 'OR', 'PA', 'VA']);

for (const state of states) {
  test(`should do something in ${state}`, async ({ page }) => {
    console.log(`Do something in ${state}`);
  });
}

[szamacz]

What do you mean by “it was fixed in 1.22”? It was working for us until now (we have over 2k parametrized tests written in such a way) so by fixing something else looks like you introduced this regression. If from now on it is not possible to have parameterized tests with randomly generated names could you please advise how we can change our tests written like one below to have this working:

const uniqueValues = [
faker.random.alpha({ count: 5 }), 
faker.random.alpha({ count: 10 })
]
 for (const value of uniqueValues ) {
 test(`do smth with ${value} , async ({page }) => {
 await functionThatUse(value); 
  });
}

It is crucial in our case to use randomly generated test data and because parametrizing tests in playwright require to have unique test name till now this was the best option that we have found.