Support for continuous (autorun) tests

A popular feature in test runners is re-running tests automatically as source code changes. I this should consider recognizing API, since the UI will have several specific behaviors for this:

• A profile-aware “toggle” button to start and stop running common across extensions and implementations
• Different behavior around when or if the peak automatically opens and state is followed in the test explorer when under an auto-run

While the former can be done through existing extension APIs such as menu contributions, it may be worth having official UI for this to provide a consistent experience, especially when multiple types of tests (and therefore extensions) are present.

I think this can be implemented in a fairly straightforward way with the addition of a supportsContinuousRun on the profile, and a boolean indicator on TestRuns indicating whether they should be continuous. I prefer the term “continuous” over “auto run” since saying “this is a continuous test run” brings in the temporal nature that “this is an autorun” does not.

API Proposal

export interface TestRunProfile {
	/**
	 * Whether this profile supports continuous running of requests. If so,
	 * then {@link TestRunRequest.autoRun} may be set to `true`.
	 */
	supportsContinuousRun: boolean;

	/**
	 * Handler called to start a test run. When invoked, the function should call
	 * {@link TestController.createTestRun} at least once, and all test runs
	 * associated with the request should be created before the function returns
	 * or the returned promise is resolved.
	 *
	 * If {@link supportsContinuousRun} is set, then `continuous` may be `true`
	 * in the request. In this case, the profile should observe changes to
	 * source code and create new test runs, until the cancellation is
	 * requested on the `token`.
	 *
	 * @param request Request information for the test run.
	 * @param cancellationToken Token that signals the used asked to abort the
	 * test run. If cancellation is requested on this token, all {@link TestRun}
	 * instances associated with the request will be
	 * automatically cancelled as well.
	 */
		runHandler: (request: TestRunRequest, token: CancellationToken) => Thenable<void> | void;
}

export class TestRunRequest {
	/**
	 * Whether the runner should run continuously as source code changes. Only
	 * effectual for profiles that set {@link TestRunProfilesupportsContinuousRun}.
	 */
	readonly continuous?: boolean;

	/**
	 * @param tests Array of specific tests to run, or undefined to run all tests
	 * @param exclude An array of tests to exclude from the run.
	 * @param profile The run profile used for this request.
	 * @param continuous Whether to run tests continuously as source changes.
	 */
		constructor(include?: readonly TestItem[], exclude?: readonly TestItem[], profile?: TestRunProfile, continuous?: boolean);
}

[karamfil]

Go Go Guys! You can do it and deliver it in November 😃

[connor4312]

I guess that these settings currently do not have any effect?

That’s right