New test level for metadata deploy - RunTestSuite

What are you trying to do It would be nice to be able to specify a test suite name as part of deployment command that would be used to specify which tests should be executed as part of the deployment, as an alternative to the RunSpecifiedTests option.

For example, if I have apackage with manifest as follows:

package.xml

<?xml version="1.0" encoding="UTF-8"?>
<Package xmlns="http://soap.sforce.com/2006/04/metadata">
  <types>
    <members>Foo</members>
    <members>FooTest</members>
    <members>Bar</members>
    <members>BarTest</members>
    <name>ApexClass</name>
  </types>

  <types>
    <members>MyTestSuite</members>
    <name>ApexTestSuite</name>
  </types>
</Package>

And the test suite specifies the tests I want executed during deployment:

MyTestsuite.testSuite

<?xml version="1.0" encoding="UTF-8"?>
<ApexTestSuite xmlns="http://soap.sforce.com/2006/04/metadata">
    <testClassName>FooTest</testClassName>
    <testClassName>BarTest</testClassName>
</ApexTestSuite>

Describe the solution you’d like I would like to use a test level named RunTestSuite for example, that would require a test suite name be specified, possibily by reusing --runtests or with another flag. With this test level I don’t have to explicitly list all my test classes I want to have run during deployment. Instead the tool would extract the list of tests from the test suite and run those.

Example:

$ sfdx force:mdapi:deploy \
  --testlevel RunTestSuite \
  --runtests MyTestSuite \
  --deploydir mypackage \
  --wait -1 \
  -u alan

# Maybe also supports multiple test suites:
$ sfdx force:mdapi:deploy \
  --testlevel RunTestSuite \
  --runtests MyTestSuite1,MyTestSuite2,MyTestSuiteN \
  --deploydir mypackage \
  --wait -1 \
  -u alan

[alan-morey]

@dcarroll You can not use --runtests currently with a test suite name.

Here is a bash script I wrote which kind of implements the feature, it requires xmlstartlet

#!/usr/bin/env bash
XPATH_PACKAGE_TEST_SUITES='/_:Package/_:types/_:name[.="ApexTestSuite"]/../_:members'
XPATH_TEST_SUITE_CLASSES='/_:ApexTestSuite/_:testClassName'
MANIFEST=package.xml

TEST_SUITES=$(xmlstarlet select --template --value-of $XPATH_PACKAGE_TEST_SUITES $MANIFEST)
TEST_SUITES=$(echo $TEST_SUITES | tr ' ' '\n' | sort | uniq)
if [ -z "$TEST_SUITES" ]; then
    echo "*WARNING* Skipping test run, $MANIFEST does not specify any test suite(s)"
    exit 1
fi

echo -e "Apex Test Suites in $MANIFEST:\n - $(echo $TEST_SUITES | sed 's/ /\n - /g')"
echo

TEST_CLASSES=''
for TEST_SUITE in $TEST_SUITES; do
    TEST_SUITE_FILE="src/testSuites/$TEST_SUITE.testSuite"
    TEST_CLASSES+="\n"$(xmlstarlet select --template --value-of $XPATH_TEST_SUITE_CLASSES $TEST_SUITE_FILE)
done

TEST_CLASSES=$(echo -e $TEST_CLASSES | tr ' ' '\n' | sort | uniq)
if [ -z "$TEST_CLASSES" ]; then
    echo "*WARNING* Skipping test run, no test classes in parsed test suite(s)"
    exit 1
fi

echo -e "Apex Test Classes:\n - $(echo $TEST_CLASSES | sed 's/ /\n - /g')"
echo

RUN_TESTS=$(echo $TEST_CLASSES | tr ' ' ,)

sfdx force:mdapi:deploy --verbose --checkonly --deploydir build/deploy --wait -1 --testlevel RunSpecifiedTests --runtests $RUN_TESTS -$@

[maelmonnier]

In case it can help, there is an helper in the SFDX plugin of Accenture to extract all test classes from a test suite : https://github.com/Accenture/sfpowerkit#sfpowerkitsourceapextestsuiteconvert (sfpowerkit:source:apextestsuite:convert)