[pipelines] generate CodeBuild reports within @aws-cdk/cdk-pipelines

Description

To be able to easily use @aws-cdk/assert within @aws-cdk/cdk-pipelines. and use the reports function of CodeBuild.

Use Case

Running tests on CDK using @aws-cdk/assert to increase the ability to Fail Fast and implement TDD.

Proposed Solution

I have considered 2 options so far: ( there may be other options, and there may well be some other/better way to do this )

Option1: add a testAction that adds another stage to codepipeline using another codebuild project :

const pipeline = new CdkPipeline(this, 'Pipeline', {
      pipelineName: 'PipelineWithTests',
      sourceAction: ...
      synthAction: ...
      testAction: simpleTestAction(...)
    });

Where the new testAction includes support for reports in the buildSpec.

Option2: Enable adding reports to the existing SimpleSynthAction(and npm yarn variants) buildSpec: adding the needed changes to the build step commands “yarn build && yarn test”. and adding along the lines of the bellow to the build spec perhaps with a testEnable Flag in the SynthAction.

reports:
  jest_reports:
    files:
      - <report filename>
    file-format: JUNITXML
    base-directory: <test report directory>

Other

I understand that this can be done by creating a codeBuild on its own and adding to the pipeline, but that breaks the elegance and simplicity of using cdk-pipelines which dose a great job at simplifying the codepipeline/buid experience.

I would expect that jest tests would be ran by yarn or npm as described in the @aws-cdk/assert docs. Reference: test-report-jest

---

This is a 🚀 Feature Request

[nbaillie]

@BrianFarnhill if you drop back to SimpleSynthAction from SimpleSynthAction.standardNpmSynth it already has the testCommand available. So would just be to do the second part as you have described above.

synthAction: new SimpleSynthAction({
        sourceArtifact,
        cloudAssemblyArtifact,
        installCommand: "yarn install --frozen-lockfile",
        buildCommand: `yarn build`,
        testCommands: [`yarn test unit`],
        synthCommand: `npx cdk synth`
}),

would become

synthAction: new SimpleSynthAction({
        sourceArtifact,
        cloudAssemblyArtifact,
        installCommand: "yarn install --frozen-lockfile",
        buildCommand: `yarn build`,
        testCommands: [`yarn test unit`],
        synthCommand: `npx cdk synth`,
        testReports: [{
            name: "GroupName" | "GroupArn",
            reportType: "Test" | "CodeCoverage",
            files: string[],
            fileFormat: "JUNITXML" | "OTHER_TYPES" | etc...,
            baseDirectory: string,
        }]
}),

[bilalq]

Based on the sample you shared, this is a working example I came up with to get both test and coverage reports working:

import { BuildSpec, ReportGroup, CfnReportGroup } from '@aws-cdk/aws-codebuild'
import * as cdk from '@aws-cdk/core'
import * as pipelines from '@aws-cdk/pipelines'
import * as iam from '@aws-cdk/aws-iam'

interface SomePipelineProps extends cdk.StackProps {
  // ...
}

export class SomePipeline extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props: SomePipelineProps) {
    super(scope, id, props)

    const repo = 'Your Repo here'
    const branch = 'Your branch here'
    const codeStarConnection = pipelines.CodePipelineSource.connection(repo, branch, {
      connectionArn: 'Your CodeStar connection ARN here',
      triggerOnPush: true
    })

    const jestReportGroup = new ReportGroup(this, 'JestReportGroup', {})
    const unitTestCoverageGroup = new ReportGroup(this, 'CoverageReportGroup', {})
    const cfnUnitTestCoverageGroup = unitTestCoverageGroup.node.defaultChild as CfnReportGroup
    cfnUnitTestCoverageGroup.type = 'CODE_COVERAGE'

    const customSynthStep = new pipelines.CodeBuildStep('ExperimentalSynth', {
      input: codeStarConnection,
      env: {
        CI: 'true'
      },
      installCommands: ['npm ci'],
      commands: ['npm run build', 'npm test', 'npx cdk synth'],
      partialBuildSpec: BuildSpec.fromObject({
        version: '0.2',
        // Make sure your jest config outputs to locations that match what's here
        reports: {
          [jestReportGroup.reportGroupArn]: {
            files: ['jest-results.xml'],
            'file-format': 'JUNITXML',
            'base-directory': 'build/reports/unit-test'
          },
          [unitTestCoverageGroup.reportGroupArn]: {
            files: ['clover.xml'],
            'file-format': 'CLOVERXML',
            'base-directory': 'build/reports/coverage'
          }
        }
      }),
      primaryOutputDirectory: 'cdk.out'
    })

    const pipeline = new pipelines.CodePipeline(this, 'SomePipeline', {
      pipelineName: 'SomePipeline',
      synth: customSynthStep
    })

    // Do whatever else you want on your pipeline
    // ...

    // Granting access here has to happen after calling buildPipeline so the CloudAssembly references resolve
    // All of this is only necessary becasue of limitations discussed in https://github.com/aws/aws-cdk/issues/10464
    pipeline.buildPipeline()
    jestReportGroup.grantWrite(customSynthStep.grantPrincipal)
    unitTestCoverageGroup.grantWrite(customSynthStep.grantPrincipal)
    // This next grant is necessary because the grantWrite command won't add the permission needed for coverage reports
    // https://github.com/aws/aws-cdk/issues/14279
    iam.Grant.addToPrincipal({
      grantee: customSynthStep.grantPrincipal,
      actions: ['codebuild:BatchPutCodeCoverages'],
      resourceArns: [unitTestCoverageGroup.reportGroupArn]
    })
  }
}