[assert] SynthUtils.toCloudFormation(stack)).toMatchSnapshot will fail with CDK Pipelines because the output changes every time

❓ General Issue

Consider the following code where PipelineStack is a stack containing CDK Pipelines and some infra code.

  const app = new cdk.App({
    context: {
      '@aws-cdk/core:newStyleStackSynthesis': 'true',
    },
  });
  const stack = new PipelineStack(app, 'PipelineStack', [], {
    env: {
      region: 'eu-central-1',
    },
  });
  // THEN
  expect(SynthUtils.toCloudFormation(stack)).toMatchSnapshot();

Running the test will fail because the output changes every time:

  ● PipelineStack

    expect(received).toMatchSnapshot()

    Snapshot name: `PipelineStack 1`

    - Snapshot  - 1
    + Received  + 1

    @@ -738,11 +738,11 @@
              \"git init && npx cdk synth\"
            ]
          }
        },
        \"artifacts\": {
    -     \"base-directory\": \"var/folders/5f/y0wjdf0122sgz4vvyt4l0h080000gn/T/cdk.outgD8H3Y\",
    +     \"base-directory\": \"var/folders/5f/y0wjdf0122sgz4vvyt4l0h080000gn/T/cdk.outfqommz\",
          \"files\": \"**/*\"
        }
      }",
                "Type": "CODEPIPELINE",
              },

The Question

Is this a bug in how CDK builds the project or would it be possible to somehow ignore changes in certain properties that are unrelated to the test? What is the recommended approach here?

Environment

• CDK CLI Version: 1.62.0 (build 8c2d7fc)
• Module Version: 1.62.0
• Node.js Version: v14.8.0
• OS: macOs
• Language (Version): TypeScript 4

Other information

[straygar]

Not a solution, but a workaround is to use a custom snapshot serializer to omit these paths (note - the replacement might a bit agressive, but will at least avoid false-positives):

/** 
 * A custom snapshot serializer, used to remove any randomly generated hash values, that would lead to false positives in snapshot tests.
*/
expect.addSnapshotSerializer({
    test: template => typeof template === 'string',
    print: template => {
        const sanitizedTemplate = (template as string)
            .replace(/\"base-directory\": \".+\"/, '"base-directory": "[HASH REMOVED]"')
            .replace(/\"value\":\"[a-z0-9]+\"/, '"value": "[HASH REMOVED]"');
        return `"${sanitizedTemplate}"`;
    },
});

[markusl]

Just adding a comment for future readers:

In addition to setting the outdir you likely want to turn off the stack traces from the test output:

test('ExampleEcsClusterStack should synthesize to expected snapshot', () => {
  const app = new cdk.App( { 
    outdir: 'cdk.out',
    stackTraces: false,
  });
  // WHEN
  const stack = new ExampleEcsClusterStack(app, 'ecs-stack', { ...props });

  // THEN
  const snapshot = SynthUtils.synthesize(stack);

  expect(snapshot).toMatchSnapshot();
});