@aws-cdk/integ-runner: Running tests makes runner think a deprecated feature flag is active for test stacks

Describe the bug

When invoking integ-runner to run a test file with an IntegTest, it always errors out with Error: Unsupported feature flag '@aws-cdk/core:enableStackNameDuplicates'. This flag existed on CDKv1 but has been removed in CDKv2. CDK will now behave as the same as when the flag is enabled.

This happened with both an existing CDK project and a new one created with cdk init --language typescript using CDKv2. This flag was not defined in project initialization nor at any point after.

Expected Behavior

• Invoke integ-runner to run a test file
• Test stacks get deployed and tests run

Current Behavior

• integ-runner errors out with Error: Unsupported feature flag '@aws-cdk/core:enableStackNameDuplicates'. This flag existed on CDKv1 but has been removed in CDKv2. CDK will now behave as the same as when the flag is enabled.
• Tests are not run

Reproduction Steps

• Install cdk: npm i -g cdk
• Create a new cdk project: cdk init --language typescript
• Install integ-tests-alpha: npm i @aws-cdk/integ-tests-alpha@2.33.0-alpha.0
• Install integ-runner: npm i -g @aws-cdk/integ-runner@1.165.0
• Create a file test/integ.test.js with the following contents:

import { IntegTest } from '@aws-cdk/integ-tests-alpha';

class StackUnderTest extends Stack {
    constructor(scope, id, props) {
        super(scope, id, props);
    }
}

// Beginning of the test suite
const app = new App();

new IntegTest(app, 'MyTestCase', {
  testCases: [
    new StackUnderTest(app, 'Stack1', { }),
  ],
});

• Run integ-runner --update-on-failed (I use aws-vault to run in a sandbox account, so my full command is aws-vault exec <profile> -- integ-runner --update-on-failed)

Possible Solution

No response

Additional Information/Context

A coworker followed a similar flow and ran into the same issue. We aren’t sure if we’re overlooking something or doing something wrong but we haven’t been able to make sense of why we’re running into this issue.

CDK CLI Version

2.31.2 (build 091f0d8)

Framework Version

No response

Node.js Version

v16.9.1

OS

macOS Monterey 12.4

Language

Typescript

Language Version

3.9.10

Other information

No response

[WinterYukky]

I ran into the same problem and did debugged this issue. Perhaps this is due to the fact that integ-runner is v1. I found following code in the integ-runner.

protected getContext(additionalContext?: Record<string, any>): Record<string, any> {
  const futureFlags: { [key: string]: any } = {};
  Object.entries(FUTURE_FLAGS)
    .filter(([k, _]) => !FUTURE_FLAGS_EXPIRED.includes(k))
    .forEach(([k, v]) => futureFlags[k] = v);

  const enableLookups = (this.actualTestSuite ?? this.expectedTestSuite)?.enableLookups;
  return {
    // if lookups are enabled then we need to synth
    // with the "dummy" context
    ...enableLookups ? DEFAULT_SYNTH_OPTIONS.context : {},
    // This is needed so that there are no differences between
    // running on v1 vs v2
    [NEW_STYLE_STACK_SYNTHESIS_CONTEXT]: '',
    ...futureFlags,
    ...this.legacyContext,
    ...additionalContext,
  };
}

This code appears to remove expired future flags in v2. running it in v2 produces the expected values.

// FUTURE_FLAGS
{
  '@aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId': true,
  '@aws-cdk/core:enableStackNameDuplicates': true,
  'aws-cdk:enableDiffNoFail': true,
  '@aws-cdk/core:stackRelativeExports': true,
  '@aws-cdk/aws-ecr-assets:dockerIgnoreSupport': true,
  '@aws-cdk/aws-secretsmanager:parseOwnedSecretName': true,
  '@aws-cdk/aws-kms:defaultKeyPolicies': true,
  '@aws-cdk/aws-s3:grantWriteWithoutAcl': true,
  '@aws-cdk/aws-ecs-patterns:removeDefaultDesiredCount': true,
  '@aws-cdk/aws-rds:lowercaseDbIdentifier': true,
  '@aws-cdk/aws-efs:defaultEncryptionAtRest': true,
  '@aws-cdk/aws-lambda:recognizeVersionProps': true,
  '@aws-cdk/aws-lambda:recognizeLayerVersion': true,
  '@aws-cdk/aws-cloudfront:defaultSecurityPolicyTLSv1.2_2021': true,
  '@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver': true,
  '@aws-cdk/aws-ec2:uniqueImdsv2TemplateName': true,
  '@aws-cdk/core:checkSecretUsage': true,
  '@aws-cdk/aws-iam:minimizePolicies': true,
  '@aws-cdk/aws-ecs:arnFormatIncludesClusterName': true,
  '@aws-cdk/core:validateSnapshotRemovalPolicy': true,
  '@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName': true,
  '@aws-cdk/aws-s3:createDefaultLoggingPolicy': true,
  '@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption': true
}
// FUTURE_FLAGS_EXPIRED
[
  '@aws-cdk/aws-ecr-assets:dockerIgnoreSupport',
  '@aws-cdk/aws-ecs-patterns:removeDefaultDesiredCount',
  '@aws-cdk/aws-efs:defaultEncryptionAtRest',
  'aws-cdk:enableDiffNoFail',
  '@aws-cdk/core:enableStackNameDuplicates',
  '@aws-cdk/aws-kms:defaultKeyPolicies',
  '@aws-cdk/aws-s3:grantWriteWithoutAcl',
  '@aws-cdk/aws-secretsmanager:parseOwnedSecretName'
]
// futureFlags ⭕
{
  '@aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId': true,
  '@aws-cdk/core:stackRelativeExports': true,
  '@aws-cdk/aws-rds:lowercaseDbIdentifier': true,
  '@aws-cdk/aws-lambda:recognizeVersionProps': true,
  '@aws-cdk/aws-lambda:recognizeLayerVersion': true,
  '@aws-cdk/aws-cloudfront:defaultSecurityPolicyTLSv1.2_2021': true,
  '@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver': true,
  '@aws-cdk/aws-ec2:uniqueImdsv2TemplateName': true,
  '@aws-cdk/core:checkSecretUsage': true,
  '@aws-cdk/aws-iam:minimizePolicies': true,
  '@aws-cdk/aws-ecs:arnFormatIncludesClusterName': true,
  '@aws-cdk/core:validateSnapshotRemovalPolicy': true,
  '@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName': true,
  '@aws-cdk/aws-s3:createDefaultLoggingPolicy': true,
  '@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption': true
}

However, in practice, the following values are obtained because the constants defined in v1 are used.

// FUTURE_FLAGS
{
  '@aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId': true,
  '@aws-cdk/core:enableStackNameDuplicates': true,
  'aws-cdk:enableDiffNoFail': true,
  '@aws-cdk/core:stackRelativeExports': true,
  '@aws-cdk/aws-ecr-assets:dockerIgnoreSupport': true,
  '@aws-cdk/aws-secretsmanager:parseOwnedSecretName': true,
  '@aws-cdk/aws-kms:defaultKeyPolicies': true,
  '@aws-cdk/aws-s3:grantWriteWithoutAcl': true,
  '@aws-cdk/aws-ecs-patterns:removeDefaultDesiredCount': true,
  '@aws-cdk/aws-rds:lowercaseDbIdentifier': true,
  '@aws-cdk/aws-efs:defaultEncryptionAtRest': true,
  '@aws-cdk/aws-lambda:recognizeVersionProps': true,
  '@aws-cdk/aws-lambda:recognizeLayerVersion': true,
  '@aws-cdk/aws-cloudfront:defaultSecurityPolicyTLSv1.2_2021': true,
  '@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver': true,
  '@aws-cdk/aws-ec2:uniqueImdsv2TemplateName': true,
  '@aws-cdk/core:checkSecretUsage': true,
  '@aws-cdk/aws-iam:minimizePolicies': true
}
// FUTURE_FLAGS_EXPIRED
[]
// futureFlags 🤔
{
  '@aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId': true,
  '@aws-cdk/core:enableStackNameDuplicates': true,
  'aws-cdk:enableDiffNoFail': true,
  '@aws-cdk/core:stackRelativeExports': true,
  '@aws-cdk/aws-ecr-assets:dockerIgnoreSupport': true,
  '@aws-cdk/aws-secretsmanager:parseOwnedSecretName': true,
  '@aws-cdk/aws-kms:defaultKeyPolicies': true,
  '@aws-cdk/aws-s3:grantWriteWithoutAcl': true,
  '@aws-cdk/aws-ecs-patterns:removeDefaultDesiredCount': true,
  '@aws-cdk/aws-rds:lowercaseDbIdentifier': true,
  '@aws-cdk/aws-efs:defaultEncryptionAtRest': true,
  '@aws-cdk/aws-lambda:recognizeVersionProps': true,
  '@aws-cdk/aws-lambda:recognizeLayerVersion': true,
  '@aws-cdk/aws-cloudfront:defaultSecurityPolicyTLSv1.2_2021': true,
  '@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver': true,
  '@aws-cdk/aws-ec2:uniqueImdsv2TemplateName': true,
  '@aws-cdk/core:checkSecretUsage': true,
  '@aws-cdk/aws-iam:minimizePolicies': true
}

Integ-runner uses this context to synthesize the stack, which causes an Error: Unsupported feature flag '@aws-cdk/core:enableStackNameDuplicates'. This flag existed on CDKv1 but has been removed in CDKv2. CDK will now behave as the same as when the flag is enabled. error in v2.

@madeline-k Is it possible to release integ-runner-alpha in v2?

[corymhall]

@orlandronen1 @WinterYukky @dontirun can you try this now? We are now publishing integ-runner for v2. https://www.npmjs.com/package/@aws-cdk/integ-runner/v/2.44.0