question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[aws-secretsmanager] Unnecessary updates of SecretRotationApplication if no resources have been modified

See original GitHub issue

The SecretRotationApplication construct is updated even if no resources of the enclosing stack have been modified.

  • Running cdk diff DbStack produces There were no differences.
  • Running cdk deploy DbStack produces:
    0/3 | 10:48:21 | UPDATE_IN_PROGRESS   | AWS::CloudFormation::Stack                  | Database/RotationSingleUser (DatabaseRotationSingleUser65F55654)
    1/3 | 10:48:32 | UPDATE_COMPLETE      | AWS::CloudFormation::Stack                  | Database/RotationSingleUser (DatabaseRotationSingleUser65F55654) 
    1/3 | 10:48:35 | UPDATE_COMPLETE_CLEA | AWS::CloudFormation::Stack                  | DbStack
    
  • Running cdk deploy DbStack --verbose, I see the following lines:
    DbStack: parameters have changed
    DbStack: deploying...
    Attempting to create ChangeSet CDK-6225bb6f-6be9-4297-a89c-c6c49a19d2f8 to update stack DbStack
    DbStack: creating CloudFormation changeset...
    
    But I cannot see which parameters actually changed. (How can I see those?)

Reproduction Steps

Database stack:

const parameterGroup = new rds.ClusterParameterGroup(this, 'ClusterParameterGroup', {
  family: 'aurora5.6',
  parameters: {
    general_log: '1',
    slow_query_log: '1',
  },
});

const db = new AuroraServerless(this, 'Database', {
  engine: rds.DatabaseClusterEngine.AURORA,
  engineVersion: '5.6.10a',
  vpc,
  vpcSubnets: { subnetType: ec2.SubnetType.ISOLATED },
  parameterGroup,
  defaultDbName: 'demo',
  scalingConfig: {
    // ...
  },
});

// Rotate secret every 30 days
db.addRotationSingleUser(cdk.Duration.days(30));

Custom AuroraServerless construct:

export class AuroraServerless extends cdk.Resource implements ec2.IConnectable, secretsmanager.ISecretAttachmentTarget {
  public readonly secret: rds.DatabaseSecret;
  private readonly secretRotationApplication: secretsmanager.SecretRotationApplication;

  // endpoints, connections, subnets, security group

  constructor(scope: cdk.Construct, id: string, props: AuroraServerlessProps) {
    super(scope, id);

    const { engine, engineVersion } = props;

    // Setup subnets and security group

    // DB secret
    const secret = new rds.DatabaseSecret(this, 'DbSecret', {
      username: 'root',
    });
    this.secret = secret;
    this.secretRotationApplication = engine.singleUserRotationApplication;

    // DB cluster
    const cluster = new rds.CfnDBCluster(this, 'DbCluster', {
      engine: engine.name,
      engineVersion,
      engineMode: 'serverless',
      masterUsername: secret.secretValueFromJson('username').toString(),
      masterUserPassword: secret.secretValueFromJson('password').toString(),
      // parameters, subnets, security group, roles, scaling config, ...
    });
    this.clusterIdentifier = cluster.ref;

    secret.attach(this);

    // Setup cluster endpoint, "connections", and clusterArn
  }


  // https://github.com/aws/aws-cdk/blob/26a69b1b090b49505f69ef2879b68d2382ea27ec/packages/%40aws-cdk/aws-rds/lib/cluster.ts#L542
  public addRotationSingleUser(automaticallyAfter?: cdk.Duration): secretsmanager.SecretRotation {
    if (!this.secret) {
      throw new Error('Cannot add single user rotation for a cluster without secret.');
    }

    const id = 'RotationSingleUser';
    const existing = this.node.tryFindChild(id);
    if (existing) {
      throw new Error('A single user rotation was already added to this cluster.');
    }

    return new secretsmanager.SecretRotation(this, id, {
      secret: this.secret,
      automaticallyAfter,
      application: this.secretRotationApplication,
      vpc: this.vpc,
      vpcSubnets: this.vpcSubnets,
      target: this,
    });
  }

  public asSecretAttachmentTarget(): secretsmanager.SecretAttachmentTargetProps {
    return {
      targetType: secretsmanager.AttachmentTargetType.RDS_DB_CLUSTER,
      targetId: this.clusterIdentifier,
    };
  }
}

Environment

  • CLI Version : 1.47.0 (build c2b499a)
  • Framework Version: 1.47.0
  • Node.js Version: v12.16.1
  • OS : macOS 10.15.5 (19F101)
  • Language (Version): TypeScript (3.9.5)

This is 🐛 Bug Report

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:12 (12 by maintainers)

github_iconTop GitHub Comments

1reaction
jogoldcommented, Jul 3, 2020

The secret rotation application creates a nested stack and CloudFormation will always attempt to update nested stacks when you start a deploy (even if they did not change).

To avoid unnecessary updates the CDK compares the templates, tags and parameters and if it can it skips deploy: https://github.com/aws/aws-cdk/blob/254556d875f9a378ac98d5c3193306250068d3c9/packages/aws-cdk/lib/api/deploy-stack.ts#L202-L203

But if any of the parameters are SSM parameters, deploy is not skipped (which makes sense): https://github.com/aws/aws-cdk/blob/254556d875f9a378ac98d5c3193306250068d3c9/packages/aws-cdk/lib/api/util/cloudformation.ts#L342-L345

Not sure what the right solution might be…

1reaction
skinny85commented, Jul 2, 2020

@asterikx Can I ask you to try something for me? Can you let me know if you get the same behavior for the “standard” DatabaseCluster from RDS, when you use the addRotationSingleUser method?

Thanks, Adam

Read more comments on GitHub >

github_iconTop Results From Across the Web

Modify an AWS Secrets Manager secret
Rotation updates both the secret in Secrets Manager and the credentials on the database or service. This keeps the secret automatically synchronized so...
Read more >
@aws-cdk/aws-secretsmanager | Yarn - Package Manager
Fast, reliable, and secure dependency management.
Read more >
AWS secrets manager, 'A previous rotation isn't complete ...
That rotation will be reattempted. I can rotate the secret without issues if i trigger the lambda function without issues. Anyone has any...
Read more >
aws_secretsmanager_secret | Resources | hashicorp/aws
Provides a resource to manage AWS Secrets Manager secret metadata. ... you fail to update will break as soon as the old credentials...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found