Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Unable to add secret rotation to an RDS database

See original GitHub issue

After using .addRotationSingleUser() on a database instance, CDK fails to deploy the rotation application stack successfully.

Reproduction Steps

Code Sample:

import * as cdk from "@aws-cdk/core";
import { Vpc, InstanceClass, InstanceSize, InstanceType } from "@aws-cdk/aws-ec2";
import { DatabaseInstance, DatabaseInstanceEngine } from "@aws-cdk/aws-rds";
import { Duration } from "@aws-cdk/core";

export class AwsCdkDatabaseSecretErrorReplicationStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const vpc = new Vpc(this, "TestVPC");

    const database = new DatabaseInstance(this, "testDatbase", {
      instanceIdentifier: "testDatabase",
      engine: DatabaseInstanceEngine.POSTGRES,
      engineVersion: "10.6",
      instanceClass: InstanceType.of(InstanceClass.BURSTABLE2, InstanceSize.MICRO),
      masterUsername: "gladmin",
      vpc,
      backupRetention: Duration.days(30),
      multiAz: true
    });

    database.addRotationSingleUser();
  }
}

Error Log

 25/35 | 6:58:38 PM | CREATE_FAILED        | AWS::CloudFormation::Stack                  | testDatbase/RotationSingleUser (testDatbaseRotationSingleUser544CCB3F) Embedded stack arn:aws:cloudformation:us-east-1:456718055477:stack/AwsCdkDatabaseSecretErrorReplicationStack-testDatbaseRotationSingleUser544CCB3F-1J7EYPVP6B872/57bcda50-917f-11ea-8713-12cf4a8c2bc2 was not successfully created: The following resource(s) failed to create: [SecretsManagerRDSPostgreSQLRotationSingleUser].
	new SecretRotation (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-secretsmanager/lib/secret-rotation.ts:239:25)
	\_ DatabaseInstance.addRotationSingleUser (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-rds/lib/instance.ts:786:12)
	\_ new AwsCdkDatabaseSecretErrorReplicationStack (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/lib/aws-cdk-database-secret-error-replication-stack.ts:23:14)
	\_ Object.<anonymous> (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/bin/aws-cdk-database-secret-error-replication.ts:7:1)
	\_ Module._compile (internal/modules/cjs/loader.js:1151:30)
	\_ Module.m._compile (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/index.ts:858:23)
	\_ Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
	\_ Object.require.extensions.<computed> [as .ts] (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/index.ts:861:12)
	\_ Module.load (internal/modules/cjs/loader.js:1000:32)
	\_ Function.Module._load (internal/modules/cjs/loader.js:899:14)
	\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
	\_ main (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/bin.ts:227:14)
	\_ Object.<anonymous> (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/bin.ts:513:3)
	\_ Module._compile (internal/modules/cjs/loader.js:1151:30)
	\_ Object.Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
	\_ Module.load (internal/modules/cjs/loader.js:1000:32)
	\_ Function.Module._load (internal/modules/cjs/loader.js:899:14)
	\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
	\_ /Users/wbprice/.nvm/versions/node/v13.9.0/lib/node_modules/npm/node_modules/libnpx/index.js:268:14
 26/35 | 6:58:39 PM | CREATE_FAILED        | AWS::EC2::NatGateway                        | TestVPC/PublicSubnet2/NATGateway (TestVPCPublicSubnet2NATGatewayBE12FD22) Resource creation cancelled
	PublicSubnet.addNatGateway (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-ec2/lib/vpc.ts:1677:17)
	\_ NatGatewayProvider.configureNat (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-ec2/lib/nat.ts:173:27)
	\_ Vpc.createNatGateways (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-ec2/lib/vpc.ts:1261:14)
	\_ new Vpc (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-ec2/lib/vpc.ts:1183:14)
	\_ new AwsCdkDatabaseSecretErrorReplicationStack (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/lib/aws-cdk-database-secret-error-replication-stack.ts:10:17)
	\_ Object.<anonymous> (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/bin/aws-cdk-database-secret-error-replication.ts:7:1)
	\_ Module._compile (internal/modules/cjs/loader.js:1151:30)
	\_ Module.m._compile (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/index.ts:858:23)
	\_ Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
	\_ Object.require.extensions.<computed> [as .ts] (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/index.ts:861:12)
	\_ Module.load (internal/modules/cjs/loader.js:1000:32)
	\_ Function.Module._load (internal/modules/cjs/loader.js:899:14)
	\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
	\_ main (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/bin.ts:227:14)
	\_ Object.<anonymous> (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/bin.ts:513:3)
	\_ Module._compile (internal/modules/cjs/loader.js:1151:30)
	\_ Object.Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
	\_ Module.load (internal/modules/cjs/loader.js:1000:32)
	\_ Function.Module._load (internal/modules/cjs/loader.js:899:14)
	\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
	\_ /Users/wbprice/.nvm/versions/node/v13.9.0/lib/node_modules/npm/node_modules/libnpx/index.js:268:14
 27/35 | 6:58:39 PM | CREATE_FAILED        | AWS::EC2::NatGateway                        | TestVPC/PublicSubnet1/NATGateway (TestVPCPublicSubnet1NATGateway6A40FA74) Resource creation cancelled
	PublicSubnet.addNatGateway (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-ec2/lib/vpc.ts:1677:17)
	\_ NatGatewayProvider.configureNat (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-ec2/lib/nat.ts:173:27)
	\_ Vpc.createNatGateways (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-ec2/lib/vpc.ts:1261:14)
	\_ new Vpc (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-ec2/lib/vpc.ts:1183:14)
	\_ new AwsCdkDatabaseSecretErrorReplicationStack (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/lib/aws-cdk-database-secret-error-replication-stack.ts:10:17)
	\_ Object.<anonymous> (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/bin/aws-cdk-database-secret-error-replication.ts:7:1)
	\_ Module._compile (internal/modules/cjs/loader.js:1151:30)
	\_ Module.m._compile (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/index.ts:858:23)
	\_ Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
	\_ Object.require.extensions.<computed> [as .ts] (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/index.ts:861:12)
	\_ Module.load (internal/modules/cjs/loader.js:1000:32)
	\_ Function.Module._load (internal/modules/cjs/loader.js:899:14)
	\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
	\_ main (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/bin.ts:227:14)
	\_ Object.<anonymous> (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/bin.ts:513:3)
	\_ Module._compile (internal/modules/cjs/loader.js:1151:30)
	\_ Object.Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
	\_ Module.load (internal/modules/cjs/loader.js:1000:32)
	\_ Function.Module._load (internal/modules/cjs/loader.js:899:14)
	\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
	\_ /Users/wbprice/.nvm/versions/node/v13.9.0/lib/node_modules/npm/node_modules/libnpx/index.js:268:14
 28/35 | 6:58:39 PM | CREATE_FAILED        | AWS::RDS::DBInstance                        | testDatbase (testDatbase2F413D3C) Resource creation cancelled
	new DatabaseInstance (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/@aws-cdk/aws-rds/lib/instance.ts:870:22)
	\_ new AwsCdkDatabaseSecretErrorReplicationStack (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/lib/aws-cdk-database-secret-error-replication-stack.ts:12:22)
	\_ Object.<anonymous> (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/bin/aws-cdk-database-secret-error-replication.ts:7:1)
	\_ Module._compile (internal/modules/cjs/loader.js:1151:30)
	\_ Module.m._compile (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/index.ts:858:23)
	\_ Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
	\_ Object.require.extensions.<computed> [as .ts] (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/index.ts:861:12)
	\_ Module.load (internal/modules/cjs/loader.js:1000:32)
	\_ Function.Module._load (internal/modules/cjs/loader.js:899:14)
	\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
	\_ main (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/bin.ts:227:14)
	\_ Object.<anonymous> (/Users/wbprice/Documents/aws-cdk-database-secret-error-replication/node_modules/ts-node/src/bin.ts:513:3)
	\_ Module._compile (internal/modules/cjs/loader.js:1151:30)
	\_ Object.Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
	\_ Module.load (internal/modules/cjs/loader.js:1000:32)
	\_ Function.Module._load (internal/modules/cjs/loader.js:899:14)
	\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
	\_ /Users/wbprice/.nvm/versions/node/v13.9.0/lib/node_modules/npm/node_modules/libnpx/index.js:268:14

Environment

  • CLI Version : aws-cli/2.0.1 Python/3.7.4 Darwin/19.4.0 botocore/2.0.0dev5
  • Framework Version: 1.38.0 (build d5fa31f)
  • OS : MacOS Catalina 10.15.4 (19E287)
  • Language : English

Other

  • I created a test case here.
  • I’m deploying this into a personal account using an admin-level user. I don’t think it’s a permission issue.

This is 🐛 Bug Report

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Reactions:1
  • Comments:8 (4 by maintainers)

github_iconTop GitHub Comments

2reactions
wbpricecommented, May 9, 2020

OK, cool. I didn’t know nested stacks appeared in Cloudformation’s stack list.

image

The error suggests that the lambda name AwsCdkDatabaseSecretErrorReplicationStacktestDatbaseRotationSingleUser90948986 is too long. I’ll try again with a shorter stack name.

0reactions
jogoldcommented, May 10, 2020
Read more comments on GitHub >

github_iconTop Results From Across the Web

Set up automatic rotation for Amazon RDS, Amazon Redshift ...
Rotation is the process of periodically updating a secret. When you rotate a secret, you update the credentials in both the secret and...
Read more >
MySQL Password Rotation with AWS Secrets Manager and ...
MySQL password rotation using Amazon RDS for MySQL, AWS Secrets Manager, and AWS ... But there is an additional use case: adding new...
Read more >
AWS Secrets Manager rotation for RDS credentials
According to the comments in Rotate Amazon RDS database credentials automatically with AWS Secrets Manager | AWS Security Blog, on RDS and ...
Read more >
Achieving RDS password rotation with Secrets Manager
In this article we will explore the Secrets Manager service and use it to store a secret for an RDS database and rotate...
Read more >
2. RDS with Secrets Manager
You now need to select the secret whose permissions will be used to rotate the secret. For this Builder Session, select Use this...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

Step Machine definition lacks proper formatting

Next page

📊Tracking: migrating CloudFormation templates to the CDK