(secretsmanager): using replicated secrets (permissions)

General Issue

cannot grant access in both primary and replicated regions

The Question

Enjoying CDK thanks.

I have created a secret via the CDK and I want to grant read access to it in lamdas that are deployed as part of the same app to different regions.

App looks something like this

const stage = 'dev'
const tableName = 'test-api-dynamo-table';
const primaryDomain = 'dev.mydomain.com'
const replicaRegions = ['ap-southeast-2'] // 1 for now more in the future

const dynamo = new DynamoStack(app, `${stage}-dynamo-db`, {
  replicaRegions,
  stage,
  env: usEast1Env,
  tableName
})

const secretStack = new SecretsManagerStack(app, `${stage}-secret`, {
  secretName: 'AuthToken',
  env: usEast1Env,
  replicaRegions,
  rotateInDays: 15
})

const apiStackUSEast = new ApiStack(app, `${stage}-${usEast1Env.region}-api`, {
  stage,
  env: usEast1Env,
  dynamoTable: dynamo.dynamoTable,
  tableName,
  primaryDomain,
  secret: secretStack.secret
});

const apiStackAPSouthEast2 = new ApiStack(app, `${stage}-${apSouthEast2Env.region}-api`, {
  stage,
  env: apSouthEast2Env,
  dynamoTable: dynamo.dynamoTable,
  tableName,
  primaryDomain,
  secret: secretStack.secret
})

1st attempt

I tried passing the secret in from the secret stack to my Api stack and grant access using the method found in the docs

   ... this code lives in an API stack that is deployed to multiple regions including the region the primary secret is deployed in
    const secret: Secret = props.secret

    // 1st attempt does not work in replicated zones
    const nameOnly = new lambda.Function(this, 'NameOnly', {
      runtime: lambda.Runtime.NODEJS_14_X,
      handler: 'name-only.handler',
      code: lambda.Code.fromAsset('name-only'),
      layers: [apiLayer],
      environment: {
        TABLE_NAME: props.tableName,
        AUTH_TOKEN_SECRET_KEY_ARN: secret.secretArn
      }
    })
    secret.grantRead(nameOnly)

The issue here is it only works for the primary region, not the replicated regions.

2nd attempt

   ... this code lives in an API stack that is deployed to multiple regions including the region the primary secret is deployed in
    // 2nd attempt create the secret in the region from the secret name
    const secretInRegion = secretsmanager.Secret.fromSecretNameV2(this, 'secret', props.secret.secretName)

    const nameOnly = new lambda.Function(this, 'NameOnly', {
      runtime: lambda.Runtime.NODEJS_14_X,
      handler: 'name-only.handler',
      code: lambda.Code.fromAsset('name-only'),
      layers: [apiLayer],
      environment: {
        TABLE_NAME: props.tableName,
        AUTH_TOKEN_SECRET_KEY_ARN: secretInRegion.secretArn
      }
    })
    secretInRegion.grantRead(nameOnly)

The issue here is that this only works in the replicate regions 😒,

policy generated in the replicated regions deployed api (works)

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "secretsmanager:GetSecretValue",
                "secretsmanager:DescribeSecret"
            ],
            "Resource": "arn:aws:secretsmanager:us-east-1:477532313330:secret:AuthToken-??????",
            "Effect": "Allow"
        }
    ]
}

policy generated in the primary deployed api (does not work)

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "secretsmanager:GetSecretValue",
        "secretsmanager:DescribeSecret"
      ],
      "Resource": "arn:aws:secretsmanager:us-east-1:477532313330:secret:AuthToken-1aoeve-??????",
      "Effect": "Allow"
    }
  ]
}

The problem with the primary region now is that it includes both ??? and the random key (bug?)

Not sure if this is a bug or feature request, I feel like my second attempt should have worked. Would be great if there was a set way to grant access to the secret even in replicated regions. I feel like this would be possible to add as a feature.

Maybe I missed something and there is a better way.

Thanks in advance!

CDK CLI Version

1.130.0

Framework Version

No response

Node.js Version

14

OS

OSX

Language

Typescript

Language Version

4.3.5

Other information

No response

[kimisme9386]

@vespertilian I try to reproduce your scenario. The whole sample code is here. I use ap-northeast-1 for primary region and us-east-1 for replicated region. The snippet code is here

Next, I write a simple lambda to get secret manager for testing permission. The both of primary region and replicated region use the same method to get secret. like below:

const secretInRegion = secrets.Secret.fromSecretNameV2(this, `secret${region}`, secret.secretName);
secretInRegion.grantRead(lambdaFunc);

snippet code is here

After deploying it, the lambda permission looks like the screenshot

primary region (ap-northeast-1)

replicated region (us-east-1)

Both of them get the same suffix granted resource -?????? . According to @pahud explain it in above https://github.com/aws/aws-cdk/issues/17382#issuecomment-962822084, I think it make sense.

Finally, the result of lambda execution work fine.

primary region (ap-northeast-1)

replicated region (us-east-1)

According to my lab, the granted resource is partial arn. it contains six ? mark which ? mark in arn is wildcard character. The document is here. So I think it make sense and work fine.

But I don’t know why your primary region doesn’t work. Maybe reference my repository to check it. If the problem still occur, I hope you can offer more sample code. thanks.

[vespertilian]

Yes.

This works in the replicated region, but not in the primary region.

const secretInRegion = secretsmanager.Secret.fromSecretNameV2(this, 'secret', props.secret.secretName)
secret.grantRead(lambda) // works replicated ap-southeast-2

fromSecretName returns an invalid arn for the primary region arn:aws:secretsmanager:us-east-1:477532313330:secret:AuthToken-1aoeve-?????? it has both 1aoeve and ??? but it should just have one or the other.

So it’s not consistent for all regions, also it suffers from the fact it uses the partial ARN, as that is what fromSecretNameV2 returns.

I could probably get what I had to work by just checking if I am in the primary region, and using the standard secret in that region, and fromSecretName in the others. The SSM store and specific ARNs for all regions feel like it might be better.