Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

(aws-cdk): After CDK v2 upgrade, my Gitlab CI/CD started failing

See original GitHub issue

General Issue

After CDK v2 upgrade, my Gitlab CI/CD started failing

The Question

Hello,

I tarted migrating my CDK v1 repos/apps to CDK v2, and my Gitlab CI/CD pipeline started failing after cdk diff command. I’m not getting any errors when I run ci/cd jobs (on the same runner) with CDK v1. Any input is appreciated.

I did run cdk boostrap beforehand which created new S3 buckets.

image

Running with gitlab-runner 14.5.1 (de104fcd)
  on it-gitlab-runner-11282021-i-0c0762964cee7b6f3 yiFuzyWh
Preparing the "docker+machine" executor
00:19
Using Docker executor with image code.acme.com:5001/acme/images/alpine-aws-cdk-lib:2.0.0 ...
Authenticating with credentials from job payload (GitLab Registry)
Pulling docker image code.acme.com:5001/acme/images/alpine-aws-cdk-lib:2.0.0 ...
Using docker image sha256:a965de674d4e3fe3cd42ad844710d2d8a4d5fd2df7d98c89e7a17a1dce4dc09c for code.acme.com:5001/acme/images/alpine-aws-cdk-lib:2.0.0 with digest code.acme.com:5001/acme/images/alpine-aws-cdk-lib@sha256:7ce4125cfa6d5d10ce711e6d8671f81bef18c9a7e33f3ec42d789d2dc930bdb0 ...
Preparing environment
00:02
Running on runner-yifuzywh-project-6307-concurrent-0 via runner-yifuzywh-gitlab-runner-1638545703-be16bac1...
Getting source from Git repository
00:01
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/acme/cf/cloudfront-factory/.git/
Created fresh repository.
Checking out 4fbc13d7 as v2-main...
Skipping Git submodules setup
Restoring cache
00:09
Checking cache for default...
Downloading cache.zip from https://sc-123456789012-pp-6qq5ww7pf2co-runnercachebucket-yzp20ogu1gd.s3.dualstack.us-west-2.amazonaws.com/project/6307/default 
Successfully extracted cache
Executing "step_script" stage of the job script
01:08
Using docker image sha256:a965de674d4e3fe3cd42ad844710d2d8a4d5fd2df7d98c89e7a17a1dce4dc09c for code.acme.com:5001/acme/images/alpine-aws-cdk-lib:2.0.0 with digest code.acme.com:5001/acme/images/alpine-aws-cdk-lib@sha256:7ce4125cfa6d5d10ce711e6d8671f81bef18c9a7e33f3ec42d789d2dc930bdb0 ...
$ npm install
removed 7 packages, and audited 735 packages in 2s
25 packages are looking for funding
  run `npm fund` for details
found 0 vulnerabilities
$ cdk doctor
ℹ️ CDK Version: 2.0.0 (build 4b6ce31)
ℹ️ AWS environment variables:
  - AWS_DEFAULT_REGION = us-west-2
  - AWS_PAGER = 
  - AWS_STS_REGIONAL_ENDPOINTS = regional
  - AWS_NODEJS_CONNECTION_REUSE_ENABLED = 1
ℹ️ CDK environment variables:
  - CDK_DEFAULT_ACCOUNT = 123456789012
  - CDK_DEFAULT_REGION = us-west-2
$ cp ./cfg/dummy/dummy.cfg.ts ./cfg/selected/config.ts
$ npm run build
> cloudfront-factory@0.1.0 build
> tsc
$ cdk synth --all
Successfully synthesized to /builds/acme/cf/cloudfront-factory/cdk.out
Supply a stack id (dummy-lambda-edge, dummyDynamicCFStack, dummyStaticCFStack) to display its template.
$ cdk list
dummy-lambda-edge
dummyDynamicCFStack
dummyStaticCFStack
$ cdk -vvv diff
CDK toolkit version: 2.0.0 (build 4b6ce31)
Command line arguments: {
  _: [ 'diff' ],
  v: 3,
  verbose: 3,
  defaultAccount: 123456789012,
  defaultRegion: 'us-west-2',
  lookups: true,
  'ignore-errors': false,
  ignoreErrors: false,
  json: false,
  j: false,
  debug: false,
  ec2creds: undefined,
  i: undefined,
  'version-reporting': undefined,
  versionReporting: undefined,
  'path-metadata': true,
  pathMetadata: true,
  'asset-metadata': true,
  assetMetadata: true,
  'role-arn': undefined,
  r: undefined,
  roleArn: undefined,
  staging: true,
  'no-color': false,
  noColor: false,
  'context-lines': 3,
  contextLines: 3,
  strict: false,
  'security-only': false,
  securityOnly: false,
  fail: false,
  '$0': '/usr/local/bin/cdk'
}
cdk.json: {
  "app": "npx ts-node --prefer-ts-exts bin/cloudfront-factory.ts",
  "watch": {
    "include": [
      "**"
    ],
    "exclude": [
      "README.md",
      "cdk*.json",
      "**/*.d.ts",
      "**/*.js",
      "tsconfig.json",
      "package*.json",
      "yarn.lock",
      "node_modules",
      "test"
    ]
  },
  "context": {
    "@aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId": true,
    "@aws-cdk/core:stackRelativeExports": true,
    "@aws-cdk/aws-rds:lowercaseDbIdentifier": true,
    "@aws-cdk/aws-lambda:recognizeVersionProps": true,
    "@aws-cdk/aws-cloudfront:defaultSecurityPolicyTLSv1.2_2021": true
  }
}
merged settings: {
  versionReporting: true,
  pathMetadata: true,
  output: 'cdk.out',
  app: 'npx ts-node --prefer-ts-exts bin/cloudfront-factory.ts',
  watch: {
    include: [ '**' ],
    exclude: [
      'README.md',
      'cdk*.json',
      '**/*.d.ts',
      '**/*.js',
      'tsconfig.json',
      'package*.json',
      'yarn.lock',
      'node_modules',
      'test'
    ]
  },
  context: {
    '@aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId': true,
    '@aws-cdk/core:stackRelativeExports': true,
    '@aws-cdk/aws-rds:lowercaseDbIdentifier': true,
    '@aws-cdk/aws-lambda:recognizeVersionProps': true,
    '@aws-cdk/aws-cloudfront:defaultSecurityPolicyTLSv1.2_2021': true
  },
  debug: false,
  assetMetadata: true,
  toolkitBucket: {},
  staging: true,
  bundlingStacks: [ '*' ],
  lookups: true
}
Determining if we're on an EC2 instance.
Looks like an EC2 instance.
Toolkit stack: CDKToolkit
Setting "CDK_DEFAULT_REGION" environment variable to us-west-2
Resolving default credentials
Retrieved account ID 123456789012 from disk cache
Setting "CDK_DEFAULT_ACCOUNT" environment variable to 123456789012
context: {
  '@aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId': true,
  '@aws-cdk/core:stackRelativeExports': true,
  '@aws-cdk/aws-rds:lowercaseDbIdentifier': true,
  '@aws-cdk/aws-lambda:recognizeVersionProps': true,
  '@aws-cdk/aws-cloudfront:defaultSecurityPolicyTLSv1.2_2021': true,
  'aws:cdk:enable-path-metadata': true,
  'aws:cdk:enable-asset-metadata': true,
  'aws:cdk:version-reporting': true,
  'aws:cdk:bundling-stacks': [ '*' ]
}
outdir: cdk.out
env: {
  CDK_DEFAULT_REGION: 'us-west-2',
  CDK_DEFAULT_ACCOUNT: '123456789012',
  CDK_CONTEXT_JSON: '{"@aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId":true,"@aws-cdk/core:stackRelativeExports":true,"@aws-cdk/aws-rds:lowercaseDbIdentifier":true,"@aws-cdk/aws-lambda:recognizeVersionProps":true,"@aws-cdk/aws-cloudfront:defaultSecurityPolicyTLSv1.2_2021":true,"aws:cdk:enable-path-metadata":true,"aws:cdk:enable-asset-metadata":true,"aws:cdk:version-reporting":true,"aws:cdk:bundling-stacks":["*"]}',
  CDK_OUTDIR: 'cdk.out',
  CDK_CLI_ASM_VERSION: '15.0.0',
  CDK_CLI_VERSION: '2.0.0'
}
Stack dummy-lambda-edge
Reading existing template for stack dummy-lambda-edge.
Retrieved account ID 123456789012 from disk cache
Assuming role 'arn:aws:iam::123456789012:role/cdk-hnb659fds-deploy-role-123456789012-us-east-1'.
[AWS cloudformation 200 0.382s 0 retries] describeStacks({ StackName: 'dummy-lambda-edge' })
[AWS cloudformation 200 0.34s 0 retries] getTemplate({
  StackName: 'dummy-lambda-edge',
  TemplateStage: 'Original'
})
Parameters
[-] Parameter AssetParametersd04c3782a3848dfcb10b99bbf4fe32595222026fbae524406c10d7c52c2b9277S3BucketBD00DE50: {"Type":"String","Description":"S3 bucket for asset \"d04c3782a3848dfcb10b99bbf4fe32595222026fbae524406c10d7c52c2b9277\""}
[-] Parameter AssetParametersd04c3782a3848dfcb10b99bbf4fe32595222026fbae524406c10d7c52c2b9277S3VersionKeyD071EABC: {"Type":"String","Description":"S3 key for asset version \"d04c3782a3848dfcb10b99bbf4fe32595222026fbae524406c10d7c52c2b9277\""}
[-] Parameter AssetParametersd04c3782a3848dfcb10b99bbf4fe32595222026fbae524406c10d7c52c2b9277ArtifactHash54FC89D7: {"Type":"String","Description":"Artifact hash for asset \"d04c3782a3848dfcb10b99bbf4fe32595222026fbae524406c10d7c52c2b9277\""}
[+] Parameter BootstrapVersion BootstrapVersion: {"Type":"AWS::SSM::Parameter::Value<String>","Default":"/cdk-bootstrap/hnb659fds/version","Description":"Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]"}
Resources
[-] AWS::Lambda::Version dummylambdaedge0CurrentVersion743479E7e3d55a6f9667647a064264ae04e4d212 destroy
[+] AWS::Lambda::Version dummy-lambda-edge-0/CurrentVersion dummylambdaedge0CurrentVersion743479E7c5fef02dd39b1e206b38c386cafab30a 
[~] AWS::Lambda::Function dummy-lambda-edge-0 dummylambdaedge085A8E25C 
 └─ [~] Code
     ├─ [~] .S3Bucket:
     │   └─ @@ -1,3 +1,1 @@
     │      [-] {
     │      [-]   "Ref": "AssetParametersd04c3782a3848dfcb10b99bbf4fe32595222026fbae524406c10d7c52c2b9277S3BucketBD00DE50"
     │      [-] }
     │      [+] "cdk-hnb659fds-assets-123456789012-us-east-1"
     └─ [~] .S3Key:
         └─ @@ -1,33 +1,1 @@
            [-] {
            [-]   "Fn::Join": [
            [-]     "",
            [-]     [
            [-]       {
            [-]         "Fn::Select": [
            [-]           0,
            [-]           {
            [-]             "Fn::Split": [
            [-]               "||",
            [-]               {
            [-]                 "Ref": "AssetParametersd04c3782a3848dfcb10b99bbf4fe32595222026fbae524406c10d7c52c2b9277S3VersionKeyD071EABC"
            [-]               }
            [-]             ]
            [-]           }
            [-]         ]
            [-]       },
            [-]       {
            [-]         "Fn::Select": [
            [-]           1,
            [-]           {
            [-]             "Fn::Split": [
            [-]               "||",
            [-]               {
            [-]                 "Ref": "AssetParametersd04c3782a3848dfcb10b99bbf4fe32595222026fbae524406c10d7c52c2b9277S3VersionKeyD071EABC"
            [-]               }
            [-]             ]
            [-]           }
            [-]         ]
            [-]       }
            [-]     ]
            [-]   ]
            [-] }
            [+] "d04c3782a3848dfcb10b99bbf4fe32595222026fbae524406c10d7c52c2b9277.zip"
[~] AWS::SSM::Parameter dummy-lambda-edge-0/Parameter dummylambdaedge0ParameterA3A1A1CE 
 └─ [~] Value
     └─ [~] .Ref:
         ├─ [-] dummylambdaedge0CurrentVersion743479E7e3d55a6f9667647a064264ae04e4d212
         └─ [+] dummylambdaedge0CurrentVersion743479E7c5fef02dd39b1e206b38c386cafab30a
Other Changes
[+] Unknown Rules: {"CheckBootstrapVersion":{"Assertions":[{"Assert":{"Fn::Not":[{"Fn::Contains":[["1","2","3","4","5"],{"Ref":"BootstrapVersion"}]}]},"AssertDescription":"CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI."}]}}
Stack dummyDynamicCFStack
Reading existing template for stack dummyDynamicCFStack.
Retrieved account ID 123456789012 from disk cache
Assuming role 'arn:aws:iam::123456789012:role/cdk-hnb659fds-deploy-role-123456789012-us-west-2'.
[AWS cloudformation 200 0.148s 0 retries] describeStacks({ StackName: 'dummyDynamicCFStack' })
[AWS cloudformation 200 0.123s 0 retries] getTemplate({ StackName: 'dummyDynamicCFStack', TemplateStage: 'Original' })
Parameters
[+] Parameter BootstrapVersion BootstrapVersion: {"Type":"AWS::SSM::Parameter::Value<String>","Default":"/cdk-bootstrap/hnb659fds/version","Description":"Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]"}
Other Changes
[+] Unknown Rules: {"CheckBootstrapVersion":{"Assertions":[{"Assert":{"Fn::Not":[{"Fn::Contains":[["1","2","3","4","5"],{"Ref":"BootstrapVersion"}]}]},"AssertDescription":"CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI."}]}}
Stack dummyStaticCFStack
Reading existing template for stack dummyStaticCFStack.
Retrieved account ID 123456789012 from disk cache
Assuming role 'arn:aws:iam::123456789012:role/cdk-hnb659fds-deploy-role-123456789012-us-west-2'.
[AWS cloudformation 200 0.177s 0 retries] describeStacks({ StackName: 'dummyStaticCFStack' })
[AWS cloudformation 200 0.118s 0 retries] getTemplate({
  StackName: 'dummyStaticCFStack',
  TemplateStage: 'Original'
})
Parameters
[-] Parameter AssetParameters45b7ed524ce2b119dd4f2b8642ae8bfaf0df45bc6bd705072ae4ee6d1a999241S3BucketF1BC72A7: {"Type":"String","Description":"S3 bucket for asset \"45b7ed524ce2b119dd4f2b8642ae8bfaf0df45bc6bd705072ae4ee6d1a999241\""}
[-] Parameter AssetParameters45b7ed524ce2b119dd4f2b8642ae8bfaf0df45bc6bd705072ae4ee6d1a999241S3VersionKey7AD83AC7: {"Type":"String","Description":"S3 key for asset version \"45b7ed524ce2b119dd4f2b8642ae8bfaf0df45bc6bd705072ae4ee6d1a999241\""}
[-] Parameter AssetParameters45b7ed524ce2b119dd4f2b8642ae8bfaf0df45bc6bd705072ae4ee6d1a999241ArtifactHash1B3D1B80: {"Type":"String","Description":"Artifact hash for asset \"45b7ed524ce2b119dd4f2b8642ae8bfaf0df45bc6bd705072ae4ee6d1a999241\""}
[+] Parameter BootstrapVersion BootstrapVersion: {"Type":"AWS::SSM::Parameter::Value<String>","Default":"/cdk-bootstrap/hnb659fds/version","Description":"Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]"}
Resources
[~] Custom::CrossRegionStringParameterReader dummy-lambda-edge-0/ArnReader dummylambdaedge0ArnReader1406476B 
 └─ [~] RefreshToken
     ├─ [-] dummylambdaedge0CurrentVersion743479E7e3d55a6f9667647a064264ae04e4d212
     └─ [+] dummylambdaedge0CurrentVersion743479E7c5fef02dd39b1e206b38c386cafab30a
[~] AWS::Lambda::Function Custom::CrossRegionStringParameterReaderCustomResourceProvider/Handler CustomCrossRegionStringParameterReaderCustomResourceProviderHandler65B5F33A 
 └─ [~] Code
     ├─ [~] .S3Bucket:
     │   └─ @@ -1,3 +1,1 @@
     │      [-] {
     │      [-]   "Ref": "AssetParameters45b7ed524ce2b119dd4f2b8642ae8bfaf0df45bc6bd705072ae4ee6d1a999241S3BucketF1BC72A7"
     │      [-] }
     │      [+] "cdk-hnb659fds-assets-123456789012-us-west-2"
     └─ [~] .S3Key:
         └─ @@ -1,33 +1,1 @@
            [-] {
            [-]   "Fn::Join": [
            [-]     "",
            [-]     [
            [-]       {
            [-]         "Fn::Select": [
            [-]           0,
            [-]           {
            [-]             "Fn::Split": [
            [-]               "||",
            [-]               {
            [-]                 "Ref": "AssetParameters45b7ed524ce2b119dd4f2b8642ae8bfaf0df45bc6bd705072ae4ee6d1a999241S3VersionKey7AD83AC7"
            [-]               }
            [-]             ]
            [-]           }
            [-]         ]
            [-]       },
            [-]       {
            [-]         "Fn::Select": [
            [-]           1,
            [-]           {
            [-]             "Fn::Split": [
            [-]               "||",
            [-]               {
            [-]                 "Ref": "AssetParameters45b7ed524ce2b119dd4f2b8642ae8bfaf0df45bc6bd705072ae4ee6d1a999241S3VersionKey7AD83AC7"
            [-]               }
            [-]             ]
            [-]           }
            [-]         ]
            [-]       }
            [-]     ]
            [-]   ]
            [-] }
            [+] "45b7ed524ce2b119dd4f2b8642ae8bfaf0df45bc6bd705072ae4ee6d1a999241.zip"
Other Changes
[+] Unknown Rules: {"CheckBootstrapVersion":{"Assertions":[{"Assert":{"Fn::Not":[{"Fn::Contains":[["1","2","3","4","5"],{"Ref":"BootstrapVersion"}]}]},"AssertDescription":"CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI."}]}}
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: exit code 1

CDK CLI Version

2.0.0 (build 4b6ce31)

Framework Version

No response

Node.js Version

v16.13.1

OS

OSx 11.6.1 Big Sur

Language

Typescript

Language Version

3.9.7

Other information

No response

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:7 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
njlynchcommented, Dec 6, 2021

Fix will go out with next v2 release.

1reaction
njlynchcommented, Dec 6, 2021

You’re correct. The way our CLI handled expired feature flags (like aws-cdk:enableDiffNoFail) was inconsistent with how the CDK library behavior; the latter correctly defaulted any expired feature flags to true, whereas the former defaulted to the (currently incorrect) “default” value, which was false.

Fix is https://github.com/aws/aws-cdk/pull/17862

Read more comments on GitHub >

github_iconTop Results From Across the Web

Continuous integration and delivery (CI/CD) using CDK ...
The CDK Toolkit upgrades your existing bootstrap stack or creates a new one if necessary. To bootstrap an environment that can provision an...
Read more >
AWS CDK && Gitlab CI - Unable to determine the default AWS ...
Situtation: When I cdk deploy locally, I can deploy the stack to the correct account. However, via gitlab-ci.yml, I get the error: Error: ......
Read more >
AWS CDK Pipelines: Real-World Tips and Tricks (Part 2)
In this article I'll share with you some useful tips and tricks when using AWS CDK Pipelines that go beyond the simple demos...
Read more >
Hey CDK, how do cross-account deployments work?
The AWS CDK makes it easy to deploy your application, regardless if ... Because CI/CD systems need a lot of permissions in your...
Read more >
Automate AWS lambda function deployments with AWS CDK
This tutorial covers: Defining your AWS CDK application and the AWS Lambda handler; Manually building and deploying your CDK application ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

(assert): there should be something like ```haveManifest(...)``` and ```haveHelmChart``` or a matcher like ```k8sobjectlike``` that allow unit tests for kubernetes objects

Next page

Lambda@Edge