Codepipeline Internal Failure

❓ General Issue

The Question

I deployed a codepipeline with a Lambda + cdk itself. But it failed at wired situation. Lambda + cdk repo: https://github.com/ookangzheng/codebuild-test

Error log

 9/14 | 8:13:10 PM | CREATE_COMPLETE      | AWS::IAM::Policy            | FrontendPipeline/Role/DefaultPolicy (FrontendPipelineRoleDefaultPolicy1C01A250) 
 10/14 | 8:13:10 PM | CREATE_COMPLETE      | AWS::IAM::Policy            | Sygna-Bridge-CodeBuildRole-dev/DefaultPolicy (SygnaBridgeCodeBuildRoledevDefaultPolicy172AB02C) 
 10/14 | 8:13:13 PM | CREATE_IN_PROGRESS   | AWS::CodePipeline::Pipeline | FrontendPipeline (FrontendPipeline0AA6540B) 
 11/14 | 8:13:14 PM | CREATE_FAILED        | AWS::CodePipeline::Pipeline | FrontendPipeline (FrontendPipeline0AA6540B) Internal Failure
        new Pipeline (/Users/loll/Desktop/codebuild/node_modules/@aws-cdk/aws-codepipeline/lib/pipeline.ts:252:26)
 12/14 | 8:13:14 PM | CREATE_FAILED        | AWS::KMS::Alias             | FrontendPipeline/ArtifactsBucketEncryptionKeyAlias (FrontendPipelineArtifactsBucketEncryptionKeyAlias9BA285A5) Resource creation cancelled
        new Alias (/Users/loll/Desktop/codebuild/node_modules/@aws-cdk/aws-kms/lib/alias.ts:153:22)

 12/14 | 8:13:15 PM | ROLLBACK_IN_PROGRESS | AWS::CloudFormation::Stack  | CodebuildStack-dev The following resource(s) failed to create: [FrontendPipelineArtifactsBucketEncryptionKeyAlias9BA285A5, FrontendPipeline0AA6540B]. . Rollback requested by user.

Environment

• CDK CLI Version: 1.31.0
• Module Version: 1.31.0
• OS: OSX Catalina
• Language: TypeScript

Sample code

File: ./lib/codebuild.ts

import * as cdk from '@aws-cdk/core';
import { Bucket } from '@aws-cdk/aws-s3';
import { Pipeline, Artifact } from '@aws-cdk/aws-codepipeline';
import { GitHubSourceAction, GitHubTrigger, CodeBuildAction } from '@aws-cdk/aws-codepipeline-actions';
import { Project, BuildSpec } from '@aws-cdk/aws-codebuild';
import { Role, ServicePrincipal, ManagedPolicy} from '@aws-cdk/aws-iam';

export class CodebuildStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const pipeline = new Pipeline(this, 'FrontendPipeline', {
      pipelineName: 'deploy-codebuild-application',
    });

    const sourceStage = pipeline.addStage({
      stageName: 'Source'
    });
  
    const buildStage = pipeline.addStage({
      stageName: 'Build',
      placement: {
        justAfter: sourceStage
      }
    });

    const sourceOutput = new Artifact();
    const sourceAction = new GitHubSourceAction({
      actionName: 'GitHub',
      owner: 'xxxx',
      repo: 'codebuild-test',
      oauthToken: cdk.SecretValue.secretsManager('GithubToken'),
      output: sourceOutput,
      branch: 'master', 
      trigger: GitHubTrigger.POLL
    });
  
    sourceStage.addAction(sourceAction);

    const role = new Role(this, 'CodeBuildRole-dev', {
      assumedBy: new ServicePrincipal('codebuild.amazonaws.com'),
      managedPolicies: [
        ManagedPolicy.fromAwsManagedPolicyName('AmazonS3FullAccess'),
      ]
    });

    const codeBuild = new Project(this, 'CodeBuildProject', {
      role,
      buildSpec: BuildSpec.fromObject({
        "version": 0.2,
        "phases": {
          "install": {
            "runtime-versions": {
              "nodejs": 12
            },
            "commands": [
              "echo installing dependencies",
              "npm -g cdk",
              "npm install",
              'printf "[profile sls-dev-jp] \nregion=ap-northeast-1\n" >> ~/.aws/config',
              'printf "[sls-dev-jp] \naws_access_key_id=xxxxx25A  \naws_secret_access_key=QXOXAaf" >> ~/.aws/credentials',
              "echo cd into lambda and install lambda",
              "cd lambda",
              "npm install"
            ]
          },
          "build": {
            "commands": [
              "echo Build started on `date`",
            ],
            "artifacts": {
              "files": [
                "**/*"
              ],
              "base-directory": "./",
              "discard-paths": "yes"
            }
          },
          "post_build": {
            "commands": [
              "echo BUILD COMPLETE and do cdk deploy",
              `cdk deploy --profile sls-dev-jp`,
              "echo cdk deploy complete"
            ]
          }
        }
      })
    });

    const buildAction = new CodeBuildAction({
      actionName: 'Build',
      input: sourceOutput,
      project: codeBuild
    });
    
    buildStage.addAction(buildAction);
  }
}

Thanks

[ookangzheng]

It does work though I think cdk cloudformation should throw out entire error instead of internal failure

I got look into cloudwatch logs and it said what cause internal failure

"errorCode": "AccessDenied",
"errorMessage": "User: arn:aws:iam::1234:user/bridge-dev is not authorized to perform: secretsmanager:GetSecretValue on resource: arn:aws:secretsmanager:ap-northeast-1:1234:secret:okzGitHubToken-EDxsF8",

[skinny85]

Unfortunately, this error is thrown in the CloudFormation service, so there’s nothing we can do on the CDK side with this.

The good news is that CloudFormation has an issue in their backlog to handle this error gracefully, so hopefully this will be improved soon.

Is there anything else we can help you with in this issue @ookangzheng ?