Setting existing:true on S3 bucket causes deployment to fail

Bug Report

Setting existing:true on S3 bucket causes deployment to fail

Description

1. What did you do? Added existing:true to S3 function. The deployment works correctly for a new bucket.

2. What happened? The deployment fails with:

...
CloudFormation - UPDATE_IN_PROGRESS - AWS::CloudFormation::Stack - pdfcountdebug-dev
CloudFormation - CREATE_IN_PROGRESS - AWS::Logs::LogGroup - S3uploadedLogGroup
CloudFormation - CREATE_IN_PROGRESS - AWS::IAM::Role - IamRoleLambdaExecution
CloudFormation - CREATE_IN_PROGRESS - AWS::IAM::Role - IamRoleCustomResourcesLambdaExecution
CloudFormation - CREATE_IN_PROGRESS - AWS::Logs::LogGroup - S3uploadedLogGroup
CloudFormation - CREATE_IN_PROGRESS - AWS::IAM::Role - IamRoleCustomResourcesLambdaExecution
CloudFormation - CREATE_COMPLETE - AWS::Logs::LogGroup - S3uploadedLogGroup
CloudFormation - CREATE_IN_PROGRESS - AWS::IAM::Role - IamRoleLambdaExecution
CloudFormation - CREATE_COMPLETE - AWS::IAM::Role - IamRoleCustomResourcesLambdaExecution
CloudFormation - CREATE_COMPLETE - AWS::IAM::Role - IamRoleLambdaExecution
CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Function - CustomDashresourceDashexistingDashs3LambdaFunction
CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Function - S3uploadedLambdaFunction
CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Function - CustomDashresourceDashexistingDashs3LambdaFunction
CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Function - S3uploadedLambdaFunction
CloudFormation - CREATE_COMPLETE - AWS::Lambda::Function - CustomDashresourceDashexistingDashs3LambdaFunction
CloudFormation - CREATE_COMPLETE - AWS::Lambda::Function - S3uploadedLambdaFunction
CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Version - S3uploadedLambdaVersioneyfdcHxB4Kq0aaxjAYGHiQwjGYOm5C3Tr4KbmAP3pY
CloudFormation - CREATE_IN_PROGRESS - Custom::S3 - S3uploadedCustomS31
CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Version - S3uploadedLambdaVersioneyfdcHxB4Kq0aaxjAYGHiQwjGYOm5C3Tr4KbmAP3pY
CloudFormation - CREATE_COMPLETE - AWS::Lambda::Version - S3uploadedLambdaVersioneyfdcHxB4Kq0aaxjAYGHiQwjGYOm5C3Tr4KbmAP3pY
CloudFormation - CREATE_IN_PROGRESS - Custom::S3 - S3uploadedCustomS31
CloudFormation - CREATE_FAILED - Custom::S3 - S3uploadedCustomS31
CloudFormation - UPDATE_ROLLBACK_IN_PROGRESS - AWS::CloudFormation::Stack - pdfcountdebug-dev
CloudFormation - UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS - AWS::CloudFormation::Stack - pdfcountdebug-dev
CloudFormation - DELETE_IN_PROGRESS - AWS::CloudFormation::CustomResource - S3uploadedCustomS31
CloudFormation - DELETE_SKIPPED - AWS::Lambda::Version - S3uploadedLambdaVersioneyfdcHxB4Kq0aaxjAYGHiQwjGYOm5C3Tr4KbmAP3pY
CloudFormation - DELETE_FAILED - AWS::CloudFormation::CustomResource - S3uploadedCustomS31
...

and then rolls back everything else. The rollback for S3uploadedCustomS31 fails of course.

1. What should’ve happened? Deployment should complete. The step that fails is the custom resource handler that attaches the necessary policies to the function handler and the existing bucket. It is not clear which step fails. I suspect it is attaching the permission to the lambda.

2. What’s the content of your serverless.yml file? Reduced file for clarity

service: pdfcountdebug

provider:
  name: aws
  runtime: nodejs10.x
  stage: ${opt:stage, 'dev'}
  region: ${opt:region, 'sa-east-1'}

  iamRoleStatements:
    - Effect: "Allow"
      Action:
        - "s3:GetObject"
        - "s3:DeleteObject"
      Resource: "*"

functions:
  s3uploaded:
    handler: src/uploaded.handler
    events:
      - s3: 
          bucket: hookohm.test.doutore-documentacao
          event: s3:ObjectCreated:*
          rules:
            - suffix: .pdf
          existing: true

1. What’s the output you get when you use the SLS_DEBUG=* environment variable (e.g. SLS_DEBUG=* serverless deploy) sls does not fail, but the AWS deployment does.

Similar or dependent issues:

• #6638 This gave a different error.

Because the whole stack rolls back due to the error, there are no AWS logs to track down the issue. Debugging CloudFormation custom resources is a bit of a dark art.

1. Environment

  Your Environment Information ---------------------------
     Operating System:          win32
     Node Version:              8.11.3
     Framework Version:         1.53.0
     Plugin Version:            3.1.0
     SDK Version:               2.1.1
     Components Core Version:   1.1.1
     Components CLI Version:    1.2.3

[ihorfito]

try to check: s3 -> bucket -> configuration -> events delete existing events it helps when you cannot deploy sls application

[pgrzesik]

@jazwiecki - The error you’re seeing it unrelated to S3 bucket - if you don’t specify a role for Lambda functions to use, one will be created for you by Serverless Framework and used by your functions. You can avoid that by specifying an existing role. You can read more about it here: https://www.serverless.com/framework/docs/providers/aws/guide/iam#iam