Deploy fails after removing service and upgrading to new version of serverless framework because an ApiGatewayLogGroup exists

Bug Report

Description

1. What did you do? Ran serverless deploy for a recently removed service.

2. What happened? The deployment failed because a resource was left over from a previous sls remove

3. What should’ve happened? The service should deploy.

4. What’s the content of your serverless.yml file?

org: fernando
app: enterprise
service: demo-email-form

frameworkVersion: '>=1.38.0 <2.0.0'

provider:
  name: aws
  runtime: nodejs10.x
  iamRoleStatements:
    - Effect: 'Allow'
      Action:
        - 'sns:createTopic'
        - 'sns:deleteTopic'
        - 'sns:publish'
        - 'dynamodb:putItem'
      Resource:
        - '*'
  environment: 
    PHONE_NUMBER: ${secrets:ALERT_NUMBER}

functions:
  formSubmit:
    handler: index.submit
    events:
      - http:
          path: submit
          method: post
          cors: true

1. What’s the output you get when you use the SLS_DEBUG=* environment variable (e.g. SLS_DEBUG=* serverless deploy)

From serverless deploy --verbose

Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Installing dependencies for custom CloudFormation resources...
Serverless: Safeguards Processing...
Serverless: Safeguards Results:

   Summary --------------------------------------------------

   passed - allowed-runtimes
   warned - no-wild-iam-role-statements
   passed - no-secret-env-vars

   Details --------------------------------------------------

   1) Warned - iamRoleStatement granting Resource='*'. Wildcard resources in iamRoleStatements are not permitted.
      details: http://slss.io/sg-no-wild-iam-role
      Prevent "*" permissions being used in AWS IAM Roles by checking for wildcards on Actions and Resources in grant statements.


Serverless: Safeguards Summary: 2 passed, 1 warnings, 0 errors
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Uploading service demo-email-form.zip file to S3 (77.86 KB)...
Serverless: Uploading custom CloudFormation resources...
Serverless: Validating template...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
CloudFormation - UPDATE_IN_PROGRESS - AWS::CloudFormation::Stack - demo-email-form-dev
CloudFormation - CREATE_IN_PROGRESS - AWS::Logs::LogGroup - FormSubmitLogGroup
CloudFormation - CREATE_IN_PROGRESS - AWS::Logs::LogGroup - ApiGatewayLogGroup
CloudFormation - CREATE_IN_PROGRESS - AWS::IAM::Role - IamRoleLambdaExecution
CloudFormation - CREATE_IN_PROGRESS - AWS::IAM::Role - IamRoleCustomResourcesLambdaExecution
CloudFormation - CREATE_IN_PROGRESS - AWS::ApiGateway::RestApi - ApiGatewayRestApi
CloudFormation - CREATE_IN_PROGRESS - AWS::Logs::LogGroup - FormSubmitLogGroup
CloudFormation - CREATE_IN_PROGRESS - AWS::IAM::Role - IamRoleCustomResourcesLambdaExecution
CloudFormation - CREATE_IN_PROGRESS - AWS::IAM::Role - IamRoleLambdaExecution
CloudFormation - CREATE_FAILED - AWS::Logs::LogGroup - ApiGatewayLogGroup
CloudFormation - CREATE_IN_PROGRESS - AWS::ApiGateway::RestApi - ApiGatewayRestApi
CloudFormation - CREATE_COMPLETE - AWS::Logs::LogGroup - FormSubmitLogGroup
CloudFormation - CREATE_COMPLETE - AWS::ApiGateway::RestApi - ApiGatewayRestApi
CloudFormation - CREATE_FAILED - AWS::IAM::Role - IamRoleCustomResourcesLambdaExecution
CloudFormation - CREATE_FAILED - AWS::IAM::Role - IamRoleLambdaExecution
CloudFormation - UPDATE_ROLLBACK_IN_PROGRESS - AWS::CloudFormation::Stack - demo-email-form-dev
CloudFormation - UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS - AWS::CloudFormation::Stack - demo-email-form-dev
CloudFormation - DELETE_IN_PROGRESS - AWS::Logs::LogGroup - FormSubmitLogGroup
CloudFormation - DELETE_IN_PROGRESS - AWS::ApiGateway::RestApi - ApiGatewayRestApi
CloudFormation - DELETE_IN_PROGRESS - AWS::IAM::Role - IamRoleLambdaExecution
CloudFormation - DELETE_IN_PROGRESS - AWS::IAM::Role - IamRoleCustomResourcesLambdaExecution
CloudFormation - DELETE_COMPLETE - AWS::Logs::LogGroup - ApiGatewayLogGroup
CloudFormation - DELETE_COMPLETE - AWS::Logs::LogGroup - FormSubmitLogGroup
CloudFormation - DELETE_COMPLETE - AWS::ApiGateway::RestApi - ApiGatewayRestApi
CloudFormation - DELETE_COMPLETE - AWS::IAM::Role - IamRoleCustomResourcesLambdaExecution
CloudFormation - DELETE_COMPLETE - AWS::IAM::Role - IamRoleLambdaExecution
CloudFormation - UPDATE_ROLLBACK_COMPLETE - AWS::CloudFormation::Stack - demo-email-form-dev
Serverless: Operation failed!
Serverless: View the full error output: https://us-east-1.console.aws.amazon.com/cloudformation/home?region=us-east-1#/stack/detail?stackId=arn%3Aaws%3Acloudformation%3Aus-east-1%3A757370802528%3Astack%2Fdemo-email-form-dev%2F0d782920-fb68-11e9-889e-0a743b4c9590
Serverless: Publishing service to the Serverless Dashboard...
Serverless: Successfully published your service to the Serverless Dashboard: https://dashboard.serverless.com/tenants/fernando/applications/enterprise/services/demo-email-form/stage/dev/region/us-east-1
 
  Serverless Error ---------------------------------------
 
  An error occurred: ApiGatewayLogGroup - /aws/api-gateway/demo-email-form-dev already exists.
 
  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com
 
  Your Environment Information ---------------------------
     Operating System:          darwin
     Node Version:              6.10.3
     Framework Version:         1.55.1
     Plugin Version:            3.2.1
     SDK Version:               2.1.2

Similar or dependent issues:

• #6091

[medikoo]

This bug can happen when following deploy flow happens:

1. Service with API Gateway logging on (provider.logs.restApi: true) is deployed
2. Same service is redeployed with provider.logs.restApi removed. This removes the log group, but logging is not turned off (it’s due to limitations on how this setting is handled in Framework, framework has no state history, and doesn’t know that for previous deployment it was true) In result after following Rest API requests (to be logged) log group is automatically recreated by AWS
3. Same service is again deployed with logging on (provider.logs.restApi: true). This crashes as there’s an attempt to create log group which apparently exists.

Workaround is to at step 2, redeploy service with logs explicitly turned off (via provider.logs.restApi: false), that should not produce log writes between deployments.

This will be effectively fixed once current handling of API Gateway stage settings is moved to custom resources (or CloudFormation when modifications to it will be supported). I have that on my todo list, but due to other priorities release window is uncertain.

[AdamGreaves-DevSecOps]

Seems a way around this is by changing your service name to something new to work around the error of the existing log.