[proposal] Add a new cfn/SAM based deployment backend

This is a proposal to add a new deployment backend that’s based on cloudformation and the AWS Serverless Application Model (AWS SAM). This addresses several points of customer feedback and makes several feature requests easier to implement.

Motivation

When chalice deploys your application, it uses boto3, the AWS SDK for Python, to make the necessary coordinated API calls to IAM, Lambda, and API Gateway. There’s several benefits with this approach. Deployments are fast as we’re making direct API calls to API gateway and Lambda. Nothing to setup (including S3 buckets because we send the zip file contents directly to lambda).

However, there’s several customer issues that I’d like to address:

• Option to not deploy to dev, but only to update the resources in IAM, lambda, and API Gateway
• Separation of chalice app and chalice deploy concerns
• Chalice delete
• Support cloud formation templates
• List of resources created

I think that most of these issues can be addressed by leveraging the serverless application model along with cloudformation stacks. Consider if the whole deployment was broken down into into two explicit and separable components:

1. Generate assets that describe what to deploy
2. Take those locally generated assets and deploy them to AWS.

This gives users insight into what we’re about to deploy, allows them to customize what’s going to be deployed (by manipulating the manifest), allows them to see a list of resources that were created, and also makes deletion trivial.

Specification

Customers will now have the option of creating a chalice application with the new cfn deployment backend (the defaults will remain unchanged). This choice must be made at project creation time. It will not be possible to migrate from the current deployment backend to the cfn deployment backend.

The eventual goal would be to make this the default (and maybe only) deployment backend.

A new parameter is added to the new-project command that indicates you want to use Serverless Application Model templates: chalice new-project --use-sam-templates --s3-path. Note you’ll also have to specify an S3 location where the deployment package can be uploaded (this is needed when using SAM/cfn).

Once you’ve created a new application with SAM templates enabled, the chalice deploy command will internally use SAM templates and cloud formation to deploy your application.

There will also be a new chalice package command, which can only be used for projects created with --use-sam-templates. This command creates local assets for your chalice app that can then be deployed at a later time (or even by other tools that can work with SAM templates). This package command takes an output directory and writes the following files to that directory:

• app.json - The generated SAM template
• deployment.zip - The deployment zip file to send to lambda

In order to create a lambda function with cfn, we’ll need to upload your deployment package to S3 and update the app.json with a path to that remote path. By default, the chalice package command does not make any remote API calls, but you can provide the --remote-assets flag to indicate you’d like chalice to handle uploading your lambda deployment package to S3 for you. It will also take care of updating the SAM template with a reference to that S3 location when --remote-assets is specified.

Example

Here’s an example of using this new functionality to create a deployment package that can be deployed with the AWS CLI using the newly introduced aws cloudformation package/deploy commands:

$ chalice new-project —use-sam-templates cfn-based-app --s3-path s3://bucket/

# This will generate a SAM template, and upload your deployment package to S3
$ chalice package --remote-assets /tmp/deploy

# You can now deploy the app using your preferred cfn tools
# Here we're using the AWS CLI to deploy our app:
$ aws cloudformation deploy 
        --template-file /tmp/deploy/app.json \
        --stack-name cfn-based-app --capabilities CAPABILITIES_IAM

# We can grab the endpoint url from the stack outputs:
$ aws cloudformation describe-stacks \
        --stack-name cfn-based-app \
        --query "Stacks[0].Outputs[?OutputKey=='EndpointUrl'] | [0].OutputValue" \
        --output text
https://api-id.execute-api.us-west-2.amazonaws.com/dev/

Future enhancements

One other thing I’d like to add in the future is the ability for users to modify the existing SAM template, possibly with their own application specific resources. A really simple example would be to allow them to specify additional cfn/SAM resources that can merged together with the template we generate:

# Project root directory (where app.py is located):
$ cat app.json
{
  "Resources" : {
    "myTable" : {
      "Type" : "AWS::DynamoDB::Table",
      "Properties" : {
        "AttributeDefinitions" : [...],
        "TableName": "mytable"
        ...
      }
    }
  }
}

[jamesls]

Following up here, this is now available in the 0.8.0 release of chalice, via the new chalice package command: chalice package /tmp/packaged-dir. I’ll keep this open until I finish writing docs for this, but you it’s available if you want to try it out.

[igable]

I wanted to ping this issue and see if this is still the plan. I’m looking for a way to use Chalice when promoting code up a CD pipeline.