Allow to create reusable cross-stage deployment ready build artifacts

With the new package/deploy semantics, the builds and deployments can be distributed on different servers. The created artifacts can then be deployed to AWS (using the CF template created in the build step).

Normally you construct your services, so that some properties are set by variables defined in the serverless.yml that are set depending on the service target (e.g. stage or region). A prominent example for that is the setting of Lambda environment variables.

Imagine you have the following service definition:

service: stashimi-bot-dash

provider:
  name: aws
  runtime: nodejs6.10
  stage: dev
  region: us-east-1
  ...
  
  environment:
    MY_VAR: ${self:custom.${self:custom.stage}.myVar}
    ...

custom:
  stage: ${opt:stage, self:provider.stage}
  ...
  dev:
    myVar: "My-3rdParty-Auth-Id-For-Dev"
  prod:
    myVar: "My-3rdParty-Auth-Id-For-Prod"

The problem with the package command is now, that the variables are already substituted there and the CF template that is transferred to the deploy stage already contains the resolved variables. As a consequence it is not possible to deploy a built/packaged service to different stages, i.e. the stage used to build also has to be used to deploy.

A workaround is to build all stages/regions independently on the build server, create artifacts for each stage and deploy the corresponding artifact on the deployment server.

A proper solution would be that the build phase creates an artifact, that is independent from the environment that the artifact is deployed to.

Proposal

The build phase should keep the literal variable references in the artifact’s CF template and the deploy phase should do the variable substitution depending on the stage/region selected at the time of deployment. This would mean, that only the variable subsitution functionality has to be moved. Of course only variables that are used/placed in the generated CF template should be deferred. So maybe the generateXXXTemplate() function that is run in the build phase would be the target for a viable implementation - then only the right variables would be affected. On the deploy phase the substitution has to be made as soon as the CF template from the artifact is loaded, to have the real values available for all plugins running in the deployment context.

Care has to be taken for plugins that might use the variable information during the build phase (imo that’s wrong anyway).

[bishwash-devops]

This seems like a major issue, a package should be immutable and reusable. Where does this feature belong in the priority?

[erstaples]

Hi @medikoo - is the roadmap published anywhere? I’m curious to see what’s prioritized ahead of this. I don’t understand how a design flaw of this magnitude could go unaddressed for such a long time despite a lot of noise from the community to fix it (e.g. https://github.com/serverless/serverless/issues/4715).