Cleanly separate Packaging and Deployment phases

Use case description

Derived from https://github.com/serverless/serverless/issues/8364 which was narrowed down to other processing flow improvements refactor

Currently internal flow of processing commands is convoluted and confusing in many places. In result there’s a dependency between parts for which there is no reason to depend on each other (e.g. we cannot do sls deploy --package outside of service context, while after all outcome doesn’t depend in any way on the current working directory context).

Additionally we should cleanly separate Packing and Deployment phase, taking into account case reported at https://github.com/serverless/serverless/issues/3528 where’s there’s reported need for builds that can be reused for any stage or region.

Ideally internal handling should be as follows

1. Packaging step, should package binaries and store resolved state of service configuration, but we should not generate CloudFormation template to be deployed at that stage.
2. Deployment should consist of:
   1. Generation of ready to deploy CloudFormation template
   2. Upload artifacts
   3. Deployment of CloudFormation stack

In this context it’ll be good to also address https://github.com/serverless/serverless/issues/2202 (provide a mean to run sls remove --name <service-name> outside of service context

Proposed solution

TBA

[nponeccop]

I understand that currently variable resolution step is performed during sls package, so essentially Serverless generates a dynamic CloudFormation file based on the SSM data during the packaging phase.

My idea is to change the process so that packaging and name resolution are separated. So as long as my project build artefacts don’t depend on data stored at SSM, there is no need to connect. The variable resolution, CloudFormation generation etc are only performed during the deployment step.

A more important change is to make the plugin architecture aware of this phase separation, so that plugins authors plan accordingly what is permitted at what stage. Some plugins may choose to work offline, some during deployment, and some both.

[medikoo]

Thanks @ankon for the input. That’s a very good point. I believe we should keep ability to generate the final CF template without deploying, and/or allowing to hook in a process after the template is generated but before deployment is initialized