[RFC] Scaffolder: Unify templating syntax used in template.yaml and the template skeleton

Status: Open for comments

Need

In a standard Backstage Software Template (Example), there is a template.yaml file which is an Entity descriptor file of the Template kind and a directory (e.g. skeleton) which contains all the templated files and folders.

Up until now, we only had one major templating action available called fetch:cookiecutter. This resulted in our skeleton files and folders to be templated with Jinja2 syntax as this is what cookiecutter expects. The templating syntax of Jinja2 looks like {{ variable | filter }} and {% statement %}.

On top of the skeleton, the template.yaml files themselves need to be templated in places like setting values as inputs for the templating action. This confusingly looks like a Jinja2 Syntax '{{ variable }}' (it isn’t exactly) and is managed using a library called handlebars.

With this context, we reflect on some of the problems that exist as of now -

• When writing a new Software Template, users need to understand two templating syntaxes, one for template.yaml (handlebars) and one for the skeleton files and folders (Jinja2).
• Using handlebars forces all variables to be wrapped in quotes, since the expression syntax used in handlebars conflicts with YAML object literal syntax.

Goals/Proposal

1. Unify templating syntax of template.yaml and the fetch:template action

A new fetch:template action was introduced in https://github.com/backstage/backstage/pull/6322, which is a node-based templating action and doesn’t need Python/Cookiecutter dependencies. This is going to be the “default” or a recommended templating action. With this in mind, we propose updating the syntax used for variables in template.yaml to match the syntax used in fetch:template, in which expressions are enclosed in ${{ }}.

2. Make it more intuitive to insert non-string variables in template.yaml

Right now it’s possible to insert non-string variables in template.yaml, but the syntax is counter-intuitive - to insert the array variable myArray as an array, the parameter needs to be set to '{{ json parameters.myArray }}'. It would be preferable to keep the syntax identical to that used inside skeletons, such that the same variable could be inserted with simply ${{ parameters.myArray }}.

The GitHub Actions workflow syntax is a good inspiration for both of these goals.

Alternatives

Open for suggestions!

Risks

To avoid this being a breaking change, we should create a new apiVersion for the template.yaml Entity description; most probably backstage.io/v1beta3

[Rugvip]

I think double curlys, both ${{ }} and {{ }} makes a lot of sense in the template contents, as it avoids collisions with for example usage of ${ } in shell scripts etc. I’d say it’s less important for us to use that in the template definition though, as we’re not mixing it with shell scripts. GitHub Actions do mix with shell scripts, which is why they need the full ${{ }} syntax.

Can distill that to two simple rules for templating with curly brackets for our use-cases:

• If the template is defined in YAML, prefix with $ to make it be treated as a string
• If the template is commonly mixed with syntax that conflicts with the single-bracket variant, use double brackets

So we could either go for ${ } or ${{ }} in our template definitions (not that we’ve been discussing anything else 😅), and so far I can think of the following reasons for either

Reasons to go with ${{ }}:

• It matches the current syntax of our template actions, making it more familiar
• It’s more forwards-compatible with additions to the template definitions that could conflict with the single-bracket version

Reasons to go with ${ }:

• It more clearly distinguishes the template definitions from the template contents, in the case where those are different.
• It is slightly less noisy to read and write

From my point of view the clarification of the relation between the template definition and contents is the most important reason to go with either syntax. I’m thinking that if we can use the exact same syntax and helpers in the template definitions as in the template action, then we should go with ${{ }}, otherwise, we should go with ${ } to avoid confusion between the two templaters. I’d also say that the ease of use of the template definition syntax is a higher priority than using the same templating syntax for the two use-cases, and that support for passing through complex values is very important.

[benjdlambert]

I’d say that the substitution in app-config.yaml is only for Environment Variables so and I think it aligns nicely with bash syntax of ${ENV}.

This I think is still open for discussion if to be honest, but I like ${{ }} as it’s pretty clear when glancing through a document what is a template var coming from using cookiecutter and handlebars before. It’s also pretty widely used in other template languages too, so i’m thinking the familiarity is good.

Thoughts?