Interchangeability of Tokens and Literals

This is definitely a design challenge we haven’t solved yet, and one of the only remaining abstraction leaks we need to resolve.

At the “L1” CloudFormation resource layer (the XxxResource constructs), we are currently using TypeScript unions to allow specifying either concrete values or tokens for every primitive list property.

This works for L1, however, it’s sub-optimal for a few reasons:

1. In some languages, which do not support unions, we get a sub-par experience. For example, in .NET, all L1 props are Object (which we might want to change to “setXxx” methods just for the type-safety). In Java, we have overloads for setters, and getters return Object. It’s not elegant or nice, or idiomatic, and we don’t want to leak this ugliness to L2…
2. Theoretically, it requires that every property in every construct we or anyone else will ever write will use unions to support these fringe use cases. Furthermore, when you interact with a prop that uses a union type, it means you will need to be type-sensitive every time you do anything with the value. This will “litter” any construct code with union types and type-checks everywhere (which are very hard in JavaScript). That’s not an acceptable developer experience (remember, we are building a framework, not just a library).

Let’s try to classify the use cases where people need to interact with tokens:

Parametric CloudFormation templates

When using the CDK “natively”, this is rarely needed since the CDK is designed to synthesize concrete CloudFormation templates for every environment the app is deployed. There are better tools in programming languages for defining abstractions, such as functions and classes. This approach also promotes deterministic deployments and healthier change governance (since configuration is checked-into code).

However, we are aware that parametric CloudFormation templates are something that people need, even as they transition to a more CDK-native code base.

Work with resource attribute values

This is actually a more long-term concern as it’s not going to go away when people go “full CDK”.

There will always be a use case for using a resource attribute value that only resolves at deploy-time as an input for another value. For example, say I want to include the generated name of the bucket in a string. Ideally, I want to write it like this:

const message = `Your file has been uploaded to the bucket ${myBucket.bucketName}`;

This will currently not work. Since bucketName is a Token (actually it’s a BucketName which extends Token, but those are just aliases), you cannot use them in string substitutions. Alternatively, you will need to do something like this:

const message = Fn.concat("Your file has been uploaded to the bucket", myBucket.bucketName);

This is not that bad, but it’s an abstraction leak. It’s not the natural thing a user will do. Moreover, in some cases, bucketName can be resolved to a concrete value during synthesis (e.g. when we explicitly specify the bucket name).

Proposed solution

As of this writing, the CloudFormation spec had 171 resource attributes, out of 157 are strings (~92%) (see analysis).

Let’s try to optimize for the common case, where tokens resolve to strings.

We can represent tokens as string variable expansions. Then, when we synthesize a template, if a property value includes the string "${id}", it will be substituted with { "Ref": "id" }, and "${id.attr}" will be substituted with { "Fn::GetAt": [ "id", "attr" ] }. The substitution can use Fn::Join in case there are string before and/or after the expansion (i.e. "hello ${foo}" => { "Fn::Join": [ "", [ "hello", { "Ref": "foo" } ] ] }.

To apply this safely (without breaking the cases where attributes/parameters are not strings), what we can do is:

1. Represent all string resource attribute as string that return ${id.attr} instead of an opaque token. Can we do that in a way that doesn’t lose the semi-strong-typing we have today for attributes? (we currently generate an attribute type class for each attribute that extends Token). Maybe we can just override token.toString for those types and have token.toString for non-string types throw an exception?
2. All resource.refs can probably just return a string ${id}
3. We can add a parameter.stringValue property which will return a string ${param-id}. We can also create a subclass StringParameter whose .value is a string.

As @rix0rrr suggested let’s also use this as a tracking issue to see how often the issues of people wanting to pass literals where Tokens are declared, or vice versa (passing CFN intrinsics where the declared type is a literal string) crop up.

[rix0rrr]

Yes, split-and-select. Although I would expect the use of that to be rare as well. Join maybe. But even those seem more designed to work at a lower level than we expect people to work at, to get information in and out of CloudFormation parameter strings.

Though if we made Token.stringValue(), we can just have it generate an unique identifier and register itself in a global map which we can find back during resolve(), and any intrinsic could be supported that way.

---

If we’re going the “all attributes are strings” way, there’s an idiom I expect people are going to write and then be confused:

const bucket = new Bucket(...);

if (bucket.bucketName == "my-bucket-1234QZY") {
  // Why is this never true? I see the bucket name like that in the console!
}

So better find good terminology to cut this off at the pass. We can’t allow bucket.bucketName to be a string itself. So it should be something like bucket.bucketName.asProperty() ?

[rix0rrr]

Superseded by #695. Closing.