Interchangeability of Tokens and Literals
See original GitHub issueThis is definitely a design challenge we haven’t solved yet, and one of the only remaining abstraction leaks we need to resolve.
At the “L1” CloudFormation resource layer (the XxxResource
constructs), we are currently using TypeScript unions to allow specifying either concrete values or tokens for every primitive list property.
This works for L1, however, it’s sub-optimal for a few reasons:
- In some languages, which do not support unions, we get a sub-par experience. For example, in .NET, all L1 props are Object (which we might want to change to “setXxx” methods just for the type-safety). In Java, we have overloads for setters, and getters return Object. It’s not elegant or nice, or idiomatic, and we don’t want to leak this ugliness to L2…
- Theoretically, it requires that every property in every construct we or anyone else will ever write will use unions to support these fringe use cases. Furthermore, when you interact with a prop that uses a union type, it means you will need to be type-sensitive every time you do anything with the value. This will “litter” any construct code with union types and type-checks everywhere (which are very hard in JavaScript). That’s not an acceptable developer experience (remember, we are building a framework, not just a library).
Let’s try to classify the use cases where people need to interact with tokens:
Parametric CloudFormation templates
When using the CDK “natively”, this is rarely needed since the CDK is designed to synthesize concrete CloudFormation templates for every environment the app is deployed. There are better tools in programming languages for defining abstractions, such as functions and classes. This approach also promotes deterministic deployments and healthier change governance (since configuration is checked-into code).
However, we are aware that parametric CloudFormation templates are something that people need, even as they transition to a more CDK-native code base.
Work with resource attribute values
This is actually a more long-term concern as it’s not going to go away when people go “full CDK”.
There will always be a use case for using a resource attribute value that only resolves at deploy-time as an input for another value. For example, say I want to include the generated name of the bucket in a string. Ideally, I want to write it like this:
const message = `Your file has been uploaded to the bucket ${myBucket.bucketName}`;
This will currently not work. Since bucketName
is a Token
(actually it’s a BucketName
which extends Token
, but those are just aliases), you cannot use them in string substitutions. Alternatively, you will need to do something like this:
const message = Fn.concat("Your file has been uploaded to the bucket", myBucket.bucketName);
This is not that bad, but it’s an abstraction leak. It’s not the natural thing a user will do. Moreover, in some cases, bucketName
can be resolved to a concrete value during synthesis (e.g. when we explicitly specify the bucket name).
Proposed solution
As of this writing, the CloudFormation spec had 171 resource attributes, out of 157 are strings (~92%) (see analysis).
Let’s try to optimize for the common case, where tokens resolve to strings.
We can represent tokens as string variable expansions. Then, when we synthesize a template, if a property value includes the string "${id}"
, it will be substituted with { "Ref": "id" }
, and "${id.attr}"
will be substituted with { "Fn::GetAt": [ "id", "attr" ] }
. The substitution can use Fn::Join
in case there are string before and/or after the expansion (i.e. "hello ${foo}"
=> { "Fn::Join": [ "", [ "hello", { "Ref": "foo" } ] ] }
.
To apply this safely (without breaking the cases where attributes/parameters are not strings), what we can do is:
- Represent all string resource attribute as
string
that return${id.attr}
instead of an opaque token. Can we do that in a way that doesn’t lose the semi-strong-typing we have today for attributes? (we currently generate an attribute type class for each attribute that extendsToken
). Maybe we can just overridetoken.toString
for those types and havetoken.toString
for non-string types throw an exception? - All
resource.ref
s can probably just return a string${id}
- We can add a
parameter.stringValue
property which will return a string${param-id}
. We can also create a subclassStringParameter
whose.value
is a string.
As @rix0rrr suggested let’s also use this as a tracking issue to see how often the issues of people wanting to pass literals where Tokens
are declared, or vice versa (passing CFN intrinsics where the declared type is a literal string
) crop up.
Issue Analytics
- State:
- Created 5 years ago
- Comments:17 (17 by maintainers)
Yes, split-and-select. Although I would expect the use of that to be rare as well. Join maybe. But even those seem more designed to work at a lower level than we expect people to work at, to get information in and out of CloudFormation parameter strings.
Though if we made
Token.stringValue()
, we can just have it generate an unique identifier and register itself in a global map which we can find back duringresolve()
, and any intrinsic could be supported that way.If we’re going the “all attributes are strings” way, there’s an idiom I expect people are going to write and then be confused:
So better find good terminology to cut this off at the pass. We can’t allow
bucket.bucketName
to be a string itself. So it should be something likebucket.bucketName.asProperty()
?Superseded by #695. Closing.