Clarifying typechecking of optional values

Spec gray area:

If I have an optional declaration T? x, and an expression refers to x in some context where (non-optional) T is expected, when should this be statically rejected as a type mismatch error, vs. considered valid albeit liable to fail at runtime?

The spec right now says x “can only be used in calls or functions that accept optional values” but this isn’t quite clear to the above.

My “first principles” instinct would be that T? x should statically fail to unify with (non-optional) T absent some explicit coercion, but:

• It’s expressly valid and desirable for interpolations in task commands, as described in “Prepending a String to an Optional Parameter.” Also relevant #227
• If one writes another declaration T y = x what should happen when?
• I infer from some examples in the wild that Cromwell accepts the pattern like T y = if defined(x) then EXPR_WITH_x else SOME_DEFAULT (ex1, ex2).
  • In those examples, EXPR_WITH_x is x * 1000 and size(x) respectively.
  • Do we check the optional quantifier statically at all?
  • Alternatively, one could imagine special-case typechecking logic for the if defined(x) ... case.
• If one specifies x to a required call input of type T what should happen when? (ex)

Appreciate if someone from Cromwell team could clarify how it approaches this (and of course comment on what would be ideal for the spec). Thanks!

[cjllanwarne]

This does indeed come up in discussions a lot!

I’ve often found that what seemed like “special cases to make things easier” has instead lead to confusion and a harder to understand language. And there are a lot of special cases around optionals (as you mentioned) that I regularly find more confusing than helpful.

So my gut feeling is to be stricter on enforcing “you can’t use optionals without unpacking them first” as long as we allow people to unpack things easily if they need to. Unfortunately the “current” way of unpacking a Int? x into Int is rather ugly: select_first([x]).

I’d be open to either a structural or expression based way of unpacking. I think @patmagee 's suggestion against my structural https://github.com/openwdl/wdl/issues/187 was very good and far less intrusive - he suggested that we could have a simple select(x) or get(x) function to turn declarations likeInt? x into Int or else throw an error. I think this would free us up to be a lot more aggressive with type-checking of optionals.

Re “task commands”, I think a recent PR (still awaiting implementation) was involved in tidying up those command sections to only allow simple expressions rather than their own entire complex syntax. I wonder if that addressed optionals.

Re the final example about optional inputs - I think that this is now tidied up in WDL 1.0. In WDL 1.0 the example would be able to indicate “the input has a default so might not be supplied, but the type of the value is not optional” like this:

workflow calulateDNAContamination {
  input {
    String docker_image = "..."
  }

[mlin]

I have written a straw-man spec diff for debate & discussion: https://github.com/openwdl/wdl/pull/290