Give the reason for failure on `mustFail` nodes in scenarios

For failing submit statements in scenarios, I get interesting error messages:

template Token
  with
    owner : Party
  where
    signatory owner

failing_test_1 = do
  alice <- getParty "Alice"
  bob <- getParty "Bob"
  submit alice do
    create Token with owner = bob

gives

Scenario execution failed on commit at Intro_2_Scenario:69:3:
  #0: create of Intro_2_Scenario:Token at DA.Internal.Prelude:377:26
      failed due to a missing authorization from 'Bob'

Last source location: DA.Internal.Prelude:377:26

Ledger time: 1970-01-01T00:00:00Z

Partial transaction:
  Sub-transactions:
     #0
     └─> create Intro_2_Scenario:Token
         with
           owner = 'Bob'

However, with a submitMustFail, the reason for failure is lost:

failing_test_2 = do
  alice <- getParty "Alice"
  bob <- getParty "Bob"
  submitMustFail alice do
    create Token with owner = bob

gives

Transactions: 
  TX #0 1970-01-01T00:00:00Z
     mustFailAt 'Alice' (Intro_2_Scenario:69:3)

Active contracts: 

Return value: {}

I would like the mustFailAt node to contain the reason for failure and, if possible, the partial sub-transaction up to the failure. According to @francesco-da , submitMustFail should not return any of this to the Scenario context, though so its purely for visual inspection.

[mjstewart]

@hurryabit @remyhaemmerle-da. I have applied your feedback and the changes can be seen in https://github.com/digital-asset/daml/compare/a212966..cb64eff#diff-e259efac0238c8c88ae4633872aef024R39

Now these feature flags are in, I get the following error in daml studio when I point to local 0.0.0 daml-sdk-head. Is this what should happen now?

ScenarioBackendException {scenarioNote = "Failed to create scenario context", ... "Scenario.mustFailAtMsg" is not supported by DAML-LF 1.Stable(7)".

2 other outstanding questions

1. The recommended change for this section https://github.com/mjstewart/daml/commit/a2129665a2ad9176c589f3531722064e3d31fe2b#r37208283

2. Are adding docs with examples a welcome inclusion to the PR? I assume submitMustFailMsg wont show up in the search bar unless the feature flag is enabled?

[mjstewart]

Hey @gerolf-da, @hurryabit

Before a potential PR, I’d like to discuss if my implementation looks feasible before continuing. https://github.com/mjstewart/daml/commit/a2129665a2ad9176c589f3531722064e3d31fe2b

1. I have modified DAML-LF to introduce.

submitMustFailMsg : Party -> Update a -> Scenario Text

It will return the text message of a user initiated error via abort or assert captured in a DamlEUserError.

1. Added assertPrefix to DA.Assert

Its common for error messages to begin with an actual message followed by a dump of the record payload which isn’t really needed for the purpose of identifying the test failure reason.

Combined with assertEq, I imagine these 2 assertions will handle 99% of use cases. Any other assertion can be manually implemented since this new feature extracts the error message.

usage examples

I find the reverse bind syntax nice to represent this compactly by feeding in the actual error message into the assertion.

  scenario do
    ...

    assertEq "age must be over 18" =<<
      submitMustFailMsg owner do
        exercise _helper TestHelper_Run with age = 18

Alternatively, the long hand form looks like this for flexibility.

scenario do
  ...

  actual <- submitMustFailMsg owner do
    exercise _helper TestHelper_Run with age = 18

  assertMsg "must end in over 18" $ Text.isSuffixOf "over 18" actual

How to add tests?

Id like to add a couple of tests in compiler/damlc/tests/daml-test-files. Is this the best place to add them? if so how can I get these tests to pickup the new keyword and assertPrefix?

Variable not in scope:
    submitMustFailMsg
  Perhaps you meant ‘submitMustFail’ (imported from Prelude)

Thanks for your feedback