Report all experiment validation errors at once

Currently, when a chaos validate command is run, it shows the first error and exits. This is a process to repeat when fixing each error, when using an experiment that contains multiple syntax issues.

It could be nice to do a full validation of the file and outputs all errors at once. It can either be default behavior or behavior to be activated with a CLI flag (--full-validation, --global, …)

The following experiment contains multiple errors:

{
    "version": "1.0.0",
    "title": "Invalid syntax experiment",
    "description": "This experiment contains multiple synatx errors to be raised at validation step",
    "tags": [
        "invalid"
    ],    
    "steady-state-hypothesis": {
        
    },
    "method": [
        {"type": "invalid"}
    ],
    "rollbacks": [{"type":"invalid"}]
}

[Lawouach]

Hi all,

Good feature request. I don’t think we need a new flag, this should become the default behavior and not impact compatibility.

However, there is a lot of workto change the code internally I’m afraid. This will take some work.

[nstjelja]

@dmartin35 i think i would suggest going through option 1. The only risk/issue/ problem i see is that the original function was written with the flow being interrupted by the exception being raised so it would require us to write code in amore defensive manner.