When using REST Assured, investigate if it's possible to defer documentation generation until after other conditions have been evaluated

Feedback from @jlstrater:

When using the rest assured version, the documentation tests fail before other conditions like the status code. I’m not sure if this is by design but it was slightly confusing

This is due to the use of a Filter and the fact that it’s called before the assertions in the then phase. I’ve just looked at the possibility of the Filter returning a Response wrapper that can then hook into then() being called, but I don’t think that would be enough to allow the documentation to be generated after all of the other conditions had been evaluated.

Another possibility (that would also read better) would be something like then().do(document(…)). This would also be a closer match for the approach with Spring MVC Test. Unfortunately, there’s no such API in REST Assured that I’m aware of and then deals purely with responses whereas REST Docs needs access to both the request and the response.

@johanhaleby If you have a moment, I’d welcome your thoughts on this one please. The current approach looks like this:

given(this.documentationSpec)
        .accept("text/plain")
        .filter(document("sample", 
                preprocessRequest(modifyUris()
                        .scheme("https")
                        .host("api.example.com")
                        .removePort())))
.when()
        .port(this.port)
        .get("/")
.then()
        .assertThat().statusCode(is(200));

The Filter that’s returned from document can throw an exception if the documentation doesn’t match the service. Ideally, the failure would occur after checking that the response’s status code was 200.

[johanhaleby]

I’ll keep this in the back of my head to see if something pops up in the future.

[wilkinsona]

Thanks for the clarification. But if linksWithRel throws exception on every failure how can you present all failures at once when using MockMvc?

It throws an exception describing all of the problems with the documentation of the links. The key thing is that this exception is thrown before any other expectations are checked. That’s the opposite of what happens with MockMvc and is what left @jlstrater slightly confused.

But it will require a lot of time to implement this (I think) and currently I cannot see any other use cases for it (which make me a bit reluctant to add support for this).

Understood. I thought I might be asking a lot.

Perhaps the conclusion is that we’ll have to live with this for now?

Yes, I think so. Thanks for taking the time to help us to explore a few possibilities.