Handle request replays by Jetty due to timeout of long-running async request

I’ve encountered an issue with async execution of long-running jobs.

If a back-end service takes longer than 30 seconds to process an asynchronous request, Jetty will replay the request, while setting response code to 500. Javalin will also replay the request as it would a normal request, except that body is this time empty. I wrote a short demo that shows how Javalin handles it. See log output below for details on what goes on. This SO question also mentions it.

(Btw, 30 second timeout is a property defined in org.eclipse.jetty.HttpChannelState.DEFAULT_TIMER and can be set via. system property which is pretty awkward. I haven’t found another way to set it.)

Here’s example code that triggers the timeout (using io.restassured to issue a request):

long asyncTimeout = Long.getLong("org.eclipse.jetty.server.HttpChannelState.DEFAULT_TIMEOUT", 30000);
CompletableFuture<Object> future = new CompletableFuture<>();

Javalin j = Javalin.create();

j.before(ctx -> {
  // cannot access async context here yet
  System.out.println(Instant.now() + " BEFORE disp=" +
    ctx.req.getDispatcherType() + " " + ctx.req.getRequestURI() +
    " " + " body='" + ctx.body() + "' " + ctx.status());
});
j.after(ctx -> {
  System.out.println(Instant.now() + " " + "AFTER uri='" +
    ctx.req.getRequestURI() + "' " + " body='" + ctx.body() + "' " +
    ctx.status() + " result='" + ctx.resultString() + "'");
});

j.get("/async", ctx -> ctx.result(future));

j.start();

Executors.newSingleThreadScheduledExecutor().schedule(() -> future.complete("result"), asyncTimeout + 5000, TimeUnit.MILLISECONDS);

given().port(j.port()).body("{\"param\":\"data\"}").get("/async").then().log().all();

and here is the output, with wire data logged in restassured:

2590  [main] DEBUG wire       -  >> "GET /async HTTP/1.1[\r][\n]" 
2591  [main] DEBUG wire       -  >> "Accept: */*[\r][\n]" 
2591  [main] DEBUG wire       -  >> "Content-Type: text/plain; charset=ISO-8859-1[\r][\n]" 
2591  [main] DEBUG wire       -  >> "Content-Length: 16[\r][\n]" 
2591  [main] DEBUG wire       -  >> "Host: localhost:7000[\r][\n]" 
2591  [main] DEBUG wire       -  >> "Connection: Keep-Alive[\r][\n]" 
2591  [main] DEBUG wire       -  >> "User-Agent: Apache-HttpClient/4.5.3 (Java/1.8.0_112)[\r][\n]" 
2591  [main] DEBUG wire       -  >> "Accept-Encoding: gzip,deflate[\r][\n]" 
2591  [main] DEBUG wire       -  >> "[\r][\n]" 
2591  [main] DEBUG wire       -  >> "{"param":"data"}" 
2018-12-04T20:08:35.365Z BEFORE disp=REQUEST /async  body='{"param":"data"}' 200
2018-12-04T20:09:05.374Z BEFORE disp=ERROR /async  body='' 500
2018-12-04T20:09:08.967Z AFTER uri='/async'  body='' 500 result='result'
36265 [main] DEBUG wire       -  << "HTTP/1.1 500 Server Error[\r][\n]" 
36267 [main] DEBUG wire       -  << "Server: Javalin[\r][\n]" 
36267 [main] DEBUG wire       -  << "Content-Type: text/plain[\r][\n]" 
36267 [main] DEBUG wire       -  << "Content-Length: 6[\r][\n]" 
36267 [main] DEBUG wire       -  << "[\r][\n]" 
2018-12-04T20:09:08.977Z AFTER uri='null'  body='{"param":"data"}' 200 result='result'
36351 [main] DEBUG wire       -  << "result" 
HTTP/1.1 500 Server Error
Server: Javalin
Content-Type: text/plain
Content-Length: 6

result

I haven’t found an explicit way to handle such timeout within Javalin. I don’t really have an opinion on how to improve it due to lack of experience with web servers (highlighted by the fact that I ran into this issue at all by designing a ‘REST’ service that regularly takes 30+ seconds to respond). Sorry 😃

My workaround was to tack a continuation to this task that sets status code to 200. Then I save the task into session attribute on first request and use it on followup requests to re-set Javalin’s ctx.result(CompletableFuture<?>). That messes up request timing, but otherwise seems to work. When the future finishes, I see after handlers being triggered for each request, but only the last one returns to client. I also had to put some code into AccessManager to skip re-authorizing these repeated requests.

[tipsy]

Added a small comment here @pikob: https://javalin.io/documentation#async-timeout-settings

[tipsy]

@pikob added an option to configure it per request in #465