Curl hangs when Header "Expect:100-continue" is set on POST > 1024 bytes

I’m using Spring Cloud Gateway Finchley.M9.

I noticed that when I proxy a large POST the gateway times out:

$ curl -X POST -H "Content-Type:application/json" -d '[{ "id": "0001", "type": "donut", "name": "Cake", "ppu": 0.55, "batters": { "batter": [{ "id": "1001", "type": "Regular" }, { "id": "1002", "type": "Chocolate" }, { "id": "1003", "type": "Blueberry" }, { "id": "1004", "type": "Devils Food" } ] }, "topping": [{ "id": "5001", "type": "None" }, { "id": "5002", "type": "Glazed" }, { "id": "5005", "type": "Sugar" }, { "id": "5007", "type": "Powdered Sugar" }, { "id": "5006", "type": "Chocolate with Sprinkles" }, { "id": "5003", "type": "Chocolate" }, { "id": "5004", "type": "Maple" } ] }, { "id": "0002", "type": "donut", "name": "Raised", "ppu": 0.55, "batters": { "batter": [{ "id": "1001", "type": "Regular" }] }, "topping": [{ "id": "5001", "type": "None" }, { "id": "5002", "type": "Glazed" }, { "id": "5005", "type": "Sugar" }, { "id": "5003", "type": "Chocolate" }, { "id": "5004", "type": "Maple" } ] }, { "id": "0003", "type": "donut", "name": "Old Fashioned", "ppu": 0.55, "batters": { "batter": [{ "id": "1001", "type": "Regular" }, { "id": "1002", "type": "Chocolate" } ] }, "topping": [{ "id": "5001", "type": "None" }, { "id": "5002", "type": "Glazed" }, { "id": "5003", "type": "Chocolate" }, { "id": "5004", "type": "Maple" } ] } ]' https://my.service.name.behind.the.gateway/my/path

curl: (52) Empty reply from server

This does not happen with smaller body payloads:

$ curl -X POST -H "Content-Type:application/json" -d '[{ "id": "0001", "type": "donut", "name": "Cake", "ppu": 0.55 }, { "id": "0002", "type": "donut", "name": "Raised", "ppu": 0.55, "batters": { "batter": [{ "id": "1001", "type": "Regular" }] }, "topping": [{ "id": "5001", "type": "None" }, { "id": "5002", "type": "Glazed" }, { "id": "5005", "type": "Sugar" }, { "id": "5003", "type": "Chocolate" }, { "id": "5004", "type": "Maple" } ] }, { "id": "0003", "type": "donut", "name": "Old Fashioned", "ppu": 0.55 } ]' https://my.service.name.behind.the.gateway/my/path

{
  "timestamp": "2018-04-10T20:46:45Z",
  "status": 404,
  "error": "Not Found",
  "message": "No message available",
  "path": "/my/path"
}

[dan-nawrocki]

This appears to be back for me. I think the regression is caused by 28166054c4e6c16d8eb24874315b72ef5b50d85f. I believe it disappeared in Finchley.RELEASE because there is no Content-Type for a HTTP 100 response, triggering a NullPointerException that somehow caused both the HTTP 100 response and the target HTTP response to be returned back to curl (?).

I don’t know much about spring reactive, but my naive guess from debugging is that spring-cloud-gateway is responding to curl with the HTTP 100 response, and not forwarding the actual response from the target. Does spring-cloud-gateway assume that each request will receive a single response from the target?

I can get curl to work by disabling the 100-Continue feature with -H 'Expect:'.

Edit: @spencergibb any ideas?

[okayrunner]

I was able to reproduce and find a workaround to the issue mentioned by @dan-nawrocki.

It seems like the issue was resolved by the netty team: https://github.com/reactor/reactor-netty/issues/293 but for some reason the Expect header still gets forwarded to the target.

The target as a result returns a 100 Continue (as it should) which results in a NullPointerException on NettyRoutingFilter:117 as it’s missing a Content-Type on the response:

exchange.getAttributes().put("original_response_content_type", headers.getContentType());

As a workaround I was able to prevent Spring Cloud Gateway from forwarding the Expect header by adding the following to my route:

.filters((t)->t.removeRequestHeader("Expect"))