HTTP/2 response bodies (strict) bigger than 16KiB are not being sent with Ember

As I pointed out in #6842, there seems to a problem with ember when responding with bodies longer than 16KiB. I was able to isolate the issue and it seems to be related exclusively with HTTP/2.

This is a scala-cli file you can use to replicate it:

//> using scala "3.2.0"
//> using lib "org.typelevel::cats-effect:3.4.2"
//> using lib "co.fs2::fs2-core:3.4.0"
//> using lib "co.fs2::fs2-io:3.4.0"
//> using lib "org.http4s::http4s-dsl:1.0.0-M37"
//> using lib "org.http4s::http4s-ember-server:1.0.0-M37"
//> using lib "org.slf4j:slf4j-simple:2.0.5"

import cats.syntax.all.*
import cats.effect.{Async, ExitCode, IO, IOApp, Resource}
import cats.effect.syntax.all.*
import org.http4s.HttpRoutes
import org.http4s.MediaType
import org.http4s.headers.`Content-Type`
import org.http4s.dsl.Http4sDsl
import org.http4s.ember.server.EmberServerBuilder
import com.comcast.ip4s.port

object RunServer extends IOApp with Http4sDsl[IO]:
  private val app = HttpRoutes.of[IO] {
    case GET -> Root / "bytes" / IntVar(numberOfBytes) =>
      val bytes = Array.fill(numberOfBytes)('A'.toByte)

      Ok(bytes).map(_.withContentType(`Content-Type`(MediaType.text.plain)))
  }.orNotFound

  private val server =
    EmberServerBuilder
      .default[IO]
      .withHttpApp(app)
      .withHttp2
      .withPort(port"5000")
      .build

  override def run(args: List[String]): IO[ExitCode] =
    server.useForever

I also uploaded an SBT project where you can check how it behaves with(out) TLS and HTTP/2 in separate services (https://github.com/janilcgarcia/http4s-bug).

Using curl:

curl --http2 localhost:5000/bytes/16384
# result is fine, you get the entire body
curl --http1.1 localhost:5000/bytes/16384
# same

curl --http2 localhost:5000/bytes/16385
# Get headers but no body,
# curl: (18) transfer closed with 16385 bytes remaining to read

curl --http1.1 localhost:5000/bytes/16385
# works ok as well

Libraries/Scala versions are up there. I’ll try to test it ember-server built from source later.

I would love to hlep, but I’d need some pointers to where to look for this bug, I never implemented anything related to an HTTP server before.

I’m on Debian 11 with OpenJDK 17 from the repository, but the same issue happens on Fedora 37 with their OpenJDK 17

[armanbilge]

@janilcgarcia awesome! Would you mind targeting your PR to the series/0.23 branch? Thanks!

[janilcgarcia]

I’m making a pull request!