question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

org.http4s.client.middleware.Retry consumes memory proportional to number of retries performed

See original GitHub issue

Please consider this simple example:

name := "http4s-retry-leak"
version := "0.1"
scalaVersion := "2.13.2"
libraryDependencies += "org.http4s" %% "http4s-blaze-client" % "0.21.4"
import cats.effect.{ExitCode, IO, IOApp}
import cats.syntax.option._
import javax.net.ssl.SSLContext
import org.http4s.client.blaze.BlazeClientBuilder
import org.http4s.client.middleware.{Retry, RetryPolicy}
import org.http4s.{Method, Request, Uri}

import scala.concurrent.ExecutionContext.global
import scala.concurrent.duration._

object App extends IOApp {
  override def run(args: List[String]): IO[ExitCode] =
    BlazeClientBuilder[IO](global, SSLContext.getDefault.some).resource
      .use { client =>
        val retryPolicy: RetryPolicy[IO] = (_, _, _) => 5.millis.some
        val clientWithRetries = Retry[IO](retryPolicy)(client)
        val request =
          Request[IO](Method.GET,
                      uri = Uri.fromString("http://unknown-host").toOption.get)
        clientWithRetries.stream(request).compile.drain
      }
      .as(ExitCode.Success)
}

Each time the client makes an attempt, a series of new objects get allocated and retained. Here’s how it looks like after ~148k attempts:

Screenshot 2020-05-15 at 16 13 33

And here you can find the heap dump itself: https://www.dropbox.com/t/GkFbbEMDge8uJdba

From what I understand, the issue might as well be in fs2, not in http4s, but I haven’t yet found a way to minimize the example further.

Issue Analytics

  • State:open
  • Created 3 years ago
  • Comments:9 (6 by maintainers)

github_iconTop GitHub Comments

2reactions
Daenythcommented, May 26, 2020

A sawtooth shape in the heap graph is very normal, it usually indicates that garbage collection ran and removed objects from the heap. If anything, that indicates that it’s maybe not a leak (where references are held and cannot be garbage collected), but rather memory pressure (lots of garbage being created and then cleaned)

0reactions
rossabakercommented, Apr 11, 2021

I wonder whether this is fixed in 1.0, with its hotswap.

Read more comments on GitHub >

github_iconTop Results From Across the Web

object Retry - http4s
Provides extension methods for using the a http4s org.http4s.client.Client import scalaz.concurrent.Task import org.http4s._ import org.http4s.client.
Read more >
Changelog - http4s
A new internal websocket client is published for implementation across repos. We expect to release this as a public API in the next...
Read more >
HTTP Client - http4s
Http4s includes some middleware Out of the Box in the org.http4s.client.middleware package. These include: Following of redirect responses (Follow Redirect) ...
Read more >
HTTP Client - http4s
Let's create a client with http4s to try our service. ... _ import org.http4s.server.middleware. ... There are a number of ways to construct...
Read more >
HTTP Client - http4s
The Client trait in http4s can submit a Request to a server and return a Response . ... EmberServerBuilder import org.http4s.server.middleware.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found