[lima] Containers haven't been removed after tests finished

Hi!

I’m trying to debug an issue with containers cleanup after tests finished.

CONTAINER ID   IMAGE                   COMMAND                  CREATED              STATUS              PORTS                                                                                         NAMES
b25b97adbc02   rabbitmq:3.8.9-alpine   "docker-entrypoint.s…"   About a minute ago   Up About a minute   4369/tcp, 5671/tcp, 15691-15692/tcp, 25672/tcp, 0.0.0.0:49156->5672/tcp, :::49156->5672/tcp   condescending_mayer

This problem reproduces for lima but doesn’t reproduce for colima.

I found that there are some call to docker daemon:

Feb 21 10:36:42 lima-default dockerd[2618]: time="2022-02-21T10:36:42.187038150Z" level=debug msg="Calling GET /_ping"
Feb 21 10:37:15 lima-default dockerd[2618]: time="2022-02-21T10:37:15.151272519Z" level=debug msg="Calling GET /v1.29/containers/json?all=1&filters=%7B%22label%22%3A%7B%22org.testcontainers.sessionId%3D8935962b-acef-4214-8923-f10e0491eb50%22%3Atrue%2C%22org.testcontainers%3Dtrue%22%3Atrue%7D%7D&limit=0"
Feb 21 10:37:15 lima-default dockerd[2618]: time="2022-02-21T10:37:15.153219019Z" level=debug msg="Calling POST /v1.29/networks/prune?filters=%7B%22label%22%3A%7B%22org.testcontainers.sessionId%3D8935962b-acef-4214-8923-f10e0491eb50%22%3Atrue%2C%22org.testcontainers%3Dtrue%22%3Atrue%7D%7D"
Feb 21 10:37:15 lima-default dockerd[2618]: time="2022-02-21T10:37:15.154972968Z" level=debug msg="Calling POST /v1.29/volumes/prune?filters=%7B%22label%22%3A%7B%22org.testcontainers.sessionId%3D8935962b-acef-4214-8923-f10e0491eb50%22%3Atrue%2C%22org.testcontainers%3Dtrue%22%3Atrue%7D%7D"
Feb 21 10:37:15 lima-default dockerd[2618]: time="2022-02-21T10:37:15.155117411Z" level=debug msg=VolumeStore.Find ByType=service.andCombinator ByValue="[[local] false 0x55e010461980 0x55e010463080]"
Feb 21 10:37:15 lima-default dockerd[2618]: time="2022-02-21T10:37:15.155951293Z" level=debug msg="Calling POST /v1.29/images/prune?filters=%7B%22dangling%22%3A%7B%22false%22%3Atrue%7D%2C%22label%22%3A%7B%22org.testcontainers.sessionId%3D8935962b-acef-4214-8923-f10e0491eb50%22%3Atrue%2C%22org.testcontainers%3Dtrue%22%3Atrue%7D%7D"

And no errors in ryuk container just information that nothing has been deleted:

2022/02/21 10:39:25 Pinging Docker...
2022/02/21 10:39:25 Docker daemon is available!
2022/02/21 10:39:25 Starting on port 8080...
2022/02/21 10:39:25 Started!
2022/02/21 10:39:26 New client connected: 172.17.0.1:52086
2022/02/21 10:39:26 Received the first connection
2022/02/21 10:39:26 Adding {"label":{"org.testcontainers.sessionId=3caaa426-0f78-48d4-85d1-1d30c6314374":true,"org.testcontainers=true":true}}
2022/02/21 10:39:50 EOF
2022/02/21 10:39:50 Client disconnected: 172.17.0.1:52086
2022/02/21 10:40:00 Timed out waiting for re-connection
2022/02/21 10:40:00 Deleting {"label":{"org.testcontainers.sessionId=3caaa426-0f78-48d4-85d1-1d30c6314374":true,"org.testcontainers=true":true}}
2022/02/21 10:40:00 Removed 0 container(s), 0 network(s), 0 volume(s) 0 image(s)

Docker info:

Client:
 Context:    rootless
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Docker Buildx (Docker Inc., v0.7.1-docker)
  scan: Docker Scan (Docker Inc., v0.12.0)

Server:
 Containers: 16
  Running: 2
  Paused: 0
  Stopped: 14
 Images: 17
 Server Version: 20.10.12
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: false
  userxattr: true
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7b11cfaabd73bb80907dd23182b9347b4245eb5d
 runc version: v1.0.2-0-g52b36a2
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
  rootless
  cgroupns
 Kernel Version: 5.13.0-28-generic
 Operating System: Ubuntu 21.10
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 3.827GiB
 Name: lima-default
 ID: SNWC:ON44:TP6A:CBG5:JN5Q:32KS:5EKF:3DUJ:QY22:O74Q:SVIG:DRCQ
 Docker Root Dir: /home/username.linux/.local/share/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Any help is appreciated.

[kiview]

@dnatic09 Please try the approach outlined in the comment first, I know of a couple of different Testcontainers users that got a working setup like this.

Also, your issue seems unrelated to what @Jasstkn reported, please open a dedicated issue if it persists. Your problem seems to be related to port publishing (rather than port mapping, which you proofed to work).

@bsideup We had a similar issue in Docker for Desktop on Windows, which was due to firewall config with regards to WSL VM and lead to published ports being opened in a blocked range, but only when --publish-all was used.

[dnatic09]

Strategy of how you build tests is VERY important. I try to start only one container per test fixture and repurpose that container for ALL tests within that fixture. There are many ways to accomplish this and it requires a degree of creativity. Do not allow each micro-test to run its own version of the container. It is not even stable using Docker Desktop.