Question: Should containers/networks be pruned after failed tests?

[bthalmayr]

Hi all, appologies if this is a totally stupid question. I just started to work on a project that uses testcontainers for integration testing. Unfortunately on my Mac the tests all fail, whereas they succeed for other. In my case the fail because of

 11:32:45.603 [Test worker] ERROR ?? [alpine/socat:latest] - Could not start container
org.testcontainers.containers.ContainerLaunchException: Aborting attempt to link to container SOME_CONTAINER as it is not running

When I use 'docker-machine ls' , 'docker container ls' and 'docker network ls' after the failed test, the docker machine is still running, as well as the containers are.

I wonder if the containers and network should have been stopped and pruned after the failed test. They seem to be stopped and pruned if the test succeed.

Software used:
testcontainers: 1.7.1
Docker vor Mac: 18.03.1-ce-mac65
Docker compose: 1.21.1
Docker machine: 0.14.0
Virtual Box: 5.1.2

I already queried the docs/how to, but I could not really find a hint.
Thanks for any pointers!

[kiview]

Hey @bthalmayr, the containers and networks should be deleted after the tests and since Testcontainers 1.6 in a very reliable fashion.

I don't know how stable our Docker-Machine support is ATM, can you test your setup with Docker for Mac directly? This is very well supported.

Or are you actually using Docker Toolbox instead of Docker for Mac?

[bthalmayr]

Thanks for your quick reply @kiview, much appreciated.

Actually deleted Docker Toolbox as documented in https://docs.docker.com/toolbox/toolbox_install_mac/#how-to-uninstall-toolbox and I thought I'm only using Docker for Mac now.

I also deleted the default machine, but then I get

Caused by: java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration
	at org.testcontainers.dockerclient.DockerClientProviderStrategy.lambda$getFirstValidStrategy$3(DockerClientProviderStrategy.java:155

when running the test.

Enhancing logging then showed me

[Test worker] DEBUG org.testcontainers.utility.TestcontainersConfiguration - Testcontainers configuration overrides loaded from TestcontainersConfiguration(properties={compose.container.image=docker/compose:1.18.0})
[Test worker] DEBUG org.testcontainers.dockerclient.DockerClientProviderStrategy - ProxiedUnixSocketClientProviderStrategy: failed with exception NoClassDefFoundError (org/rnorth/tcpunixsocketproxy/TcpToUnixSocketProxy). Root cause ClassNotFoundException (org.rnorth.tcpunixsocketproxy.TcpToUnixSocketProxy)
[Test worker] DEBUG org.testcontainers.utility.CommandLine - Executing shell command: `docker-machine ls -q`

and reading the Docker for Mac docs told me

if you are using Docker for Mac

Docker for Mac uses HyperKit, a lightweight macOS virtualization solution built on top of the Hypervisor.framework.

Currently, there is no docker-machine create driver for HyperKit, so use the virtualbox driver to >create local machines. "

so I thought Virtual box and 'docker-machine' is needed.

The test uses DockerComposeContainer; is this meant to use 'docker-machine'?

I'm really sorry for my insufficient knowledge, I'm trying to get up to speed as fast as possible.

[kiview]

Hey @bthalmayr,

You can use Docker for Mac without Docker-Machine and Virtualbox, it's totally fine.
DockerComposeContainer doesn't use Docker-Machine per default, Testcontainers actually tries to detect the Docker environment by itself. In your case, Testcontainers detected Docker-Machine as it's environment, which isn't really what you want, you would want to use Docker for Mac directly I assume.

I'm sure @bsideup can give you a hint about how to cleanup your installation to correctly use Docker for Mac (I'm no Mac user).

Can you also please update to Testcontainers 1.7.3, just to be sure?

[bthalmayr]

This is quite intersting @kiview , thanks for pointing this out.

After removing Docker Toolbox no 'docker-machine' command was available in the shell, but after I installed Docker for Mac

/usr/local/bin/docker-machine

was created. In fact this is a symbolic link to a binary installed by Docker for Mac app.

lrwxr-xr-x  1 USERID  staff  62 Jun  7 15:31 /usr/local/bin/docker-machine -> /Applications/Docker.app/Contents/Resources/bin/docker-machine

so I try to remove the link, so that DockerComposeContainer will not be able to detect it.

It seems that DockerComposeContainer first choice is to use docker-machine.

[bthalmayr]

too bad, when I remove the symbolic link, then no Docker environmenment can be determined at all

[Test worker] DEBUG org.testcontainers.utility.TestcontainersConfiguration - Testcontainers configuration overrides loaded from TestcontainersConfiguration(properties={compose.container.image=docker/compose:1.18.0})
[Test worker] DEBUG org.testcontainers.dockerclient.DockerClientProviderStrategy - ProxiedUnixSocketClientProviderStrategy: failed with exception NoClassDefFoundError (org/rnorth/tcpunixsocketproxy/TcpToUnixSocketProxy). Root cause ClassNotFoundException (org.rnorth.tcpunixsocketproxy.TcpToUnixSocketProxy)
[Test worker] ERROR org.testcontainers.dockerclient.DockerClientProviderStrategy - Could not find a valid Docker environment. Please check configuration. Attempted configurations were:
[Test worker] ERROR org.testcontainers.dockerclient.DockerClientProviderStrategy -     ProxiedUnixSocketClientProviderStrategy: failed with exception NoClassDefFoundError (org/rnorth/tcpunixsocketproxy/TcpToUnixSocketProxy). Root cause ClassNotFoundException (org.rnorth.tcpunixsocketproxy.TcpToUnixSocketProxy)
[Test worker] ERROR org.testcontainers.dockerclient.DockerClientProviderStrategy - As no valid configuration was found, execution cannot continue

but even 'trace' level logging does not reveal more info.

[bsideup]

@bthalmayr the error you get (NoClassDefFoundError (org/rnorth/tcpunixsocketproxy/TcpToUnixSocketProxy)) looks like #693 and was fixed in 1.7.3 release, please update

[kiview]

Don't delete the symbolic link, it's okay to have Docker-Machine installed as part of Docker for Mac and should not be related to your problem.

Also what @bsideup said, I mentioned the update in my previous comment 😉

[bthalmayr]

Thanks @bsideup , @kiview

The strange thing is that the other co-workers have the same config (Docker for Mac) and there it's working fine, so I did not think updating testcontainer to version 1.7.3 would solve it.

To make sure it's not related to the integrationTest of the project, I created a simple test based on the description from https://www.testcontainers.org/usage/docker_compose.html

Unfortunately I was getting the same error.

After upgrading testcontainer version to 1.7.3 the simple test succeeded.

I really wonder what's the difference in our team's setup.

[bsideup]

@bthalmayr you have Mac OS X 10.11 while your co-workers have 10.12+? :)

[bthalmayr]

I'll check this @bsideup

Thanks a lot for your quick help!