`ElasticsearchContainer` in 1.17.x does not handle `latest` tags correctly

[brentryan]

We're currently trying to leverage testcontainers-java with the elasticsearch module and noticed that the most recent version 1.17.1 breaks our use of the elasticsearch module. I think it's due to this commit c7449ed

We're currently trying to leverage elasticsearch version 7.4.2 but our containers fail to start with the error

org.testcontainers.containers.ContainerLaunchException: Container startup failed
Caused by: org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception
Caused by: org.testcontainers.containers.ContainerLaunchException: Could not create/start container
Caused by: com.github.dockerjava.api.exception.NotFoundException: 
Status 404: {"message":"Could not find the file /usr/share/elasticsearch/config/certs/http_ca.crt in container df2a5216dc041fb543e4d156a05ec268f1db4cadb5efef170200b841d712b2a9"}

Please advise?

[bsideup]

Hey @spinscale, do you have any clue what could be happening?

[spinscale]

This looks, as if the check for version 8 is somehow returning that this is indeed a version 8 running. This check seems to return the version is version 8...

new ComparableVersion(dockerImageName.getVersionPart()).isGreaterThanOrEqualTo("8.0.0");

@brentryan can you share some code how you are initializing the container and what version you are using?

The following snippet worked for me, but maybe there is more to it:

        try (ElasticsearchContainer container = new ElasticsearchContainer(DockerImageName.parse("docker.elastic.co/elasticsearch/elasticsearch").withTag("7.4.2"))) {
            // Start the container. This step might take some time...
            container.start();
        }

[brentryan]

It looks like this is because the underlying image has a tagged version of "latest" which then triggers this code which fails.

The code we're using to initialize our container is loading some elasticsearch plugins so we have to do it slightly differently. Let us know if there's a better way, but here's what the init code looks like that fails:

                ImageFromDockerfile dockerImage = new ImageFromDockerfile()
                        .withDockerfileFromBuilder(builder -> {
                            builder.from(getDockerImageName() + ":" + getDockerImageVersion());
                            builder.run(getEsPluginName());
                            if (getEsPluginPartsName().length > 0) {
                                builder.run(getEsPluginPartsName());
                            }
                            builder.build();
                        });

                container = new ElasticsearchContainer(
                        DockerImageName.parse(dockerImage.get())
                                .asCompatibleSubstituteFor(DEFAULT_ELASTICSEARCH_DEFAULT_IMAGE)
                );

[spinscale]

can you try the following

try (ElasticsearchContainer container = new ElasticsearchContainer("YOUR_VERSION_GOES_HERE"){
            @Override
            protected void containerIsStarted(InspectContainerResponse containerInfo) {
            }
        }) {

and see if that works?

[spinscale]

This seems indeed the culprit as new ComparableVersion("latest").isGreaterThanOrEqualTo("8.0.0") returns true.

It might make sense to just log a warning, if the CA cert file is not available instead of throwing a hard exception, as the version can be anything with custom images. I'll open a PR.

[brentryan]

is there a better way to load custom plugins and use custom images like this?

It seems like I'd want ElasticsearchContainer to have a constructor that takes a type ImageFromDockerfile and does the right thing.