OpenTelemetryMetricsContainerConnectionDetailsFactoryIntegrationTests is flaky when running locally

[philwebb]

I seem to often have this test fail locally. See https://ge.spring.io/scans/tests?search.tags=Mac%20OS%20X&search.timeZoneId=America%2FLos_Angeles&tests.container=org.springframework.boot.testcontainers.service.connection.otlp.OpenTelemetryMetricsContainerConnectionDetailsFactoryIntegrationTests&tests.test=connectionCanBeMadeToOpenTelemetryCollectorContainer()#

[wilkinsona]

Some of the logs from the failures are interesting:

2024-06-14T10:18:11.445-07:00  WARN 6244 --- [      cb-events] com.couchbase.endpoint                   : [com.couchbase.endpoint][EndpointConnectionFailedEvent][802us] Connect attempt 19 failed because of : finishConnect(..) failed: Connection refused: localhost/127.0.0.1:60909 - Check server ports and cluster encryption setting. {"circuitBreaker":"DISABLED","coreId":"0x19911f6e00000001","remote":"localhost:60909","type":"KV"}	
com.couchbase.client.core.endpoint.BaseEndpoint$2: finishConnect(..) failed: Connection refused: localhost/127.0.0.1:60909 - Check server ports and cluster encryption setting.	
2024-06-14T10:18:11.732-07:00  INFO 6244 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient   : [Producer clientId=producer-1] Node 1 disconnected.	
2024-06-14T10:18:11.732-07:00  WARN 6244 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient   : [Producer clientId=producer-1] Connection to node 1 (localhost/127.0.0.1:62193) could not be established. Broker may not be available.	
2024-06-14T10:18:11.804-07:00  INFO 6244 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-test-group-1, groupId=test-group] Node 1 disconnected.	
2024-06-14T10:18:11.804-07:00  WARN 6244 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-test-group-1, groupId=test-group] Connection to node 1 (localhost/127.0.0.1:62193) could not be established. Broker may not be available.	
2024-06-14T10:18:11.872-07:00  INFO 6244 --- [aitility-thread] o.a.http.impl.client.DefaultHttpClient   : I/O exception (org.apache.http.NoHttpResponseException) caught when processing request to {}->http://localhost:62485/: The target server failed to respond

I can see Couchbase (on thread cb-events), Kafka (on threads ad | producer-1 and ntainer#0-0-C-1) and what I assume is the test that's failing (on thread aitility-thread). I would guess the Couchbase and Kafka stuff are remnants of earlier tests. It looks like the containers that they were using have been stopped but that the application contexts have been cached and remain active.

I'm not sure that the above is causing the flakiness, but it is a bit messy and shows a mismatch in lifecycle between the @Testcontainers-managed container and that application context. OpenTelemetryMetricsContainerConnectionDetailsFactoryIntegrationTests has @DirtiesContext on it. It might help if we could use that more liberally and added it to KafkaContainerConnectionDetailsFactoryIntegrationTests and the like. Alternatively, we could disable the context cache as I don't think we'll be benefiting from it with every test class likely to have different merged context configuration.

[philwebb]

We have #38237 for the idea of automatically applying @DirtiesContext.

[wilkinsona]

I looked through the tests in spring-boot-testcontainers and the cache doesn't seem to provide any benefit so I've disabled it in #41124. It'll be interesting to see if it has any effect on the flakiness.