1.16.1 fails for Selenium 4.0.0 and Chrome browser

[tobiasstadler]

I am using Selenium 4.0.0 with Chrome as web browser. When I upgrade testcontainers to 1.16.1 I get the following error:

java.lang.NoSuchMethodError: org.openqa.selenium.chrome.ChromeOptions.addArguments([Ljava/lang/String;)Lorg/openqa/selenium/chrome/ChromeOptions;
		at org.testcontainers.containers.BrowserWebDriverContainer.configure(BrowserWebDriverContainer.java:173)
		at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:322)
		at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:317)

The error does not exist on 1.16.0.

[plagov]

@tobiasstadler what image have you used?
I tried selenium 4 with both 1.16.0 and 1.16.1 and none of them is able to start the container. Curious if the root cause of our issues is the same:

2021-10-20 17:26:15,817 INFO  [?.0.0-20211013]] No capabilities provided, falling back to ChromeOptions
2021-10-20 17:26:15,823 INFO  [?.0.0-20211013]] Creating container for image: selenium/standalone-chrome:4.0.0-20211013
2021-10-20 17:26:16,622 INFO  [?.0.0-20211013]] Starting container with ID: d0ffee14f43df4a1174b71440a6aba8a27a7b9c9900c5a27337c02645f558197
2021-10-20 17:26:17,086 INFO  [?.0.0-20211013]] Container selenium/standalone-chrome:4.0.0-20211013 is starting: d0ffee14f43df4a1174b71440a6aba8a27a7b9c9900c5a27337c02645f558197
2021-10-20 17:26:32,138 ERROR [?.0.0-20211013]] Could not start container
org.rnorth.ducttape.TimeoutException: java.util.concurrent.TimeoutException
	at org.rnorth.ducttape.timeouts.Timeouts.callFuture(Timeouts.java:70)
	at org.rnorth.ducttape.timeouts.Timeouts.doWithTimeout(Timeouts.java:60)
	at org.testcontainers.containers.wait.strategy.WaitAllStrategy.waitUntilReady(WaitAllStrategy.java:53)
	at org.testcontainers.containers.GenericContainer.waitUntilContainerStarted(GenericContainer.java:923)
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:466)
	at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:329)
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81)
	at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:327)
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:315)
Caused by: java.util.concurrent.TimeoutException: null
	at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:204)
	at org.rnorth.ducttape.timeouts.Timeouts.callFuture(Timeouts.java:65)
	... 32 common frames omitted
2021-10-20 17:26:32,170 ERROR [?.0.0-20211013]] Log output from the failed container:
2021-10-20 17:26:17,304 INFO Included extra file "/etc/supervisor/conf.d/selenium.conf" during parsing
2021-10-20 17:26:17,308 INFO supervisord started with pid 9
2021-10-20 17:26:18,311 INFO spawned: 'xvfb' with pid 11
2021-10-20 17:26:18,315 INFO spawned: 'vnc' with pid 12
2021-10-20 17:26:18,318 INFO spawned: 'novnc' with pid 13
2021-10-20 17:26:18,320 INFO spawned: 'selenium-standalone' with pid 14
Setting up SE_NODE_GRID_URL...
2021-10-20 17:26:18,332 INFO success: xvfb entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2021-10-20 17:26:18,333 INFO success: vnc entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2021-10-20 17:26:18,333 INFO success: novnc entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2021-10-20 17:26:18,333 INFO success: selenium-standalone entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
Selenium Grid Standalone configuration: 
[network]
relax-checks = true

[node]
session-timeout = "300"
override-max-sessions = false
detect-drivers = false
max-sessions = 1

[[node.driver-configuration]]
display-name = "chrome"
stereotype = '{"browserName": "chrome", "browserVersion": "94.0", "platformName": "Linux"}'
max-sessions = 1

[tobiasstadler]

You probably need #4580. Just add waitingFor(Wait.forLogMessage(".*Started Selenium Standalone.*", 1)) to your container.

[derTobsch]

We have a similar issue with 1.16.1 that the class java.lang.NoClassDefFoundError: org/openqa/selenium/chrome/ChromeOptions is not found. The thing is that we never used Chrome. In the pom of

  <groupId>org.testcontainers</groupId>
  <artifactId>selenium</artifactId>
  <version>1.16.1</version>

these two dependencies on compile scope appeared.

    <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-remote-driver</artifactId>
      <version>3.141.59</version>
      <scope>compile</scope>
    </dependency>
    <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-chrome-driver</artifactId>
      <version>3.141.59</version>
      <scope>compile</scope>
    </dependency>
``

[tobiasstadler]

As far as I can see, the problem is that Selenium 4 introduced org.openqa.selenium.chromium.ChromiumOptions as a superclass for org.openqa.selenium.chromium.ChromeOptions and moved the addArguments methods to that class. So the Signature of addArguments is now public org.openqa.selenium.chromium.ChromiumOptions org.openqa.selenium.chromium.ChromiumOptions.addArguments(java.lang.String[]) but was public org.openqa.selenium.chromium.ChromeOptions org.openqa.selenium.chromium.ChromeOptions.addArguments(java.lang.String[]).

@kiview Would it be ok to not execute addArguments("--disable-gpu"); in BrowserWebDriverContainer if the Selenium version >= 4.0.0?

[kiview]

This fix should be applied depending on the Chrome version, it is independent of the Selenium version. Since not applying this flag for images that require it has catastrophic consequences, we shouldn't yet neglect this.

[tobiasstadler]

Ok, but how should we proceed then? Is using reflection an option?

[kiview]

No need for reflection, we have access to the image name when we configure the container and can therefore do an educated guess based on the tag format of the official images.

I have not yet looked into the root cause of the API change in Selenium, so did not yet make up my mind of how to solve this in the most compatible way.

As I understand it BrowserWebDriverContainer is currently broken with Chrome if users have Selenium version >= 4.0.0 on their classpath, correct?

[tobiasstadler]

yes, BrowserWebDriverContainer is currently broking with Chrome if users have Selenium version >= 4.0.0 on their classpath

[tobiasstadler]

I actually meant calling addArgument via reflection to circumvent the changed api.

[derTobsch]

It is also broken with selenium < 4 and Firefox as browser, because since 1.16.1 the Chrome dependency is directly integrated and "ChromeOptions" is needed but not found.

see https://github.com/synyx/urlaubsverwaltung/runs/3978441190?check_suite_focus=true
and the branch https://github.com/synyx/urlaubsverwaltung/tree/dependabot/maven/testcontainers.version-1.16.2

[kiview]

Oh right, that's not good, thanks for reporting @derTobsch. Note that we are using ChromeOptions in other parts of the code as well and it has been like this since 2018.

selenium-chrome-driver has provided scope in our Gradle build.

We could also think about removing options.addArguments("--disable-gpu") and adding an explicit note about the need for this for specific Chrome versions to the docs. This originates in an upstream bug from Chrome.

[derTobsch]

Oh right, that's not good, thanks for reporting @derTobsch. Note that we are using ChromeOptions in other parts of the code as well and it has been like this since 2018.

selenium-chrome-driver has provided scope in our Gradle build.

We could also think about removing options.addArguments("--disable-gpu") and adding an explicit note about the need for this for specific Chrome versions to the docs. This originates in an upstream bug from Chrome.

Maybe I am wrong but in the maven build the dependencies

    <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-remote-driver</artifactId>
      <version>3.141.59</version>
      <scope>compile</scope>
    </dependency>
    <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-chrome-driver</artifactId>
      <version>3.141.59</version>
      <scope>compile</scope>
    </dependency>

are now declared compile and provided at the same time.