[🐛 Bug]: Timeout (10 seconds) when running SM in Java bindings

[bonigarcia]

What happened?

In selenium-java 4.12.0-SNAPSHOT, there is a hardcoded timeout of 10 seconds for the execution of Selenium Manager. This timeout is insufficient to download browsers, and maybe even for drivers in slow connections.

How can we reproduce the issue?

Any test using the automated browser management (Chrome or Firefox) in Java will be broken because of this timeout. For example:

import static org.assertj.core.api.Assertions.assertThat;

import java.util.Arrays;
import java.util.logging.Level;
import java.util.logging.Logger;

import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.firefox.FirefoxOptions;
import org.openqa.selenium.manager.SeleniumManager;

class BasicTest {

    WebDriver driver;

    @BeforeEach
    void setup() {
        Arrays.stream(Logger.getLogger("").getHandlers()).forEach(handler -> {
            handler.setLevel(Level.FINE);
        });
        Logger.getLogger(SeleniumManager.class.getName()).setLevel(Level.FINE);

        FirefoxOptions options = new FirefoxOptions();
        options.setBrowserVersion("118");
        driver = new FirefoxDriver(options);
    }

    @AfterEach
    void teardown() {
        if (driver != null) {
            driver.quit();
        }
    }

    @Test
    void test() {
        driver.get("https://bonigarcia.dev/selenium-webdriver-java/");
        assertThat(driver.getTitle()).contains("Selenium WebDriver");
    }

}

Relevant log output

org.openqa.selenium.remote.NoSuchDriverException: Unable to obtain: Capabilities {acceptInsecureCerts: true, browserName: firefox, browserVersion: 118, moz:debuggerAddress: true, moz:firefoxOptions: {}}, error Failed to run command: [--browser, firefox, --output, json, --browser-version, 118, --debug]
	at org.openqa.selenium.remote.service.DriverFinder.getPath(DriverFinder.java:25)
	at org.openqa.selenium.remote.service.DriverFinder.getPath(DriverFinder.java:13)
	at org.openqa.selenium.firefox.FirefoxDriver.generateExecutor(FirefoxDriver.java:141)
	at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:132)
	at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:127)
	at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:112)
	at io.github.bonigarcia.wdm.test.BasicTest.setup(BasicTest.java:46)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
Caused by: org.openqa.selenium.WebDriverException: Failed to run command: [--browser, firefox, --output, json, --browser-version, 118, --debug]
	at org.openqa.selenium.manager.SeleniumManager.runCommand(SeleniumManager.java:121)
	at org.openqa.selenium.manager.SeleniumManager.getDriverPath(SeleniumManager.java:286)
	at org.openqa.selenium.remote.service.DriverFinder.getPath(DriverFinder.java:22)
	... 9 more
Caused by: org.openqa.selenium.TimeoutException: Process timed out after waiting for 10000 ms.
Build info: version: '4.12.0-SNAPSHOT', revision: '1104c8f'
System info: os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0', java.version: '11.0.15'
Driver info: driver.version: FirefoxDriver
	at org.openqa.selenium.os.OsProcess.waitFor(OsProcess.java:183)
	at org.openqa.selenium.os.CommandLine.waitFor(CommandLine.java:117)
	at org.openqa.selenium.manager.SeleniumManager.runCommand(SeleniumManager.java:114)
	... 11 more

Operating System

Not relevant

Selenium version

4.12.0-SNAPSHOT

What are the browser(s) and version(s) where you see this issue?

Chrome, Firefox

What are the browser driver(s) and version(s) where you see this issue?

Not relevant

Are you using Selenium Grid?

No

[github-actions]

@bonigarcia, thank you for creating this issue. We will troubleshoot it as soon as we can.

---

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

[bonigarcia]

Note: I am going to send two different PR's to solve this issue (to chose one or another).

[bonigarcia]

Here it is two alternative solutions for this problem:

1. [java] Increase execution timeout of SM in Java from 10 to 120 seconds #12631 - that increases the execution timeout from 10 to 120 seconds, with a hardcoded value. As discussed on Slack, maybe the most efficient way to do it is with a configurable value, but this maybe is enough for most users (depending on the network connection speed).
2. [java] Remove timeout of 10 seconds when executing SM #12632 - that removes the execution timeout.

I think the most optimal solution would be to have a configurable value for the execution timeout. But this solution should be the same across the different languages. Since we don't have time to implement it for 4.12.0, my opinion is that #12632 is better since it is the approach done in other bindings (see dotnet). Moreover, this way, we are delegating in the SM timeout, which is already configurable using the configuration file or the env SE_TIMEOUT (which has a default value of 300 seconds).

@diemol @titusfortner Please merge the solution you think is more convenient and discard the other.

[bonigarcia]

@diemol @titusfortner I think #12632 would be better to solve this problem because of the following:

1. This timeout in Java breaks the "principle" of doing the same across all bindings. I reviewed all the implementations of the SeleniumManager class in the other bindings, and I believe Java is the only one using an execution timeout for SM.
2. Not using a timeout for the execution of SM is a way of delegating this responsibility to the SM binary. Which somehow is another "principle" we are following lately (i.e., not adding complexity in the bindings and centralizing the underlying logic in SM). Of course, SM could have a bug or something unexpected and freeze forever, but that should not happen in theory.
3. I believe 120 seconds can be enough for most users. Still, I feel 2 minutes could not be enough for downloading and uncompressing browsers for slow connections or slow processing machines.
4. A hardcoded value is a substantial limitation that cannot be overridden in the case of slow connection/processing. On the other hand, SM already has a configurable value for request timeout.

[diemol]

I don't have a strong opinion. We can remove the timeout and see how it plays after release.

It would be nice to have to stream Selenium Manager's activity when it is downloading things. Even if it takes two minutes, the user can see what is happening and avoid thinking that "Selenium is hanging."

If you wish, push directly to trunk the change :)

[bonigarcia]

It would be nice to have to stream Selenium Manager's activity when it is downloading things. Even if it takes two minutes, the user can see what is happening and avoid thinking that "Selenium is hanging."

Yes, I agree. A low cost alternative could be displaying a message from the bindings with something similar to "Executing Selenium Manager - this can take a while when downloading browsers for the first time".

If you wish, push directly to trunk the change :)

I am not changing this, but thanks. My opinion is clear: 120 seconds is not the best. But Titus preferred that solution over the PR without a wait.

[diemol]

@titusfortner, what do you think?

[titusfortner]

I only picked it because I didn't want to think too hard about why/when a timeout is needed, and it seemed a reasonable temp option while we sort what other things are doing. I thought I was about to do a release as soon as I merged it, but then Jason gave me more work to do 😂