Skip to content

Tests requiring the conftest binary fail with newer versions of the utility #2600

@tpickett66

Description

@tpickett66

When running the tests on a fresh system the following failure occurs:

--- FAIL: TestGitHubWorkflowWithPolicyCheck (0.03s)
    events_controller_e2e_test.go:1324: conftest >= 0.25.0 must be installed to run this test
FAIL
FAIL	github.com/runatlantis/atlantis/server/controllers/events	131.108s
FAIL

When the latest version of conftest (0.34.0) is installed using homebrew this failure persists despite the binary being on $PATH. Digging into the code I found that the tests are looking for a binary named conftest0.25.0, adding a symlink with this name allowed the tests to find the binary but they still fail with a different error.

» go test ./server/controllers/events
--- FAIL: TestGitHubWorkflowWithPolicyCheck (0.13s)
    events_controller_e2e_test.go:1335: could not parse contest version from Conftest: 0.34.0
        OPA: 0.43.0
FAIL
FAIL	github.com/runatlantis/atlantis/server/controllers/events	138.859s
FAIL

It seems like there are two problems here.

  1. We're looking for a binary with a version in the name rather than the generic binary name
  2. The version detection logic checking the program output is not handling the new output correctly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions