GitHub CI Improvement

[yordis]

Intent: The main objective of this effort is to offer a broader range of pre-built templates that can cater to most of the common scenarios for maintaining a healthy package setup during the CI phase, excluding CD for now. By providing these templates, we aim to make setting up the CI environment smoother and more streamlined, reducing the potential for any issues that may arise. The result of this initiative is to maintain a highly predictable quality of CI setup across the entire ecosystem, ensuring that all users can benefit from an efficient and reliable CI process.

Tasks to take into consideration:

• Testing (mix test)
• Linting (credo)
• Type Check (dialyzer)
• Formatting (mix format)
• Docs (inch_ex?!)

Extras

Maybe we could consider even extra setups around:

• Check CHANGELOG.md, LICENSE, CODE_OF_CONDUCT.md and [CONTRIBUTING.md](http://CONTRIBUTING.md) formatting, linting or [Insert Idea Here].

Links

• https://keepachangelog.com/en/1.1.0/
• Publish action for mix and rebar3_hex  build-and-packaging-wg#56 The intent focused too much on the CD and publishing the packages, overshadowing the CI improvement.
• https://fly.io/phoenix-files/github-actions-for-elixir-ci/
• https://github.com/felt/ultimate-elixir-ci
• https://github.com/straw-hat-team/github-actions-workflows/blob/master/.github/workflows/elixir-quality-assurance.md
• https://github.com/zachdaniel/git_ops
• https://github.com/mtrudel/bandit/blob/main/.github/workflows/elixir.yml
• https://github.com/stordco/actions-elixir/blob/main/setup/action.yaml

[zachdaniel]

We also maintain a suite of CI steps for Elixir, which is what Ash uses for all of its projects: https://github.com/team-alembic/staple-actions

[zachdaniel]

Another library to consider: https://github.com/karolsluszniak/ex_check

[zachdaniel]

I think it might be pretty interesting to see if we could get options added to mix new that would include --with ex_check and --with github_ci. It sounds unlikely that they would want that, but if they would, we could have both of those add required dependencies and files, and could be a very powerful getting started tool for users. If they are not interested in that, we could potentially provide something like mix our_own_custom_thing which supports those options, first calling mix new and then modifying the output files.

[zachdaniel]

Could even do --with git_ops

[barnabasJ]

Gleam is already doing something like this. It creates github-actions by default when creating a new project.

They have the --skip-github flag if you do not want this.

[wingyplus]

I vote a custom mix task way. We know that GitHub workflow is mostly used but we also expand to other CI tools that can be covered, such as GitLab CI, BitBucket, container-based tools, dagger, etc.

[btkostner]

Speaking from experience at Stord and the https://github.com/stordco/actions-elixir workflows, we created those workflows because we have a lot of custom stuff like GitHub authentication for non published packages, hex authentication, oban authentication, etc. I'm not sure how generalized this could be for other organizations though.

One of the issues we've hit is with generalizing things like doc building, some projects require other comments (like erd building or openapi generation) to run before doc building. The solution we came up with is having all projects alias these commands in their mix.exs file to clean things up. Ideally I'd love it if this was the way the Elixir community went but it might be a large ask.

I like the idea of having a mix command to generate these files because I have a feeling a lot of orgs will need to customize these workflows down the line. At the same time, I think updating and expanding on the erlef/setup-beam action with more caching would go a long way. It already supports reading from .tool-versions which ends up being most of the setup boilerplate work.

[zachdaniel]

I'm typically not a fan of code generation as it makes it impossible for the original maintainers to make changes to. I don't know if there is any other option really in this case, though, because you wouldn't want CI, build behavior or any other thing changing underneath you.

[Nezteb]

One scalable way to do code generation is if you also write your own Credo checks to validate that the generated code is still correct / up to date. As the generator and its respective Credo checks evolve, it should point out and stale generated code. I've been meaning to write a blog post about this, but...

Then again, most people probably don't want to write/maintain a lot of Credo checks. 😅

[zachdaniel]

Yeah, but the main problem is that by doing code generation you're effectively making a statement: "we expect you to modify this". If we expect them to keep it the way we made it, and use our options to change it, then a library is better, i.e some mix task plus config file that manages everything.

[Nezteb]

Random mind dump:

• Elixir + Docker for CI/tests
  • https://elixirforum.com/t/excontainers-throwaway-containers-for-your-tests/36920
    • https://github.com/dallagi/excontainers
    • https://github.com/testcontainers/testcontainers-elixir
• CI tooling that promotes local CI runs
  • https://github.com/harness/gitness (30.6k stars)
  • https://github.com/earthly/earthly (9.9k stars)
  • https://github.com/dagger/dagger (9.3k stars)
  • https://github.com/tektoncd/pipeline (8.1k stars)
  • https://github.com/garden-io/garden (3.1k stars)

I'm also working on a blog post about using Nix with Elixir, including for CI: https://github.com/Nezteb/nix_elixir

[jarlah]

im totally biased but: its very easy to use testcontainers for elixir in GitHub actions :) testcontainers for elixir is the successor of dallagi/excontainers

[jarlah]

https://github.com/testcontainers/testcontainers-elixir also supports podman now. Just saying.