Move GitLab output rendering to `ruff_db` #20117

ntBre · 2025-08-27T16:43:15Z

Summary

This PR is a first step toward adding a GitLab output format to ty. It converts the GitlabEmitter from ruff_linter to a GitlabRenderer in ruff_db and updates its implementation to handle non-Ruff files and diagnostics without primary spans. I tried to break up the changes here so that they're easy to review commit-by-commit, or at least in groups of commits:

This wasn't strictly necessary, but I also added some Serialize structs instead of calling json! to make it a little clearer that we weren't modifying the schema (e4c4bee).

I plan to follow this up with a separate PR exposing this output format in the ty CLI, which should be quite straightforward.

Test Plan

Existing tests, especially the two that show up in the diff as renamed nearly without changes

it was added specifically for gitlab in #3475 and isn't used anywhere else, so this prepares to move it with the gitlab module to ruff_db

this is a clean move of all the Serialize code and the two helper methods

instead of having everything in the if-let, map the part that actually needs the span, falling back on a default value as required by the spec

this won't affect Ruff, where all diagnostics are still errors, but this should work in ty

github-actions · 2025-08-27T16:45:20Z

Diagnostic diff on typing conformance tests

No changes detected when running ty on typing conformance tests ✅

github-actions · 2025-08-27T16:47:21Z

`mypy_primer` results

No ecosystem changes detected ✅
No memory usage changes detected ✅

github-actions · 2025-08-27T17:06:24Z

`ruff-ecosystem` results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

Formatter (stable)

✅ ecosystem check detected no format changes.

Formatter (preview)

✅ ecosystem check detected no format changes.

ntBre · 2025-08-27T17:18:32Z

crates/ruff_db/src/diagnostic/render/gitlab.rs

+fn relativize_path_to<P: AsRef<Path>, R: AsRef<Path>>(path: P, project_root: R) -> String {
+    format!(
+        "{}",
+        pathdiff::diff_paths(&path, project_root)


I suspect that we could just use our render::relativize_path helper here and drop this dependency, but I kept it around for now. We don't have any existing tests covering the CI_PROJECT_DIR case.

Dropping a dependency sounds great. :-)

I think I'll open a follow-up issue here. I'm pretty sure that we can drop this entirely and just use the CWD-relative path, but I'm not 100% sure and don't want to break GitLab users.

It's a one-function dependency, so we could also vendor it. The behavior seems a bit different from our relativize_path if the project_root isn't a prefix, if we really need that.

BurntSushi

LGTM.

Possible idea for next time: if you have 3 logical groupings of commits, it might make sense to just squash that into 3 commits. Then you can write a commit message for that grouping and it should hopefully make review easier. (And avoid needing to write out the links for the individual groups of commits. Which I assume will go bad if you rebase.)

BurntSushi · 2025-08-27T19:05:12Z

crates/ruff_db/src/diagnostic/render/gitlab.rs

+                diagnostics,
+                resolver: self.resolver,
+                project_dir: std::env::var("CI_PROJECT_DIR").ok().as_deref(),
+            })


How come serde_json::json! here instead of serde_json::to_string? I'd expect the latter to be faster since it doesn't have to construct an intermediate serde_json::Value, but that's just a wild guess.

I guess it's a shame we can't just stream this right to a std::fmt::Write without going through some intermediate value first.

json! hides the unwrap call in the macro and we used it in the other JSON formats, but those aren't very good reasons 😬

But yeah, the Emitters in ruff_linter take std::io::Writes instead of std::fmt::Writes, making this a little easier. We added an adapter in the junit PR, but I haven't used it any of the other formats yet:

ruff/crates/ruff_db/src/diagnostic/render/junit.rs

Lines 150 to 154 in ef0586d

struct FmtAdapter<'a> {

fmt: &'a mut dyn std::fmt::Write,

}

impl std::io::Write for FmtAdapter<'_> {

I switched it over to to_string for now!

BurntSushi · 2025-08-27T19:09:27Z

crates/ruff_db/src/diagnostic/render/gitlab.rs

+fn relativize_path_to<P: AsRef<Path>, R: AsRef<Path>>(path: P, project_root: R) -> String {
+    format!(
+        "{}",
+        pathdiff::diff_paths(&path, project_root)


Dropping a dependency sounds great. :-)

ntBre · 2025-08-27T19:29:54Z

Thanks for the review!

Possible idea for next time: if you have 3 logical groupings of commits, it might make sense to just squash that into 3 commits. Then you can write a commit message for that grouping and it should hopefully make review easier. (And avoid needing to write out the links for the individual groups of commits. Which I assume will go bad if you rebase.)

Will do, sorry about that. I thought each commit would be helpful initially, but then they got a bit too small. I should have squashed after identifying the groups. It's definitely a pain to maintain the links!

this also reorders the fields in the snapshots, but that should be fine

* main: Fix mdtest ignore python code blocks (#20139) [ty] add support for cyclic legacy generic protocols (#20125) [ty] add cycle detection for find_legacy_typevars (#20124) Use new diff rendering format in tests (#20101) [ty] Fix 'too many cycle iterations' for unions of literals (#20137) [ty] No boundness analysis for implicit instance attributes (#20128) Bump 0.12.11 (#20136) [ty] Benchmarks for problematic implicit instance attributes cases (#20133) [`pyflakes`] Fix `allowed-unused-imports` matching for top-level modules (`F401`) (#20115) Move GitLab output rendering to `ruff_db` (#20117) [ty] Evaluate reachability of non-definitely-bound to Ambiguous (#19579) [ty] Introduce a representation for the top/bottom materialization of an invariant generic (#20076) [`flake8-async`] Implement `blocking-http-call-httpx` (`ASYNC212`) (#20091) [ty] print diagnostics with fully qualified name to disambiguate some cases (#19850) [`ruff`] Preserve relative whitespace in multi-line expressions (`RUF033`) (#19647)

## Summary This wires up the GitLab output format moved into `ruff_db` in #20117 to the ty CLI. While I was here, I made one unrelated change to the CLI docs. Clap was rendering the escapes around the `\[default\]` brackets for the `full` output, so I just switched those to parentheses: ``` --output-format <OUTPUT_FORMAT> The format to use for printing diagnostic messages Possible values: - full: Print diagnostics verbosely, with context and helpful hints \[default\] - concise: Print diagnostics concisely, one per line - gitlab: Print diagnostics in the JSON format expected by GitLab Code Quality reports ``` ## Test Plan New CLI test, and a manual test with `--config 'terminal.output-format = "gitlab"'` to make sure this works as a configuration option too. I also tried piping the output through jq to make sure it's at least valid JSON

## Summary This PR is a first step toward adding a GitLab output format to ty. It converts the `GitlabEmitter` from `ruff_linter` to a `GitlabRenderer` in `ruff_db` and updates its implementation to handle non-Ruff files and diagnostics without primary spans. I tried to break up the changes here so that they're easy to review commit-by-commit, or at least in groups of commits: - [preparatory changes in-place in `ruff_linter` and a `ruff_db` skeleton](https://github.com/astral-sh/ruff/pull/20117/files/0761b73a615537fe75fc54ba8f7d5f52c28b2a2a) - [moving the code over with no implementation changes mixed in](https://github.com/astral-sh/ruff/pull/20117/files/0761b73a615537fe75fc54ba8f7d5f52c28b2a2a..8f909ea0bb0892107824b311f3982eb3cb1833ed) - [tidying up the code now in `ruff_db`](https://github.com/astral-sh/ruff/pull/20117/files/9f047c4f9f1c826b963f0fa82ca2411d01d6ec83..e5e217fcd611ab24f516b2af4fa21e25eadc5645) This wasn't strictly necessary, but I also added some `Serialize` structs instead of calling `json!` to make it a little clearer that we weren't modifying the schema (e4c4bee). I plan to follow this up with a separate PR exposing this output format in the ty CLI, which should be quite straightforward. ## Test Plan Existing tests, especially the two that show up in the diff as renamed nearly without changes

## Summary This wires up the GitLab output format moved into `ruff_db` in astral-sh#20117 to the ty CLI. While I was here, I made one unrelated change to the CLI docs. Clap was rendering the escapes around the `\[default\]` brackets for the `full` output, so I just switched those to parentheses: ``` --output-format <OUTPUT_FORMAT> The format to use for printing diagnostic messages Possible values: - full: Print diagnostics verbosely, with context and helpful hints \[default\] - concise: Print diagnostics concisely, one per line - gitlab: Print diagnostics in the JSON format expected by GitLab Code Quality reports ``` ## Test Plan New CLI test, and a manual test with `--config 'terminal.output-format = "gitlab"'` to make sure this works as a configuration option too. I also tried piping the output through jq to make sure it's at least valid JSON

## Summary This is the GitHub analog to #20117. This PR prepares to add a GitHub output format to ty by moving the implementation from `ruff_linter` to `ruff_db`. Hopefully this one is a bit easier to review commit-by-commit. Almost all of the refactoring this time is in the first commit, then the second commit adds the new `OutputFormat` variant and moves the file into `ruff_db`. The third commit is just a small touch up to use a private method that accommodates ty files so that we can run the tests and update/move the snapshots. I had to push a fourth commit to fix and test diagnostics without a span/file. ## Test Plan Existing tests

ntBre added 11 commits August 27, 2025 10:32

move relativize_path_to to gitlab.rs

0a4ef22

it was added specifically for gitlab in #3475 and isn't used anywhere else, so this prepares to move it with the gitlab module to ruff_db

use a FileResolver in SerializedMessages

875a415

add DiagnosticFormat::Gitlab and module skeleton

2a976ff

use UnifiedFile::relative_path instead of fs helper

0761b73

move implementation to ruff_db, wire it up

9f047c4

this is a clean move of all the Serialize code and the two helper methods

add insta filter feature

8f909ea

handle non-ruff files, update snapshots

6a652d2

use structs instead of json!

e4c4bee

don't skip diagnostics without primary spans

01267eb

instead of having everything in the if-let, map the part that actually needs the span, falling back on a default value as required by the spec

use the real diagnostic severity

735b0cf

this won't affect Ruff, where all diagnostics are still errors, but this should work in ty

update fingerprint argument name

2b4a846

ntBre added internal An internal refactor or improvement diagnostics Related to reporting of diagnostics. labels Aug 27, 2025

Position -> Positions to match field name

e5e217f

ntBre commented Aug 27, 2025

View reviewed changes

ntBre marked this pull request as ready for review August 27, 2025 17:19

ntBre requested review from MichaReiser, carljm, dcreager and sharkdp as code owners August 27, 2025 17:19

ntBre requested review from BurntSushi and removed request for carljm, dcreager and sharkdp August 27, 2025 17:19

BurntSushi approved these changes Aug 27, 2025

View reviewed changes

use serde_json::to_string_pretty instead of json!

d0cfdf2

this also reorders the fields in the snapshots, but that should be fine

ntBre mentioned this pull request Aug 27, 2025

Drop pathdiff dependency for GitLab output #20119

Open

ntBre merged commit 1ce6571 into main Aug 28, 2025
38 checks passed

ntBre deleted the brent/gitlab branch August 28, 2025 12:56

ntBre mentioned this pull request Aug 29, 2025

[ty] Add GitLab output format #20155

Merged

ntBre mentioned this pull request Sep 9, 2025

Move GitHub rendering to ruff_db #20320

Merged

	struct FmtAdapter<'a> {
	fmt: &'a mut dyn std::fmt::Write,
	}

	impl std::io::Write for FmtAdapter<'_> {

Move GitLab output rendering to ruff_db #20117

Move GitLab output rendering to ruff_db #20117

Uh oh!

Conversation

ntBre commented Aug 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test Plan

Uh oh!

github-actions bot commented Aug 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Diagnostic diff on typing conformance tests

Uh oh!

github-actions bot commented Aug 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

mypy_primer results

Uh oh!

github-actions bot commented Aug 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

ruff-ecosystem results

Linter (stable)

Linter (preview)

Formatter (stable)

Formatter (preview)

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

BurntSushi left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ntBre commented Aug 27, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Move GitLab output rendering to `ruff_db` #20117

Move GitLab output rendering to `ruff_db` #20117

ntBre commented Aug 27, 2025 •

edited

Loading

github-actions bot commented Aug 27, 2025 •

edited

Loading

github-actions bot commented Aug 27, 2025 •

edited

Loading

`mypy_primer` results

github-actions bot commented Aug 27, 2025 •

edited

Loading

`ruff-ecosystem` results