feat: Improve claude log preprocessing command #1786
Conversation
Signed-off-by: litt3 <[email protected]>
litt3
changed the title
| #### Profile 1: Test Failures | ||
| For standard test output failures: | ||
| ```bash | ||
| rg --line-number --ignore-case --json -C 5 -- "^[-]{3} FAIL:|\\s+FAIL\$|\\s+FAIL\\t|\\[FAILED\\]|panic: test timed out" "<original_log_directory>/analysis/shards/" > "<original_log_directory>/analysis/search_results/test_failures_search.jsonl" |
There was a problem hiding this comment.
one question I have is how do we make these files more readable for humans. I would want to request comments that explain what this is doing so that a human can improve on it in the future, but I'm assuming if we put comments here they will be ingested by the LLM, and potentially take some context needlessly, or worse confuse it?
There was a problem hiding this comment.
Yeah, I intentionally didn't leave comments on what these commands are specifically doing. It ends up looking like a regurgitation of the rg docs, which generally isn't very helpful to claude, and it introduces content duplication.
Is it just these explicit rg commands that you find unreadable, or do you think the document in general suffers from poor readability? IMO the document itself ought to be digestible (and if it isn't, then this needs more work), but it's not the end of the world if a specific command like this is tailored for an agent. While not as easily accessible as a comment, it's also not too hard to just ask claude to summarize what such a command is doing, if an engineer wants to understand specifics.
There was a problem hiding this comment.
that's a good point actually. Makes it even more explicit now that comments should almost exclusively be targetted for expressing the author's intentions. "explanation" docs can be generated automatically by llms at read time.
You might need to close adn reopen the PR to get these checks to run. Or maybe you can just "merge when ready" and it'll send it automatically to the merge queue. Not sure...
|
|
The latest Buf updates on your PR. Results from workflow Buf Proto / buf (pull_request).
|
Makes some small improvements to the batch preprocessing claude command: