Docs: add reusable markdown fences for Holmes toolset configuration #815
Conversation
WalkthroughAdds a new MkDocs custom fences module and registers two YAML fence types, then replaces multiple per-method YAML examples in docs with unified fenced blocks (yaml-toolset-config or yaml-helm-values) across several documentation pages. Changes
Sequence Diagram(s)sequenceDiagram
participant Doc as Markdown file
participant MkDocs as MkDocs build
participant SuperFences as pymdownx.superfences
participant Formatter as custom fence formatter
participant Output as Generated HTML
Doc->>MkDocs: contains ```yaml-toolset-config / ```yaml-helm-values
MkDocs->>SuperFences: parse fenced blocks
SuperFences->>Formatter: call appropriate formatter
Formatter->>Formatter: escape YAML, adjust indentation, build tabbed HTML
Formatter-->>SuperFences: return HTML fragment
SuperFences-->>MkDocs: embed fragment
MkDocs-->>Output: finalize page with tabs
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Suggested reviewers
Tip 🔌 Remote MCP (Model Context Protocol) integration is now available!Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats. ✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (6)
docs/custom_fences.py (1)
24-27: Minor DRY: extract YAML indentation helper.Both functions duplicate the “indent by two spaces” logic. Consider a small helper for maintainability. Optional.
+def _indent_yaml_two_spaces(yaml_text: str) -> str: + lines = yaml_text.split("\n") + return "\n".join((" " + ln) if ln else "" for ln in lines) @@ - robusta_yaml_lines = yaml_content.split("\n") - robusta_yaml_indented = "\n".join( - " " + line if line else "" for line in robusta_yaml_lines - ) + robusta_yaml_indented = _indent_yaml_two_spaces(yaml_content) @@ - robusta_yaml_lines = yaml_content.split("\n") - robusta_yaml_indented = "\n".join( - " " + line if line else "" for line in robusta_yaml_lines - ) + robusta_yaml_indented = _indent_yaml_two_spaces(yaml_content)Also applies to: 76-79
docs/data-sources/remote-mcp-servers.md (2)
12-25: yaml-helm-values usage is appropriate here.Good example of mcp_servers; the fence will render Robusta with holmes: nesting automatically. One nit: tiny grammar in the comment.
- # human-readable description of the mcp server (this is not seen by the AI model - its just for users) + # human-readable description of the MCP server (this is not seen by the AI model — it's just for users)
172-172: Minor copy edit (optional).Consider: “After deployment completes, you can use HolmesGPT and ask questions like …” Slightly crisper, but current wording is fine.
docs/data-sources/builtin-toolsets/grafanaloki.md (3)
39-50: Good switch to reusable yaml-toolset-config; add a security hint for api_keyThe example is clear and correctly disables kubernetes/logs. Please add a short comment guiding users not to store secrets in plaintext configs.
toolsets: grafana/loki: enabled: true config: - api_key: <your grafana API key> + # Avoid committing secrets to source control. Prefer env vars or secret management per your deployment. + api_key: <your grafana API key> url: https://xxxxxxx.grafana.net # Your Grafana cloud account URL grafana_datasource_uid: <the UID of the loki data source in Grafana>One more check: In other toolsets (e.g., azuremonitorlogs), “enabled: true” is shown as an example and the toolset is disabled by default. Please confirm the default for grafana/loki and, if similarly disabled by default, consider adding a brief note near this snippet clarifying that “enabled: true” is illustrative and must be set by users. I can propose the wording if desired.
58-69: Direct connection example looks good; consider noting auth header usageThis configuration accurately omits grafana_datasource_uid and shows multitenancy via X-Scope-OrgID. Add a comment to hint how to pass auth when Loki is protected.
url: http://loki.logging headers: X-Scope-OrgID: "<tenant id>" # Set the X-Scope-OrgID if loki multitenancy is enabled + # If your Loki requires authentication, add the appropriate header, e.g.: + # Authorization: "Bearer <token>" kubernetes/logs: enabled: false # HolmesGPT's default logging mechanism MUST be disabled
77-86: Advanced labels config is correct; small clarity improvementThe example correctly nests labels under config. Add a brief comment to make it explicit that shown values reflect the defaults and only need changing if your Loki uses different label names.
url: ... labels: + # Defaults shown; change only if your Loki uses different label keys pod: "pod" namespace: "namespace"
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (9)
docs/custom_fences.py(1 hunks)docs/data-sources/builtin-toolsets/coralogix-logs.md(1 hunks)docs/data-sources/builtin-toolsets/grafanaloki.md(1 hunks)docs/data-sources/builtin-toolsets/kafka.md(1 hunks)docs/data-sources/builtin-toolsets/opensearch-status.md(1 hunks)docs/data-sources/builtin-toolsets/prometheus.md(1 hunks)docs/data-sources/builtin-toolsets/rabbitmq.md(1 hunks)docs/data-sources/remote-mcp-servers.md(2 hunks)mkdocs.yml(1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py
📄 CodeRabbit Inference Engine (CLAUDE.md)
**/*.py: ALWAYS place Python imports at the top of the file, not inside functions or methods
Use Ruff for formatting and linting (configured in pyproject.toml)
Type hints required (mypy configuration in pyproject.toml)
Pre-commit hooks enforce quality checks
Don't add convenience logs that give away the problem
Don't write logs that directly state the issue
Ensure historical timestamps are properly handled in logs (especially with Loki)
Files:
docs/custom_fences.py
🧠 Learnings (4)
📚 Learning: 2025-08-05T00:42:23.792Z
Learnt from: vishiy
PR: robusta-dev/holmesgpt#782
File: config.example.yaml:31-49
Timestamp: 2025-08-05T00:42:23.792Z
Learning: In robusta-dev/holmesgpt config.example.yaml, the azuremonitorlogs toolset configuration shows "enabled: true" as an example of how to enable the toolset, not as a default setting. The toolset is disabled by default and requires explicit enablement in user configurations.
Applied to files:
docs/data-sources/builtin-toolsets/coralogix-logs.mddocs/data-sources/builtin-toolsets/opensearch-status.mddocs/data-sources/builtin-toolsets/rabbitmq.mddocs/data-sources/builtin-toolsets/prometheus.mddocs/data-sources/builtin-toolsets/kafka.mddocs/data-sources/builtin-toolsets/grafanaloki.md
📚 Learning: 2025-08-10T06:02:54.308Z
Learnt from: CR
PR: robusta-dev/holmesgpt#0
File: CLAUDE.md:0-0
Timestamp: 2025-08-10T06:02:54.308Z
Learning: Applies to holmes/plugins/toolsets/**/*.yaml : The only valid top-level fields for toolsets in YAML are: enabled, name, description, additional_instructions, prerequisites, tools, docs_url, icon_url, installation_instructions, config, url (for MCP toolsets only)
Applied to files:
docs/data-sources/builtin-toolsets/coralogix-logs.mddocs/data-sources/builtin-toolsets/opensearch-status.mddocs/data-sources/builtin-toolsets/rabbitmq.mddocs/data-sources/builtin-toolsets/prometheus.mddocs/data-sources/builtin-toolsets/kafka.md
📚 Learning: 2025-08-10T06:02:54.308Z
Learnt from: CR
PR: robusta-dev/holmesgpt#0
File: CLAUDE.md:0-0
Timestamp: 2025-08-10T06:02:54.308Z
Learning: Applies to holmes/plugins/toolsets/**/*.yaml : When configuring toolsets in toolsets.yaml files, ALL toolset-specific configuration must go under a config field
Applied to files:
docs/data-sources/builtin-toolsets/coralogix-logs.mddocs/data-sources/builtin-toolsets/opensearch-status.mddocs/data-sources/builtin-toolsets/rabbitmq.mddocs/data-sources/builtin-toolsets/prometheus.mddocs/data-sources/builtin-toolsets/kafka.md
📚 Learning: 2025-08-10T06:02:54.308Z
Learnt from: CR
PR: robusta-dev/holmesgpt#0
File: CLAUDE.md:0-0
Timestamp: 2025-08-10T06:02:54.308Z
Learning: Applies to holmes/plugins/toolsets/**/*.yaml : Toolsets must be located at holmes/plugins/toolsets/{name}.yaml or {name}/
Applied to files:
docs/data-sources/builtin-toolsets/coralogix-logs.mddocs/data-sources/builtin-toolsets/rabbitmq.mddocs/data-sources/builtin-toolsets/prometheus.mddocs/data-sources/builtin-toolsets/kafka.md
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
- GitHub Check: build (3.12)
- GitHub Check: build (3.11)
- GitHub Check: build (3.10)
- GitHub Check: Pre-commit checks
- GitHub Check: llm_evals
🔇 Additional comments (7)
mkdocs.yml (1)
109-114: ✅ Import path for custom fences verifiedImporting
docs.custom_fencessucceeds in a Python REPL (module loaded fromdocs/custom_fences.py), so no changes are needed.docs/data-sources/builtin-toolsets/rabbitmq.md (1)
12-22: Unified yaml-toolset-config fence looks correct.Structure matches our toolset schema (enabled + config under toolsets.rabbitmq/core). Good consolidation.
docs/data-sources/builtin-toolsets/coralogix-logs.md (1)
19-30: Consolidated config LGTM.
- Correct top-level toolsets mapping.
- Explicitly disables kubernetes/logs to avoid overlap. Clear and consistent.
docs/data-sources/builtin-toolsets/kafka.md (1)
9-27: Kafka toolset block is consistent and valid.Fields are under config.kafka_clusters as expected. Nice cleanup into a single reusable fence.
docs/data-sources/builtin-toolsets/opensearch-status.md (1)
9-27: OpenSearch status config fence reads well.Schema follows toolsets → opensearch/status → enabled/config. Matches our docs conventions.
docs/data-sources/builtin-toolsets/prometheus.md (1)
12-22: Prometheus unified fence is correct.prometheus/metrics under toolsets with config.prometheus_url. Optional headers commented is helpful.
docs/data-sources/remote-mcp-servers.md (1)
165-170: Second MCP example LGTM.Clear, minimal example for Helm values; works well with the new fence.
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (3)
docs/custom_fences.py (3)
25-30: Strip before escaping to normalize whitespace across tabsThis ensures CLI/Helm tabs don’t preserve stray leading/trailing whitespace while Robusta uses stripped content.
- # Escape HTML in the source to prevent XSS - escaped_source = html.escape(source) - - # Strip any leading/trailing whitespace - yaml_content = source.strip() + # Strip any leading/trailing whitespace and escape for XSS + yaml_content = source.strip() + escaped_source = html.escape(yaml_content) @@ - # Escape HTML in the source to prevent XSS - escaped_source = html.escape(source) - - # Strip any leading/trailing whitespace - yaml_content = source.strip() + # Strip any leading/trailing whitespace and escape for XSS + yaml_content = source.strip() + escaped_source = html.escape(yaml_content)Also applies to: 83-88
9-11: Add typing imports and prefer uuid4 import for clarity and mypy/Ruff compliancePer repo guidelines (mypy + Ruff), add explicit typing imports and import uuid4 directly. This also pairs with the signature typing changes below.
-import html -import uuid +import html +from typing import Any, Mapping +from uuid import uuid4
13-17: Type-hint fence formatters and underscore unused params to satisfy mypy/RuffProvide explicit types and underscore unused parameters to silence Ruff’s unused-arg warnings while preserving pymdownx.superfences’ expected callable signature.
-def toolset_config_fence_format(source, language, css_class, options, md, **kwargs): +def toolset_config_fence_format( + source: str, + _language: str, + _css_class: str, + _options: Mapping[str, Any], + _md: Any, + **_kwargs: Any, +) -> str: @@ -def helm_tabs_fence_format(source, language, css_class, options, md, **kwargs): +def helm_tabs_fence_format( + source: str, + _language: str, + _css_class: str, + _options: Mapping[str, Any], + _md: Any, + **_kwargs: Any, +) -> str:Also applies to: 72-76
🧹 Nitpick comments (3)
docs/custom_fences.py (3)
18-24: Good call on unique radio groups; nit: prefer uuid4().hex over str(...).replace("-", "_")uuid4().hex avoids string replacement and yields an ID safe for attribute usage.
- tab_group_id = str(uuid.uuid4()).replace("-", "_") + tab_group_id = uuid4().hex @@ - tab_group_id = str(uuid.uuid4()).replace("-", "_") + tab_group_id = uuid4().hexAlso applies to: 77-82
31-35: Optional: DRY the YAML indentation logic with a tiny helperBoth functions duplicate the same “indent by 2 spaces” logic. A small helper improves readability and consistency.
- robusta_yaml_lines = yaml_content.split("\n") - robusta_yaml_indented = "\n".join( - " " + line if line else "" for line in robusta_yaml_lines - ) + robusta_yaml_indented = indent_yaml(yaml_content, spaces=2) @@ - robusta_yaml_lines = yaml_content.split("\n") - robusta_yaml_indented = "\n".join( - " " + line if line else "" for line in robusta_yaml_lines - ) + robusta_yaml_indented = indent_yaml(yaml_content, spaces=2)Add this helper near the imports:
def indent_yaml(content: str, spaces: int = 2) -> str: prefix = " " * spaces # Keep empty lines empty to avoid trailing spaces in code blocks return "\n".join((prefix + line) if line else "" for line in content.splitlines())Also applies to: 89-93
112-115: Consider makingenableHolmesGPToptional in theyaml-helm-valuesfenceThe
yaml-helm-valuesfence is also used in non-Holmes contexts:
- docs/data-sources/remote-mcp-servers.md:12–15
- docs/data-sources/remote-mcp-servers.md:165–168
Both of these examples simply show MCP server configuration, so automatically injecting
enableHolmesGPT: truecan be misleading.
I recommend refactoring
custom_fences.pyto either:
- Add a flag (e.g.
<!-- holmes=true -->) so the fence only prependsenableHolmesGPTwhen explicitly requested, or- Introduce a separate
yaml-holmes-valuesfence for HolmesGPT-specific snippets.Please adjust the implementation or confirm that always including
enableHolmesGPT: trueis indeed the intended behavior.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (2)
docs/custom_fences.py(1 hunks)mkdocs.yml(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- mkdocs.yml
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py
📄 CodeRabbit Inference Engine (CLAUDE.md)
**/*.py: ALWAYS place Python imports at the top of the file, not inside functions or methods
Type hints are required (project is type-checked with mypy)
Use Ruff for formatting and linting (configured in pyproject.toml)
Files:
docs/custom_fences.py
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: Pre-commit checks
- GitHub Check: llm_evals
- GitHub Check: Pre-commit checks
🔇 Additional comments (1)
docs/custom_fences.py (1)
4-7: MkDocs fence registration confirmedThe
mkdocs.ymlalready registers both custom fences underpymdownx.superfenceswith the correct paths:
yaml-helm-values→docs.custom_fences.helm_tabs_fence_formatyaml-toolset-config→docs.custom_fences.toolset_config_fence_formatNo further changes needed.
2 participants
Before this PR, the docs had to repeat YAML configurations for Holmes multiple times - once for cli, once for helm chart etc.
Now there is a reusable yaml fence to output markdown with several tabs, without repeating the config each time.