Skip to content

Commit c27dc19

Browse files
aantnaantn
authored
add docs on modellist (#813)
1 parent 63d95d9 commit c27dc19

File tree

6 files changed

+233
-4
lines changed

6 files changed

+233
-4
lines changed

‎CLAUDE.md‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -322,3 +322,18 @@ When asked about content from the HolmesGPT documentation website (https://robus
322322
- Kubernetes deployment: `docs/installation/kubernetes-installation.md`
323323
- Toolset documentation: `docs/data-sources/builtin-toolsets/`
324324
- API reference: `docs/reference/`
325+
326+
## MkDocs Formatting Notes
327+
328+
When writing documentation in the `docs/` directory:
329+
- **Lists after headers**: Always add a blank line between a header/bold text and a list, otherwise MkDocs won't render the list properly
330+
```markdown
331+
**Good:**
332+
333+
- item 1
334+
- item 2
335+
336+
**Bad:**
337+
- item 1
338+
- item 2
339+
```

‎docs/ai-providers/index.md‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,13 +12,18 @@ HolmesGPT supports multiple AI providers, giving you flexibility in choosing the
1212
- [:simple-ollama:{ .lg .middle } **Ollama**](ollama.md)
1313
- [:simple-openai:{ .lg .middle } **OpenAI**](openai.md)
1414
- [:material-api:{ .lg .middle } **OpenAI-Compatible**](openai-compatible.md)
15+
- [:material-layers-triple:{ .lg .middle } **Using Multiple Providers**](using-multiple-providers.md)
1516

1617
</div>
1718

1819
## Quick Start
1920

2021
!!! tip "Recommended for New Users"
21-
**OpenAI GPT-4o** provides the best balance of accuracy and speed. Get started with:
22+
**OpenAI models** provide a good balance of accuracy and speed.
23+
24+
**Anthropic models** often give better results at the expense of speed.
25+
26+
To get started with an OpenAI model:
2227

2328
1. Get an [OpenAI API key](https://platform.openai.com/api-keys){:target="_blank"}
2429
2. Set `export OPENAI_API_KEY="your-api-key"`

‎docs/ai-providers/using-multiple-providers.md‎

Lines changed: 154 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,154 @@
1+
# Using Multiple Providers
2+
3+
Configure multiple AI providers to give users flexibility in choosing models through the Robusta UI.
4+
5+
!!! note "UI Feature Only"
6+
Multiple provider configuration currently only impacts investigations run from the **Robusta UI**. When multiple providers are defined, users can select which model to use from a dropdown in the UI. This feature does not affect CLI usage or investigations triggered from other sources.
7+
8+
## Configuration
9+
10+
Configure multiple models using the `modelList` parameter in your Helm values, along with the necessary environment variables.
11+
12+
### Step 1: Create the Kubernetes Secret
13+
14+
First, create a secret with your API keys (only include the ones you need):
15+
16+
```bash
17+
# Example with all providers - only include what you're using
18+
kubectl create secret generic holmes-secrets \
19+
--from-literal=openai-api-key="sk-..." \
20+
--from-literal=anthropic-api-key="sk-ant-..." \
21+
--from-literal=azure-api-key="..." \
22+
--from-literal=aws-access-key-id="AKIA..." \
23+
--from-literal=aws-secret-access-key="..." \
24+
-n <namespace>
25+
26+
# Example with just OpenAI and Anthropic
27+
kubectl create secret generic holmes-secrets \
28+
--from-literal=openai-api-key="sk-..." \
29+
--from-literal=anthropic-api-key="sk-ant-..." \
30+
-n <namespace>
31+
```
32+
33+
### Step 2: Configure Helm Values
34+
35+
```yaml
36+
# values.yaml
37+
# Reference only the API keys you created in the secret
38+
additionalEnvVars:
39+
- name: AZURE_API_KEY
40+
valueFrom:
41+
secretKeyRef:
42+
name: holmes-secrets
43+
key: azure-api-key
44+
- name: ANTHROPIC_API_KEY
45+
valueFrom:
46+
secretKeyRef:
47+
name: holmes-secrets
48+
key: anthropic-api-key
49+
- name: AWS_ACCESS_KEY_ID
50+
valueFrom:
51+
secretKeyRef:
52+
name: holmes-secrets
53+
key: aws-access-key-id
54+
- name: AWS_SECRET_ACCESS_KEY
55+
valueFrom:
56+
secretKeyRef:
57+
name: holmes-secrets
58+
key: aws-secret-access-key
59+
- name: OPENAI_API_KEY
60+
valueFrom:
61+
secretKeyRef:
62+
name: holmes-secrets
63+
key: openai-api-key
64+
65+
# Configure the model list using the environment variables
66+
modelList:
67+
# Standard OpenAI
68+
openai-4o:
69+
api_key: "{{ env.OPENAI_API_KEY }}"
70+
model: openai/gpt-4o
71+
temperature: 0
72+
73+
# Azure OpenAI Models
74+
azure-4o:
75+
api_key: "{{ env.AZURE_API_KEY }}"
76+
model: azure/gpt-4o
77+
api_base: https://your-resource.openai.azure.com/
78+
api_version: "2025-01-01-preview"
79+
temperature: 0
80+
81+
azure-gpt-5:
82+
api_key: "{{ env.AZURE_API_KEY }}"
83+
model: azure/gpt-5-chat
84+
api_base: https://your-resource.openai.azure.com/
85+
api_version: "2025-01-01-preview"
86+
temperature: 1 # only 1 is supported for gpt-5 models
87+
88+
# Anthropic Models
89+
claude-sonnet-4:
90+
api_key: "{{ env.ANTHROPIC_API_KEY }}"
91+
model: claude-sonnet-4-20250514
92+
temperature: 1
93+
thinking:
94+
budget_tokens: 10000
95+
type: enabled
96+
97+
# AWS Bedrock
98+
bedrock-claude:
99+
aws_access_key_id: "{{ env.AWS_ACCESS_KEY_ID }}"
100+
aws_region_name: eu-south-2
101+
aws_secret_access_key: "{{ env.AWS_SECRET_ACCESS_KEY }}"
102+
model: bedrock/eu.anthropic.claude-sonnet-4-20250514-v1:0
103+
temperature: 1
104+
thinking:
105+
budget_tokens: 10000
106+
type: enabled
107+
```
108+
109+
110+
## Model Parameters
111+
112+
Each model in `modelList` can accept any parameter supported by LiteLLM for that provider. The `model` parameter is required, while authentication requirements vary by provider. Any additional LiteLLM parameters will be passed directly through to the provider.
113+
114+
### Required Parameter
115+
- `model`: Model identifier (provider-specific format)
116+
117+
### Common Parameters
118+
- `api_key`: API key for authentication where required (can use `{{ env.VAR_NAME }}` syntax)
119+
- `temperature`: Creativity level (0-2, lower is more deterministic)
120+
121+
### Additional Parameters
122+
123+
You can pass any LiteLLM-supported parameter for your provider. Examples include:
124+
125+
- **Azure**: `api_base`, `api_version`, `deployment_id`
126+
- **Anthropic**: `thinking` (with `budget_tokens` and `type`)
127+
- **AWS Bedrock**: `aws_access_key_id`, `aws_secret_access_key`, `aws_region_name`, `aws_session_token`
128+
- **Google Vertex**: `vertex_project`, `vertex_location`
129+
130+
Refer to [LiteLLM documentation](https://docs.litellm.ai/docs/providers) for the complete list of parameters supported by each provider.
131+
132+
## User Experience
133+
134+
When multiple models are configured:
135+
136+
1. Users see a **model selector dropdown** in the Robusta UI
137+
2. Each model appears with its configured name (e.g., "azure-4o", "claude-sonnet-4")
138+
3. Users can switch between models for different investigations
139+
140+
## Best Practices
141+
142+
1. **Use descriptive names**: Name models clearly (e.g., `fast-gpt4`, `accurate-claude`, `budget-mini`)
143+
2. **Secure API keys**: Always use Kubernetes secrets for API keys
144+
145+
## Limitations
146+
147+
- **UI Only**: Model selection currently only works in the Robusta UI
148+
- **No automatic failover**: If a selected model fails, users must manually switch
149+
150+
## See Also
151+
152+
- [UI Installation](../installation/ui-installation.md)
153+
- [Helm Configuration](../reference/helm-configuration.md)
154+
- Individual provider documentation for specific configuration details

‎docs/installation/ui-installation.md‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -144,8 +144,11 @@ The fastest way to use HolmesGPT is via the managed Robusta SaaS platform.
144144
2. **Connect your cluster:** Follow the in-app wizard to install the Robusta agent and configure data sources.
145145
3. **Investigate:** Use the "Ask Holmes" chat to analyze alerts or ask questions like:
146146

147-
1. “What pods are failing in production?”
148-
2. “Why did this alert fire?”
147+
1. "What pods are failing in production?"
148+
2. "Why did this alert fire?"
149+
150+
!!! tip "Multiple AI Providers"
151+
You can configure multiple AI models for users to choose from in the UI. See [Using Multiple Providers](../ai-providers/using-multiple-providers.md) for configuration details.
149152

150153
---
151154

‎docs/reference/helm-configuration.md‎

Lines changed: 52 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,7 @@ toolsets:
5353
|-----------|-------------|---------|
5454
| `additionalEnvVars` | Environment variables (API keys, etc.) | `[]` |
5555
| `toolsets` | Enable/disable specific toolsets | (see values.yaml) |
56+
| `modelList` | Configure multiple AI models for UI selection. See [Using Multiple Providers](../ai-providers/using-multiple-providers.md) | `{}` |
5657
| `openshift` | Enable OpenShift compatibility mode | `false` |
5758
| `image` | HolmesGPT image name | `holmes:0.0.0` |
5859
| `registry` | Container registry | `robustadev` |
@@ -191,7 +192,8 @@ enableAccountsCreate: true
191192
# MCP servers configuration
192193
mcp_servers: {}
193194

194-
# Model list configuration
195+
# Model list configuration for multiple AI providers (UI only)
196+
# See: https://holmesgpt.dev/ai-providers/using-multiple-providers/
195197
modelList: {}
196198
```
197199
@@ -226,6 +228,55 @@ toolsets:
226228
enabled: false
227229
```
228230
231+
### Multiple AI Providers Setup
232+
233+
```yaml
234+
# values.yaml
235+
additionalEnvVars:
236+
- name: OPENAI_API_KEY
237+
valueFrom:
238+
secretKeyRef:
239+
name: holmes-secrets
240+
key: openai-api-key
241+
- name: ANTHROPIC_API_KEY
242+
valueFrom:
243+
secretKeyRef:
244+
name: holmes-secrets
245+
key: anthropic-api-key
246+
- name: AWS_ACCESS_KEY_ID
247+
valueFrom:
248+
secretKeyRef:
249+
name: holmes-secrets
250+
key: aws-access-key-id
251+
- name: AWS_SECRET_ACCESS_KEY
252+
valueFrom:
253+
secretKeyRef:
254+
name: holmes-secrets
255+
key: aws-secret-access-key
256+
257+
modelList:
258+
gpt-4o:
259+
api_key: "{{ env.OPENAI_API_KEY }}"
260+
model: openai/gpt-4o
261+
temperature: 0
262+
anthropic-sonnet-4:
263+
api_key: "{{ env.ANTHROPIC_API_KEY }}"
264+
model: anthropic/claude-sonnet-4-20250514
265+
temperature: 1
266+
thinking:
267+
budget_tokens: 10000
268+
type: enabled
269+
bedrock-sonnet-4:
270+
aws_access_key_id: "{{ env.AWS_ACCESS_KEY_ID }}"
271+
aws_region_name: us-east-1
272+
aws_secret_access_key: "{{ env.AWS_SECRET_ACCESS_KEY }}"
273+
model: bedrock/anthropic.claude-sonnet-4-20250514-v1:0
274+
temperature: 1
275+
thinking:
276+
budget_tokens: 10000
277+
type: enabled
278+
```
279+
229280
230281
### OpenShift Setup
231282

‎mkdocs.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -149,6 +149,7 @@ nav:
149149
- Ollama: ai-providers/ollama.md
150150
- OpenAI: ai-providers/openai.md
151151
- OpenAI-Compatible: ai-providers/openai-compatible.md
152+
- Using Multiple Providers: ai-providers/using-multiple-providers.md
152153

153154
- Data Sources:
154155
- data-sources/index.md

0 commit comments

Comments
 (0)