Skip to content

Commit bb7a9d6

Browse files
siddharthsambharia-portkeysiddharthsambharia-portkey
authored
Merge pull request #391 from Portkey-AI/new-section-in-cookbook
chore/anthropic computer use-fix + evals section
2 parents 12ce33b + 25c9c06 commit bb7a9d6

File tree

5 files changed

+315
-258
lines changed

5 files changed

+315
-258
lines changed

‎docs.json

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -381,6 +381,7 @@
381381
"integrations/ai-apps",
382382
"integrations/libraries/anythingllm",
383383
"integrations/libraries/claude-code",
384+
"integrations/libraries/anthropic-computer-use",
384385
"integrations/libraries/cline",
385386
"integrations/libraries/goose",
386387
"integrations/libraries/janhq",
@@ -788,6 +789,13 @@
788789
{
789790
"tab": "Cookbook",
790791
"groups": [
792+
{
793+
"group": "Evals",
794+
"pages": [
795+
"guides/use-cases/run-batch-evals",
796+
"guides/prompts/llm-as-a-judge"
797+
]
798+
},
791799
{
792800
"group": "Prompt Engineering",
793801
"pages": [
@@ -855,7 +863,6 @@
855863
"pages": [
856864
"guides/use-cases",
857865
"guides/use-cases/librechat-web-search",
858-
"guides/use-cases/run-batch-evals",
859866
"guides/use-cases/few-shot-prompting",
860867
"guides/use-cases/enforcing-json-schema-with-anyscale-and-together",
861868
"guides/use-cases/emotions-with-gpt-4o",

‎integrations/libraries.mdx

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,8 @@ title: "Overview"
1313

1414
<Card title="Claude Code" href="/integrations/libraries/claude-code" />
1515

16+
<Card title="Anthropic Computer Use" href="/integrations/libraries/anthropic-computer-use" />
17+
1618

1719
<Card title="Instructor" href="/integrations/libraries/instructor" />
1820

‎integrations/libraries/anthropic-computer-use.mdx

Lines changed: 296 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,296 @@
1+
---
2+
title: "Anthropic Computer Use"
3+
---
4+
5+
Anthropic computer use is fully supported in Portkey.
6+
For more information on the computer use tool, please refer to the [Anthropic documentation](https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/computer-use-tool).
7+
8+
9+
### Usage
10+
<CodeGroup>
11+
```py Python
12+
from portkey_ai import Portkey
13+
14+
# Initialize the Portkey client
15+
portkey = Portkey(
16+
api_key="PORTKEY_API_KEY", # Replace with your Portkey API key
17+
virtual_key="VIRTUAL_KEY",
18+
strict_open_ai_compliance="false"
19+
20+
21+
)
22+
23+
# Create the request
24+
response = portkey.chat.completions.create(
25+
anthropic_beta="computer-use-2025-01-24",
26+
model="claude-opus-4-20250514",
27+
max_tokens=3000,
28+
thinking={
29+
"type": "enabled",
30+
"budget_tokens": 2030
31+
},
32+
stream=False,
33+
34+
tools=[
35+
{
36+
"type": "computer",
37+
"computer": {
38+
"name": "computer_20250124",
39+
"display_width_px": 1024,
40+
"display_height_px": 768,
41+
"display_number": 1,
42+
}
43+
},
44+
{
45+
"type": "text_editor_20250429",
46+
"name": "str_replace_based_edit_tool"
47+
},
48+
{
49+
"type": "bash_20250124",
50+
"name": "bash"
51+
}
52+
],
53+
messages=[
54+
{
55+
"role": "user",
56+
"content": "Save a picture of a cat to my desktop."
57+
}
58+
]
59+
)
60+
print(response)
61+
```
62+
```ts NodeJS
63+
import Portkey from 'portkey-ai';
64+
65+
// Initialize the Portkey client
66+
const portkey = new Portkey({
67+
apiKey: "PORTKEY_API_KEY", // Replace with your Portkey API key
68+
virtualKey: "VIRTUAL_KEY", // Add your anthropic's virtual key
69+
strictOpenAiCompliance: false
70+
});
71+
72+
// Generate a chat completion
73+
async function getChatCompletionFunctions() {
74+
const response = await portkey.chat.completions.create({
75+
model: "claude-4-opus-20250514",
76+
anthropic_beta: "computer-use-2025-01-24",
77+
max_tokens: 3000,
78+
thinking: {
79+
type: "enabled",
80+
budget_tokens: 2030
81+
},
82+
stream: false,
83+
tools: [
84+
{
85+
type: "computer",
86+
computer: {
87+
name: "computer_20250124", // This is the version of the tool
88+
display_width_px: 1024,
89+
display_height_px: 768,
90+
display_number: 1
91+
}
92+
},
93+
{
94+
type: "text_editor_20250429",
95+
name: "str_replace_based_edit_tool"
96+
},
97+
{
98+
type: "bash_20250124",
99+
name: "bash"
100+
}
101+
],
102+
"messages": [
103+
{
104+
role: "user",
105+
content: "Save a picture of a cat to my desktop."
106+
}
107+
]
108+
});
109+
console.log(response);
110+
}
111+
// Call the function
112+
getChatCompletionFunctions();
113+
```
114+
```sh cURL
115+
curl "https://api.portkey.ai/v1/chat/completions" \
116+
-H "Content-Type: application/json" \
117+
-H "x-portkey-api-key: $PORTKEY_API_KEY" \
118+
-H "x-portkey-provider: anthropic" \
119+
-H "x-api-key: $ANTHROPIC_API_KEY" \
120+
-H "x-portkey-strict-open-ai-compliance: false" \
121+
-d '{
122+
"model": "claude-4-opus-20250514",
123+
"anthropic_beta": "computer-use-2025-01-24",
124+
"max_tokens": 3000,
125+
"thinking": {
126+
"type": "enabled",
127+
"budget_tokens": 2030
128+
},
129+
"stream": false,
130+
"tools": [
131+
{
132+
"type": "computer",
133+
"computer": {
134+
"name": "computer_20250124",
135+
"display_width_px": 1024,
136+
"display_height_px": 768,
137+
"display_number": 1
138+
}
139+
},
140+
{
141+
"type": "text_editor_20250429",
142+
"name": "str_replace_based_edit_tool"
143+
},
144+
{
145+
"type": "bash_20250124",
146+
"name": "bash"
147+
}
148+
],
149+
"messages": [
150+
{
151+
"role": "user",
152+
"content": "Save a picture of a cat to my desktop."
153+
}
154+
]
155+
}'
156+
```
157+
</CodeGroup>
158+
159+
160+
161+
162+
163+
# Portkey Features
164+
Now that you have enterprise-grade Anthropic Computer Use setup, let's explore the comprehensive features Portkey provides to ensure secure, efficient, and cost-effective AI-assisted development.
165+
166+
### 1. Comprehensive Metrics
167+
Using Portkey you can track 40+ key metrics including cost, token usage, response time, and performance across all your LLM providers in real time. Filter these metrics by developer, team, or project using custom metadata.
168+
169+
<Frame>
170+
<img src="/images/integrations/observability.png" width="600"/>
171+
</Frame>
172+
173+
### 2. Advanced Logs
174+
Portkey's logging dashboard provides detailed logs for every request made by Anthropic Computer Use. These logs include:
175+
- Complete request and response tracking
176+
- Code context and generation metrics
177+
- Developer attribution
178+
- Cost breakdown per coding session
179+
180+
<Frame>
181+
<img src="/images/llms/openai/logs.png"></img>
182+
</Frame>
183+
184+
### 3. Unified Access to 250+ LLMs
185+
186+
Easily switch between 250+ LLMs for different coding tasks. Use GPT-4 for complex architecture decisions, Claude for detailed code reviews, or specialized models for specific languages - all through a single interface.
187+
188+
### 4. Advanced Metadata Tracking
189+
Track coding patterns and productivity metrics with custom metadata:
190+
- Language and framework usage
191+
- Code generation vs completion tasks
192+
- Time-of-day productivity patterns
193+
- Project-specific metrics
194+
195+
<Card title="Custom Metadata" icon="tag" href="/docs/product/ai-gateway/metadata">
196+
</Card>
197+
198+
### 5. Enterprise Access Management
199+
200+
<CardGroup cols={2}>
201+
<Card title="Budget Controls" icon="coins" href="/docs/product/ai-gateway/virtual-keys/budget-limits">
202+
Set and manage spending limits per developer or team. Prevent budget overruns with automatic cutoffs.
203+
</Card>
204+
205+
<Card title="Single Sign-On (SSO)" icon="key" href="/docs/product/enterprise-offering/org-management/sso">
206+
Enterprise-grade SSO integration for seamless developer onboarding and offboarding.
207+
</Card>
208+
209+
<Card title="Organization Management" icon="building" href="/docs/product/enterprise-offering/org-management">
210+
Hierarchical structure with teams, projects, and role-based access control for development organizations.
211+
</Card>
212+
213+
<Card title="Access Rules & Audit Logs" icon="shield-check" href="/docs/product/enterprise-offering/access-control-management#audit-logs">
214+
Comprehensive audit logging for security compliance and code generation tracking.
215+
</Card>
216+
</CardGroup>
217+
218+
### 6. Reliability Features
219+
<CardGroup cols={3}>
220+
<Card title="Fallbacks" icon="life-ring" href="/product/ai-gateway/fallbacks">
221+
Automatically switch between models if one fails, ensuring uninterrupted coding.
222+
</Card>
223+
<Card title="Conditional Routing" icon="route" href="/product/ai-gateway/conditional-routing">
224+
Route requests based on code complexity or language requirements.
225+
</Card>
226+
<Card title="Load Balancing" icon="balance-scale" href="/docs/product/ai-gateway/load-balancing">
227+
Distribute requests across multiple API keys or providers.
228+
</Card>
229+
<Card title="Caching" icon="database" href="/product/ai-gateway/caching">
230+
Cache common code patterns to reduce costs and improve response times.
231+
</Card>
232+
<Card title="Smart Retries" icon="refresh" href="/product/ai-gateway/retries">
233+
Automatic retry handling for failed requests with exponential backoff.
234+
</Card>
235+
<Card title="Budget Limits" icon="shield-check" href="/product/ai-gateway/virtual-keys/budget-limits">
236+
Enforce spending limits to control development costs.
237+
</Card>
238+
</CardGroup>
239+
240+
### 7. Advanced Guardrails
241+
242+
Protect your codebase and enhance security with real-time checks on AI interactions:
243+
- Prevent exposure of API keys and secrets
244+
- Block generation of malicious code patterns
245+
- Enforce coding standards and best practices
246+
- Custom security rules for your organization
247+
- License compliance checks
248+
249+
<Card title="Guardrails" icon="shield-check" href="/docs/product/guardrails">
250+
Implement real-time protection for your development environment with automatic detection and filtering of sensitive code, credentials, and security vulnerabilities.
251+
</Card>
252+
253+
# FAQs
254+
<AccordionGroup>
255+
256+
<Accordion title="How do I track costs per developer?">
257+
Portkey provides several ways to track developer costs:
258+
- Create separate Virtual Keys for each developer
259+
- Use metadata tags to identify developers
260+
- Set up developer-specific API keys
261+
- View detailed analytics in the dashboard
262+
</Accordion>
263+
264+
<Accordion title="What happens if a developer exceeds their budget?">
265+
When a developer reaches their budget limit:
266+
1. Further requests will be blocked
267+
2. The developer and admin receive notifications
268+
3. Coding history remains available
269+
4. Admins can adjust limits as needed
270+
</Accordion>
271+
272+
<Accordion title="Can I use Anthropic Computer Use with local or self-hosted models?">
273+
Yes! Portkey supports local models through Ollama and other self-hosted solutions. Configure your local endpoint as a custom provider in Portkey and use it with Anthropic Computer Use just like any other provider.
274+
</Accordion>
275+
276+
<Accordion title="How do I ensure code security with AI assistance?">
277+
Portkey provides multiple security layers:
278+
- Guardrails to prevent sensitive data exposure
279+
- Request/response filtering
280+
- Audit logs for all interactions
281+
- Custom security rules
282+
- PII detection and masking
283+
</Accordion>
284+
</AccordionGroup>
285+
286+
# Next Steps
287+
288+
**Join our Community**
289+
- [Discord Community](https://portkey.sh/discord-report)
290+
- [GitHub Repository](https://github.com/Portkey-AI)
291+
292+
293+
294+
<Note>
295+
For enterprise support and custom features for your development teams, contact our [enterprise team](https://calendly.com/portkey-ai).
296+
</Note>

‎integrations/libraries/autogen.mdx

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,14 @@
11
---
2-
title: "Autogen"
2+
title: "Autogen (DEPRECATED)"
33
description: "AutoGen is a framework that enables the development of LLM applications using multiple agents that can converse with each other to solve tasks."
44
---
55

6+
7+
<Card title="This Integration is DEPRECATED" icon="robot" href="/integrations/agents/autogen">
8+
Click here to check out the latest integration of Autogen with Portkey
9+
</Card>
10+
11+
612
<Frame>
713
<img src="/images/libraries/libraries-1.png"/>
814
</Frame>

0 commit comments

Comments
 (0)