Skip to content

Commit c173d4f

Browse files
- Add POST /api/upload_scenarios endpoint for creating agents in Azure AI Foundry
- Add POST /api/upload_team_config endpoint for team configuration management - Implement FoundryAgentService for bulk agent creation with error handling - Add comprehensive RAI validation for all uploaded content - Add team selection UI with search, delete, and validation features - Add model deployment and search index validation services
1 parent fbc261e commit c173d4f

11 files changed

+1379
-40
lines changed

SCENARIO_UPLOAD_DOCUMENTATION.md

Lines changed: 263 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,263 @@
1+
# Scenario Upload and Agent Creation Feature
2+
3+
## Overview
4+
5+
The `/api/upload_scenarios` endpoint allows users to upload JSON files containing scenario definitions and automatically create agents in Azure AI Foundry. This feature includes comprehensive RAI (Responsible AI) validation to ensure all content meets safety guidelines.
6+
7+
## API Endpoint
8+
9+
### POST `/api/upload_scenarios`
10+
11+
Upload scenario data and create agents in Azure AI Foundry.
12+
13+
**Headers:**
14+
- `user_principal_id`: User ID extracted from authentication header (required)
15+
16+
**Parameters:**
17+
- `file`: JSON file containing scenario data (required)
18+
19+
**Response Codes:**
20+
- `200`: Scenarios processed and agents created successfully
21+
- `400`: Invalid request, file format, or RAI validation failure
22+
- `401`: Missing or invalid user information
23+
- `500`: Internal server error
24+
25+
## JSON Structure
26+
27+
The uploaded JSON file must follow this structure:
28+
29+
```json
30+
{
31+
"scenarios": [
32+
{
33+
"name": "Scenario Name",
34+
"description": "Scenario description",
35+
"agents": [
36+
{
37+
"name": "Agent Name",
38+
"description": "Agent description",
39+
"instructions": "Detailed agent instructions",
40+
"deployment_name": "model-deployment-name",
41+
"type": "agent_type",
42+
"tools": [
43+
{
44+
"type": "function",
45+
"function": {
46+
"name": "tool_name",
47+
"description": "Tool description"
48+
}
49+
}
50+
]
51+
}
52+
]
53+
}
54+
]
55+
}
56+
```
57+
58+
### Required Fields
59+
60+
**Scenario Level:**
61+
- `name`: Scenario name
62+
- `description`: Scenario description
63+
- `agents`: Array of agent definitions
64+
65+
**Agent Level:**
66+
- `name`: Agent name (must be unique)
67+
- `description`: Agent description
68+
- `instructions`: Agent system instructions
69+
- `deployment_name`: Model deployment name in Azure AI Foundry
70+
- `type`: Agent type identifier
71+
72+
**Optional Fields:**
73+
- `tools`: Array of tool definitions for the agent
74+
75+
## Validation Process
76+
77+
The endpoint performs the following validation steps:
78+
79+
### 1. File Validation
80+
- Verifies file is provided
81+
- Checks file has `.json` extension
82+
- Validates JSON syntax
83+
84+
### 2. Structure Validation
85+
- Ensures JSON contains valid object
86+
- Validates presence of `scenarios` array
87+
- Checks that at least one scenario is provided
88+
89+
### 3. RAI (Responsible AI) Validation
90+
- Validates all description and instruction fields
91+
- Rejects content containing:
92+
- Harmful or dangerous instructions
93+
- Biased or discriminatory content
94+
- Inappropriate or offensive material
95+
- Content violating ethical AI principles
96+
97+
### 4. Agent Creation
98+
- Extracts agent configurations from scenarios
99+
- Creates agents in Azure AI Foundry using the AI Project client
100+
- Tracks success and failure counts
101+
102+
## Response Format
103+
104+
### Success Response
105+
```json
106+
{
107+
"status": "success",
108+
"message": "Successfully created all 5 agents in Azure AI Foundry",
109+
"results": {
110+
"total_agents": 5,
111+
"created_count": 5,
112+
"failed_count": 0,
113+
"created_agents": [
114+
{
115+
"name": "Agent Name",
116+
"agent_id": "agent-id-in-foundry",
117+
"scenario": "Scenario Name",
118+
"deployment_name": "gpt-4o"
119+
}
120+
],
121+
"failed_agents": []
122+
}
123+
}
124+
```
125+
126+
### Partial Success Response
127+
```json
128+
{
129+
"status": "partial_success",
130+
"message": "Created 3 agents successfully, but 2 failed",
131+
"results": {
132+
"total_agents": 5,
133+
"created_count": 3,
134+
"failed_count": 2,
135+
"created_agents": [...],
136+
"failed_agents": [
137+
{
138+
"name": "Failed Agent",
139+
"scenario": "Scenario Name",
140+
"error": "Error message"
141+
}
142+
]
143+
}
144+
}
145+
```
146+
147+
### Error Response
148+
```json
149+
{
150+
"detail": "RAI validation failed: Content contains inappropriate material"
151+
}
152+
```
153+
154+
## Implementation Details
155+
156+
### Services
157+
158+
**FoundryAgentService** (`src/backend/services/foundry_agent_service.py`)
159+
- Handles scenario processing and agent creation
160+
- Integrates with Azure AI Foundry API
161+
- Manages RAI validation workflow
162+
163+
### Key Methods
164+
165+
1. **`validate_scenario_descriptions()`**
166+
- Uses existing RAI validation infrastructure
167+
- Validates all text content in scenarios
168+
169+
2. **`extract_scenarios_and_agents()`**
170+
- Parses JSON structure
171+
- Extracts agent configurations
172+
- Validates required fields
173+
174+
3. **`create_foundry_agent()`**
175+
- Creates individual agents in Azure AI Foundry
176+
- Handles API errors gracefully
177+
178+
4. **`create_agents_from_scenarios()`**
179+
- Orchestrates the complete workflow
180+
- Returns comprehensive results
181+
182+
### Error Handling
183+
184+
- **RAI Failures**: Complete rejection of file processing
185+
- **Individual Agent Failures**: Partial success with detailed error reporting
186+
- **API Errors**: Graceful handling with retry logic where appropriate
187+
- **Validation Errors**: Clear error messages for structure issues
188+
189+
## Usage Examples
190+
191+
### Example 1: Customer Support Scenario
192+
```json
193+
{
194+
"scenarios": [
195+
{
196+
"name": "Customer Support Automation",
197+
"description": "Multi-agent customer support system",
198+
"agents": [
199+
{
200+
"name": "Triage Agent",
201+
"description": "Routes customer inquiries to appropriate specialists",
202+
"instructions": "Categorize and route customer inquiries professionally and efficiently",
203+
"deployment_name": "gpt-4o",
204+
"type": "support_triage"
205+
}
206+
]
207+
}
208+
]
209+
}
210+
```
211+
212+
### Example 2: E-commerce Processing
213+
```json
214+
{
215+
"scenarios": [
216+
{
217+
"name": "Order Processing",
218+
"description": "Automated order validation and fulfillment",
219+
"agents": [
220+
{
221+
"name": "Order Validator",
222+
"description": "Validates incoming orders",
223+
"instructions": "Review orders for completeness and accuracy",
224+
"deployment_name": "gpt-35-turbo",
225+
"type": "order_validation",
226+
"tools": [
227+
{
228+
"type": "function",
229+
"function": {
230+
"name": "validate_payment",
231+
"description": "Validate payment information"
232+
}
233+
}
234+
]
235+
}
236+
]
237+
}
238+
]
239+
}
240+
```
241+
242+
## Testing
243+
244+
Use the provided test files:
245+
- `example_scenarios.json`: Valid scenario data for testing success path
246+
- `test_scenarios_rai_fail.json`: Invalid content for testing RAI validation
247+
- `test_scenario_processing.py`: Test script for local validation
248+
249+
## Security Considerations
250+
251+
1. **Authentication**: Requires valid user authentication
252+
2. **RAI Validation**: Mandatory content safety checks
253+
3. **Input Validation**: Comprehensive structure and format validation
254+
4. **Error Handling**: No sensitive information exposed in error messages
255+
5. **Audit Trail**: All operations are logged and tracked
256+
257+
## Future Enhancements
258+
259+
1. **Batch Processing**: Support for larger scenario uploads
260+
2. **Template Library**: Pre-built scenario templates
261+
3. **Agent Versioning**: Version management for created agents
262+
4. **Advanced Tools**: Support for more complex tool definitions
263+
5. **Integration Testing**: Automated testing of created agents

0 commit comments

Comments
 (0)