|
| 1 | +# Scenario Upload and Agent Creation Feature |
| 2 | + |
| 3 | +## Overview |
| 4 | + |
| 5 | +The `/api/upload_scenarios` endpoint allows users to upload JSON files containing scenario definitions and automatically create agents in Azure AI Foundry. This feature includes comprehensive RAI (Responsible AI) validation to ensure all content meets safety guidelines. |
| 6 | + |
| 7 | +## API Endpoint |
| 8 | + |
| 9 | +### POST `/api/upload_scenarios` |
| 10 | + |
| 11 | +Upload scenario data and create agents in Azure AI Foundry. |
| 12 | + |
| 13 | +**Headers:** |
| 14 | +- `user_principal_id`: User ID extracted from authentication header (required) |
| 15 | + |
| 16 | +**Parameters:** |
| 17 | +- `file`: JSON file containing scenario data (required) |
| 18 | + |
| 19 | +**Response Codes:** |
| 20 | +- `200`: Scenarios processed and agents created successfully |
| 21 | +- `400`: Invalid request, file format, or RAI validation failure |
| 22 | +- `401`: Missing or invalid user information |
| 23 | +- `500`: Internal server error |
| 24 | + |
| 25 | +## JSON Structure |
| 26 | + |
| 27 | +The uploaded JSON file must follow this structure: |
| 28 | + |
| 29 | +```json |
| 30 | +{ |
| 31 | + "scenarios": [ |
| 32 | + { |
| 33 | + "name": "Scenario Name", |
| 34 | + "description": "Scenario description", |
| 35 | + "agents": [ |
| 36 | + { |
| 37 | + "name": "Agent Name", |
| 38 | + "description": "Agent description", |
| 39 | + "instructions": "Detailed agent instructions", |
| 40 | + "deployment_name": "model-deployment-name", |
| 41 | + "type": "agent_type", |
| 42 | + "tools": [ |
| 43 | + { |
| 44 | + "type": "function", |
| 45 | + "function": { |
| 46 | + "name": "tool_name", |
| 47 | + "description": "Tool description" |
| 48 | + } |
| 49 | + } |
| 50 | + ] |
| 51 | + } |
| 52 | + ] |
| 53 | + } |
| 54 | + ] |
| 55 | +} |
| 56 | +``` |
| 57 | + |
| 58 | +### Required Fields |
| 59 | + |
| 60 | +**Scenario Level:** |
| 61 | +- `name`: Scenario name |
| 62 | +- `description`: Scenario description |
| 63 | +- `agents`: Array of agent definitions |
| 64 | + |
| 65 | +**Agent Level:** |
| 66 | +- `name`: Agent name (must be unique) |
| 67 | +- `description`: Agent description |
| 68 | +- `instructions`: Agent system instructions |
| 69 | +- `deployment_name`: Model deployment name in Azure AI Foundry |
| 70 | +- `type`: Agent type identifier |
| 71 | + |
| 72 | +**Optional Fields:** |
| 73 | +- `tools`: Array of tool definitions for the agent |
| 74 | + |
| 75 | +## Validation Process |
| 76 | + |
| 77 | +The endpoint performs the following validation steps: |
| 78 | + |
| 79 | +### 1. File Validation |
| 80 | +- Verifies file is provided |
| 81 | +- Checks file has `.json` extension |
| 82 | +- Validates JSON syntax |
| 83 | + |
| 84 | +### 2. Structure Validation |
| 85 | +- Ensures JSON contains valid object |
| 86 | +- Validates presence of `scenarios` array |
| 87 | +- Checks that at least one scenario is provided |
| 88 | + |
| 89 | +### 3. RAI (Responsible AI) Validation |
| 90 | +- Validates all description and instruction fields |
| 91 | +- Rejects content containing: |
| 92 | + - Harmful or dangerous instructions |
| 93 | + - Biased or discriminatory content |
| 94 | + - Inappropriate or offensive material |
| 95 | + - Content violating ethical AI principles |
| 96 | + |
| 97 | +### 4. Agent Creation |
| 98 | +- Extracts agent configurations from scenarios |
| 99 | +- Creates agents in Azure AI Foundry using the AI Project client |
| 100 | +- Tracks success and failure counts |
| 101 | + |
| 102 | +## Response Format |
| 103 | + |
| 104 | +### Success Response |
| 105 | +```json |
| 106 | +{ |
| 107 | + "status": "success", |
| 108 | + "message": "Successfully created all 5 agents in Azure AI Foundry", |
| 109 | + "results": { |
| 110 | + "total_agents": 5, |
| 111 | + "created_count": 5, |
| 112 | + "failed_count": 0, |
| 113 | + "created_agents": [ |
| 114 | + { |
| 115 | + "name": "Agent Name", |
| 116 | + "agent_id": "agent-id-in-foundry", |
| 117 | + "scenario": "Scenario Name", |
| 118 | + "deployment_name": "gpt-4o" |
| 119 | + } |
| 120 | + ], |
| 121 | + "failed_agents": [] |
| 122 | + } |
| 123 | +} |
| 124 | +``` |
| 125 | + |
| 126 | +### Partial Success Response |
| 127 | +```json |
| 128 | +{ |
| 129 | + "status": "partial_success", |
| 130 | + "message": "Created 3 agents successfully, but 2 failed", |
| 131 | + "results": { |
| 132 | + "total_agents": 5, |
| 133 | + "created_count": 3, |
| 134 | + "failed_count": 2, |
| 135 | + "created_agents": [...], |
| 136 | + "failed_agents": [ |
| 137 | + { |
| 138 | + "name": "Failed Agent", |
| 139 | + "scenario": "Scenario Name", |
| 140 | + "error": "Error message" |
| 141 | + } |
| 142 | + ] |
| 143 | + } |
| 144 | +} |
| 145 | +``` |
| 146 | + |
| 147 | +### Error Response |
| 148 | +```json |
| 149 | +{ |
| 150 | + "detail": "RAI validation failed: Content contains inappropriate material" |
| 151 | +} |
| 152 | +``` |
| 153 | + |
| 154 | +## Implementation Details |
| 155 | + |
| 156 | +### Services |
| 157 | + |
| 158 | +**FoundryAgentService** (`src/backend/services/foundry_agent_service.py`) |
| 159 | +- Handles scenario processing and agent creation |
| 160 | +- Integrates with Azure AI Foundry API |
| 161 | +- Manages RAI validation workflow |
| 162 | + |
| 163 | +### Key Methods |
| 164 | + |
| 165 | +1. **`validate_scenario_descriptions()`** |
| 166 | + - Uses existing RAI validation infrastructure |
| 167 | + - Validates all text content in scenarios |
| 168 | + |
| 169 | +2. **`extract_scenarios_and_agents()`** |
| 170 | + - Parses JSON structure |
| 171 | + - Extracts agent configurations |
| 172 | + - Validates required fields |
| 173 | + |
| 174 | +3. **`create_foundry_agent()`** |
| 175 | + - Creates individual agents in Azure AI Foundry |
| 176 | + - Handles API errors gracefully |
| 177 | + |
| 178 | +4. **`create_agents_from_scenarios()`** |
| 179 | + - Orchestrates the complete workflow |
| 180 | + - Returns comprehensive results |
| 181 | + |
| 182 | +### Error Handling |
| 183 | + |
| 184 | +- **RAI Failures**: Complete rejection of file processing |
| 185 | +- **Individual Agent Failures**: Partial success with detailed error reporting |
| 186 | +- **API Errors**: Graceful handling with retry logic where appropriate |
| 187 | +- **Validation Errors**: Clear error messages for structure issues |
| 188 | + |
| 189 | +## Usage Examples |
| 190 | + |
| 191 | +### Example 1: Customer Support Scenario |
| 192 | +```json |
| 193 | +{ |
| 194 | + "scenarios": [ |
| 195 | + { |
| 196 | + "name": "Customer Support Automation", |
| 197 | + "description": "Multi-agent customer support system", |
| 198 | + "agents": [ |
| 199 | + { |
| 200 | + "name": "Triage Agent", |
| 201 | + "description": "Routes customer inquiries to appropriate specialists", |
| 202 | + "instructions": "Categorize and route customer inquiries professionally and efficiently", |
| 203 | + "deployment_name": "gpt-4o", |
| 204 | + "type": "support_triage" |
| 205 | + } |
| 206 | + ] |
| 207 | + } |
| 208 | + ] |
| 209 | +} |
| 210 | +``` |
| 211 | + |
| 212 | +### Example 2: E-commerce Processing |
| 213 | +```json |
| 214 | +{ |
| 215 | + "scenarios": [ |
| 216 | + { |
| 217 | + "name": "Order Processing", |
| 218 | + "description": "Automated order validation and fulfillment", |
| 219 | + "agents": [ |
| 220 | + { |
| 221 | + "name": "Order Validator", |
| 222 | + "description": "Validates incoming orders", |
| 223 | + "instructions": "Review orders for completeness and accuracy", |
| 224 | + "deployment_name": "gpt-35-turbo", |
| 225 | + "type": "order_validation", |
| 226 | + "tools": [ |
| 227 | + { |
| 228 | + "type": "function", |
| 229 | + "function": { |
| 230 | + "name": "validate_payment", |
| 231 | + "description": "Validate payment information" |
| 232 | + } |
| 233 | + } |
| 234 | + ] |
| 235 | + } |
| 236 | + ] |
| 237 | + } |
| 238 | + ] |
| 239 | +} |
| 240 | +``` |
| 241 | + |
| 242 | +## Testing |
| 243 | + |
| 244 | +Use the provided test files: |
| 245 | +- `example_scenarios.json`: Valid scenario data for testing success path |
| 246 | +- `test_scenarios_rai_fail.json`: Invalid content for testing RAI validation |
| 247 | +- `test_scenario_processing.py`: Test script for local validation |
| 248 | + |
| 249 | +## Security Considerations |
| 250 | + |
| 251 | +1. **Authentication**: Requires valid user authentication |
| 252 | +2. **RAI Validation**: Mandatory content safety checks |
| 253 | +3. **Input Validation**: Comprehensive structure and format validation |
| 254 | +4. **Error Handling**: No sensitive information exposed in error messages |
| 255 | +5. **Audit Trail**: All operations are logged and tracked |
| 256 | + |
| 257 | +## Future Enhancements |
| 258 | + |
| 259 | +1. **Batch Processing**: Support for larger scenario uploads |
| 260 | +2. **Template Library**: Pre-built scenario templates |
| 261 | +3. **Agent Versioning**: Version management for created agents |
| 262 | +4. **Advanced Tools**: Support for more complex tool definitions |
| 263 | +5. **Integration Testing**: Automated testing of created agents |
0 commit comments