[New Plugin] WalledAi #1198
[New Plugin] WalledAi #1198
Conversation
|
DescriptionSummary By MatterAI
🔄 What ChangedThis Pull Request integrates the WalledAI plugin into the Portkey platform, providing guardrail functionality. Key changes include:
🔍 Impact of the ChangeThe changes enhance the WalledAI plugin's reliability by ensuring correct content is always processed for moderation. Improved error handling provides clearer feedback when content is missing. The refactored tests make it easier to understand, maintain, and extend the plugin's test suite, contributing to overall code quality and future development efficiency. 📁 Total Files Changed
🧪 Test AddedThe following integration tests were added/modified to verify the plugin's functionality:
🔒Security VulnerabilitiesNo new security vulnerabilities were detected in this Pull Request. The changes primarily focus on content extraction, error handling, and test improvements, which do not introduce new attack vectors. MotivationIntegrating the WalledAI plugin provides a new guardrail capability for the Portkey platform, addressing issue #1197 by enabling content moderation. Type of Change
How Has This Been Tested?
Screenshots (if applicable)N/A Checklist
Tip Quality Recommendations
Sequence DiagramsequenceDiagram
participant User as "End User"
participant PluginHandler as "WalledAI Plugin Handler"
participant Utils as "Plugin Utils"
participant WalledAIApi as "WalledAI API"
User->>PluginHandler: triggerHook(context, params, eventType)
Note over PluginHandler: beforeRequestHook / afterResponseHook
PluginHandler->>PluginHandler: validateCredentials(parameters.credentials)
alt API Key Missing
PluginHandler-->>User: { error: "apiKey missing", verdict: true, data: null }
else API Key Present
PluginHandler->>+Utils: getCurrentContentPart(context, eventType)
Utils-->>-PluginHandler: { content, textArray }
alt Content is Empty
PluginHandler-->>User: { error: { message: 'request or response json is empty' }, verdict: true, data: null }
else Content Available
PluginHandler->>PluginHandler: prepareText(textArray)
Note over PluginHandler: text = textArray.filter((text) => text).join('
')
PluginHandler->>PluginHandler: prepareRequestBody(text, parameters)
PluginHandler->>PluginHandler: prepareRequestOptions(apiKey)
PluginHandler->>+Utils: post(API_URL, requestBody, requestOptions)
Note over Utils: HTTP POST to https://services.walled.ai/v1/guardrail/moderate
Utils->>+WalledAIApi: POST /v1/guardrail/moderate (requestBody)
WalledAIApi-->>-Utils: API Response (moderationResult)
Utils-->>-PluginHandler: moderationResult
PluginHandler->>PluginHandler: processApiResponse(moderationResult)
PluginHandler-->>User: { verdict, error, data }
end
end
|
![matter-code-review[bot]](https://avatars.githubusercontent.com/in/474549?s=60&v=4){kind=small}
There was a problem hiding this comment.
This PR adds a new WalledAI guardrails plugin for content moderation. The implementation looks solid with good test coverage, but I've identified a few improvements that could enhance error handling and code quality.
Skipped files
conf.json: Skipped file patternplugins/walledai/manifest.json: Skipped file pattern
plugins/walledai/guardrails.ts
Outdated
| console.log(e) | ||
| error = e instanceof Error ? e.message : String(e); | ||
| verdict = true; | ||
| data = null; |
There was a problem hiding this comment.
🛠️ Code Refactor
Issue: The error handling in the catch block uses console.log which is not ideal for production environments and doesn't provide structured logging.
Fix: Replace direct console.log with a more structured error handling approach.
Impact: Improves error traceability and debugging in production environments.
| console.log(e) | |
| error = e instanceof Error ? e.message : String(e); | |
| verdict = true; | |
| data = null; | |
| console.error('WalledAI API error:', e); | |
| error = e instanceof Error ? e.message : String(e); | |
| verdict = true; | |
| data = null; |
plugins/walledai/guardrails.ts
Outdated
| if ( | ||
| data.safety[0]?.isSafe==false | ||
| ) { | ||
| verdict = false; | ||
| } |
There was a problem hiding this comment.
🛠️ Code Refactor
Issue: The condition for checking if content is unsafe is not robust and could be more explicit.
Fix: Add a more comprehensive check that handles potential undefined values and provides clearer intent.
Impact: Prevents potential runtime errors and makes the code more maintainable.
| if ( | |
| data.safety[0]?.isSafe==false | |
| ) { | |
| verdict = false; | |
| } | |
| if ( | |
| data?.safety?.[0]?.isSafe === false | |
| ) { | |
| verdict = false; | |
| } |
| // Prepare request body | ||
| const requestBody = { | ||
| text: text, | ||
| text_type: parameters.text_type || 'prompt', | ||
| generic_safety_check: parameters.generic_safety_check ?? true, | ||
| greetings_list: parameters.greetings_list || ['generalgreetings'], | ||
| }; |
There was a problem hiding this comment.
⚡️ Performance Improvement
Issue: The requestBody object is created with unnecessary property assignments when default values could be used more efficiently.
Fix: Use object spread with defaults to create a more concise and efficient requestBody.
Impact: Improves code readability and slightly optimizes object creation.
| // Prepare request body | |
| const requestBody = { | |
| text: text, | |
| text_type: parameters.text_type || 'prompt', | |
| generic_safety_check: parameters.generic_safety_check ?? true, | |
| greetings_list: parameters.greetings_list || ['generalgreetings'], | |
| }; | |
| // Prepare request body | |
| const requestBody = { | |
| text, | |
| text_type: parameters.text_type || 'prompt', | |
| generic_safety_check: parameters.generic_safety_check ?? true, | |
| greetings_list: parameters.greetings_list || ['generalgreetings'], | |
| ...(parameters.pii_list && { pii_list: parameters.pii_list }), | |
| ...(parameters.compliance_list && { compliance_list: parameters.compliance_list }) | |
| }; |
There was a problem hiding this comment.
code looks good, I've not been able to sign up on https://www.walled.ai/#contactEmailSection
If you're associated with walled.ai and if possible please provide a working API key over discord
https://discord.gg/nqxNXj3W
I'm @segadog in discord
plugins/index.ts
Outdated
| import { handler as panwPrismaAirsintercept } from './panw-prisma-airs/intercept'; | ||
| import { handler as defaultjwt } from './default/jwt'; | ||
| import { handler as defaultrequiredMetadataKeys } from './default/requiredMetadataKeys'; | ||
| import { handler as defaultregexMatch } from "./default/regexMatch" |
There was a problem hiding this comment.
please use the included .prettierrc for formatting, please revert these formatting changes
There was a problem hiding this comment.
Hi @narengogi I added a new commit and used the .prettierrc formatting
|
Important PR Review SkippedPR review skipped as per the configuration setting. Run a manually review by commenting /matter review 💡Tips to use Matter AICommand List
|
| textGuard: pangeatextGuard, | ||
| pii: pangeapii, | ||
| }, | ||
| promptfoo: { |
There was a problem hiding this comment.
please add these back, have you removed them by accident?
There was a problem hiding this comment.
Hi , I was building the plugins using npm run build-plugin command with the provided config.json ,and all the services were not mentioned in the config file , hence when i build it , the other plugins did not get added in the index file . However I have manually added and updated the index file , in my latest commit . This issue with the index files has been solved @narengogi
|
Important PR Review SkippedPR review skipped as per the configuration setting. Run a manually review by commenting /matter review 💡Tips to use Matter AICommand List
|
There was a problem hiding this comment.
LGTM, verified with the key provided
example curl
curl --location 'http://localhost:8787/v1/chat/completions' \
--header 'x-portkey-provider: openai' \
--header 'authorization: sk-' \
--header 'Content-Type: application/json' \
--header 'x-portkey-config: {"afterRequestHooks":[{"type":"guardrail","id":"my_solid_guardrail","checks":[{"id":"walledai.guardrails","parameters":{"credentials":{"apiKey":""}}}]}],"cache":{"mode":"semantic"}}' \
--data '{
"model": "gpt-3.5-turbo",
"max_tokens": 100,
"stream": false,
"messages": [
{
"role": "system",
"content": "You are a saahelpful aaassistant"
},
{
"role": "user",
"content": "hey there"
}
]
}'
|
Important PR Review SkippedPR review skipped as per the configuration setting. Run a manually review by commenting /matter review 💡Tips to use Matter AICommand List
|
|
Important PR Review SkippedPR review skipped as per the configuration setting. Run a manually review by commenting /matter review 💡Tips to use Matter AICommand List
|
|
Important PR Review SkippedPR review skipped as per the configuration setting. Run a manually review by commenting /matter review 💡Tips to use Matter AICommand List
|
|
Important PR Review SkippedPR review skipped as per the configuration setting. Run a manually review by commenting /matter review 💡Tips to use Matter AICommand List
|
4 participants
Description
Integrating walledai plugin, for guardrails on Portkey platform
Issue Link: #1197
Motivation
Type of Change
How Has This Been Tested?
Screenshots (if applicable)
Checklist
Related Issues