-
Notifications
You must be signed in to change notification settings - Fork 6.5k
feat: Add support for Bedrock Guardrails streamProcessingMode #20150
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Add support for Bedrock Guardrails streamProcessingMode #20150
Conversation
| not in [ | ||
| "tools", | ||
| "guardrail_identifier", | ||
| "guardrail_version", | ||
| "trace", | ||
| "thinking", | ||
| ] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't feel that there was a need to add guardrail_stream_processing_mode here, as it's not pulled from kwargs.
To be fair, I'm also not too sure why guardrail_identifier and guardrail_version were put here either
|
Not too sure about the failed prettier linting 😅 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
|
For linting, I advise running: uv pip install pre-commit
pre-commit install
pre-commit run -a
git add .
git commit -m "ci: lint" # linting is now applied at commit time by pre-commit hooks
git push <your-origin> <your-branch> |
4d6bc86 to
e2cc73a
Compare
|
@AstraBert Thanks for the review! I just rebased the branch & bumped the version to Regarding the linting, it looked good on my end hence why I was confused about the "prettier" failure 😅 edit: Looks like it's passing now anyways 🤷♂️ |
|
Yeah, most of the times you just need to update your branch to make these "unexplainable" linting problems go away :) |
Description
By default, when using Bedrock Guardrails alongside ConverseStream requests, the guardrails operate in "Synchronous mode", which may or may not be desired.
This PR adds a new attribute,
guardrail_stream_processing_mode, toBedrockConverse, allowing users to configure the streaming response behavior when using guardrails in a streaming fashion.Supporting Documentation:
Related:
Version Bump?
Did I bump the version in the
pyproject.tomlfile of the package I am updating? (Except for thellama-index-corepackage)Type of Change
Please delete options that are not relevant.
How Has This Been Tested?
Your pull-request will likely not be merged unless it is covered by some form of impactful unit testing.
Suggested Checklist:
uv run make format; uv run make lintto appease the lint gods