Connection errors in production #1363
Description
What Happened?
Hi all,
I'm occasionally running into several errors likely related to connection health in my production traffic. I can't reproduce locally even when I load test with >40 concurrent requests. I have observed these errors with openai, anthropic, and mistral. Curious if others have experienced the same or have any ideas how these could be reproduced/mitigated. Thanks in advance!
Failed to close the writer: anthropic TypeError [ERR_INVALID_STATE]: Invalid state: WritableStream is closed
at writableStreamClose (node:internal/webstreams/writablestream:705:7)
at writableStreamDefaultWriterClose (node:internal/webstreams/writablestream:1082:10)
at WritableStreamDefaultWriter.close (node:internal/webstreams/writablestream:409:12)
at file:///app/portkey-ai-gateway/build/start-server.js:2:359033
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
What Should Have Happened?
Relevant Code Snippet
No response
Your Twitter/LinkedIn
No response