Enhancement: Support for Latest Llama 3.3 70b on AWS Bedrock

[dirkpetersen]

What features would you like to see added?

I am getting this error

LibreChat-API     | 2024-12-21T23:36:08.073Z error: [api/server/controllers/agents/client.js #sendCompletion] Unhandled error type Invocation of model ID meta.llama3-3-70b-instruct-v1:0 with on-demand t... [truncated]
LibreChat-API     | 2024-12-21T23:36:08.074Z error: [handleAbortError] AI response error; aborting request: Invocation of model ID meta.llama3-3-70b-instruct-v1:0 with on-demand throughput isn’t support... [truncated

More details

Here is the latest list of supported Bedrock models :

amazon.titan-tg1-large
amazon.titan-embed-g1-text-02
amazon.titan-text-lite-v1:0:4k
amazon.titan-text-lite-v1
amazon.titan-text-express-v1:0:8k
amazon.titan-text-express-v1
amazon.nova-pro-v1:0
amazon.nova-lite-v1:0
amazon.nova-micro-v1:0
amazon.titan-embed-text-v1:2:8k
amazon.titan-embed-text-v1
amazon.titan-embed-text-v2:0
amazon.titan-embed-image-v1:0
amazon.titan-embed-image-v1
amazon.titan-image-generator-v1:0
amazon.titan-image-generator-v1
amazon.titan-image-generator-v2:0
amazon.rerank-v1:0
stability.stable-diffusion-xl-v1:0
stability.stable-diffusion-xl-v1
stability.sd3-large-v1:0
stability.sd3-5-large-v1:0
stability.stable-image-core-v1:0
stability.stable-image-core-v1:1
stability.stable-image-ultra-v1:0
stability.stable-image-ultra-v1:1
anthropic.claude-3-5-sonnet-20241022-v2:0:18k
anthropic.claude-3-5-sonnet-20241022-v2:0:51k
anthropic.claude-3-5-sonnet-20241022-v2:0:200k
anthropic.claude-3-5-sonnet-20241022-v2:0
anthropic.claude-3-5-haiku-20241022-v1:0
anthropic.claude-instant-v1:2:100k
anthropic.claude-instant-v1
anthropic.claude-v2:0:18k
anthropic.claude-v2:0:100k
anthropic.claude-v2:1:18k
anthropic.claude-v2:1:200k
anthropic.claude-v2:1
anthropic.claude-v2
anthropic.claude-3-sonnet-20240229-v1:0:28k
anthropic.claude-3-sonnet-20240229-v1:0:200k
anthropic.claude-3-sonnet-20240229-v1:0
anthropic.claude-3-haiku-20240307-v1:0:48k
anthropic.claude-3-haiku-20240307-v1:0:200k
anthropic.claude-3-haiku-20240307-v1:0
anthropic.claude-3-opus-20240229-v1:0:12k
anthropic.claude-3-opus-20240229-v1:0:28k
anthropic.claude-3-opus-20240229-v1:0:200k
anthropic.claude-3-opus-20240229-v1:0
anthropic.claude-3-5-sonnet-20240620-v1:0:18k
anthropic.claude-3-5-sonnet-20240620-v1:0:51k
anthropic.claude-3-5-sonnet-20240620-v1:0:200k
anthropic.claude-3-5-sonnet-20240620-v1:0
cohere.command-text-v14:7:4k
cohere.command-text-v14
cohere.command-r-v1:0
cohere.command-r-plus-v1:0
cohere.command-light-text-v14:7:4k
cohere.command-light-text-v14
cohere.embed-english-v3:0:512
cohere.embed-english-v3
cohere.embed-multilingual-v3:0:512
cohere.embed-multilingual-v3
cohere.rerank-v3-5:0
meta.llama3-8b-instruct-v1:0
meta.llama3-70b-instruct-v1:0
meta.llama3-1-8b-instruct-v1:0:128k
meta.llama3-1-8b-instruct-v1:0
meta.llama3-1-70b-instruct-v1:0:128k
meta.llama3-1-70b-instruct-v1:0
meta.llama3-1-405b-instruct-v1:0
meta.llama3-2-11b-instruct-v1:0
meta.llama3-2-90b-instruct-v1:0
meta.llama3-2-1b-instruct-v1:0
meta.llama3-2-3b-instruct-v1:0
meta.llama3-3-70b-instruct-v1:0
mistral.mistral-7b-instruct-v0:2
mistral.mixtral-8x7b-instruct-v0:1
mistral.mistral-large-2402-v1:0
mistral.mistral-large-2407-v1:0

Which components are impacted by your request?

No response

Pictures

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

See: aws-samples/bedrock-chat#508 (comment)

If in the US, you need to use us.meta.llama3-3-70b-instruct-v1:0