[BUG]: 503 service unavailable

[pgowda1107]

deploy-dynamo-vllm-aggregated.yaml

Dynamo-issue.txt

Describe the Bug

We are rying to do aggregated deployment on vllm backend for Llama3.2-3B model (with 1-2 GPUs). We are getting “503 Service Unavailable” with default configuration tried.
Attaching the issue details along with deployment yaml file used for this deployment.
Followed the example of – [https://github.com/ai-dynamo/dynamo/blob/c9d7d95f4be01e6352c51196ed1858ddcb03fce5/examples/backends/vllm/deploy/agg_router.yaml]

Steps to Reproduce

https://github.com/ai-dynamo/dynamo/blob/c9d7d95f4be01e6352c51196ed1858ddcb03fce5/examples/backends/vllm/deploy/agg_router.yaml

Expected Behavior

Run without issuess

Actual Behavior

Server issue

Environment

DGX H100

Additional Context

No response

Screenshots

No response

[athreesh]

@ganeshku1 @nicolasnoble seems like there's an issue with MX pulling the HF model. can you take a look at the logs?

Error logs from Frontend Pod :
Pod logs show:
2025-12-02T10:20:24.656530Z WARN dynamo_llm::hub: ModelExpress download failed for model '/models/Llama-3.2-3B-Instruct': Failed to fetch model '/models/Llama-3.2-3B-Instruct' from HuggingFace. Is this a valid HuggingFace ID? Error: request error: HTTP status client error (404 Not Found) for url (https://huggingface.co/api/models//models/Llama-3.2-3B-Instruct/revision/main)
2025-12-02T10:20:24.656562Z ERROR dynamo_llm::discovery::watcher: Error adding model from discovery model_name="/models/Llama-3.2-3B-Instruct" namespace="vllm-agg" error="Failed to fetch model '/models/Llama-3.2-3B-Instruct' from HuggingFace. Is this a valid HuggingFace ID? Error: request error: HTTP status client error (404 Not Found) for url (https://huggingface.co/api/models//models/Llama-3.2-3B-Instruct/revision/main)"

[nicolasnoble]

Error 404 is an authentication error usually, meaning the HF credentials may not be properly set? The yaml file above pulls a secret, is it properly set in the namespace you're deploying?

Error 503 is a server-side problem, where it's overloaded or down and should be intermittent.

[pgowda1107]

The expectation is to load the local model path and not from hugging face and the connection is not needed for the same right?