Skip to content

Provide updated documentaion to show users alternative deployment model using base-slim.yaml #373

@petecheslock

Description

@petecheslock

Component

Quickstart

Describe the bug

From a conversation with a new user:

Trying to mess around with llm-d following the instructions here: https://github.com/llm-d/llm-d-deployer/tree/main/quickstart and I noticed that my instruct-decode pod was in crashloopbackoff. Checking the logs of the pod, it says I need to accept the end user agreement on hugging face. This is mentioned in the docs as well. So I go here: https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct and look through what I need to do.
It wants me to fill out my particulars like name etc. and says it will be shared with the developers.
It is also asking for my date of birth.

We already call out that the user will need to accept terms for whatever model, and the user in this case found the base-slim.yaml to use instead via ./llmd-installer.sh --values-file ./examples/base/slim/base-slim.yaml

I want to update the quickstart to better call out this alternative deployment option

Steps to reproduce

Following the default quickstart on the README https://github.com/llm-d/llm-d-deployer/tree/main/quickstart

Additional context or screenshots

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    quickstartRelated to quickstarts

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions