Skip to content

Configuring llama.cpp server

Rickard Edén edited this page Oct 5, 2024 · 4 revisions

Llama.cpp works works simply using the server:

  1. Start the llama.cpp server with the model and settings you would like to use with server -m ...
  2. Open llm_config.yaml in the /tale folder and change the following fields:
  • URL: "http://127.0.0.1:8080" or wherever you server is running

or if you want to use OAI like interface(legacy):

  1. Start the llama.cpp server with the model and settings you would like to use with server -m ...
  2. Start an 'OAI-like' server interface, ie run python examples/server/api_like_OAI.py
  3. Open llm_config.yaml in the /tale folder and change the following fields:
  • URL: "http://127.0.0.1:8081" or wherever you server is running
  • ENDPOINT: "/v1/chat/completions"
  • BACKEND: "openai"

The game will be using OPENAI_BODY if you want to change any message settings, not DEFAULT_BODY

Clone this wiki locally