forked from irmen/Tale
-
-
Notifications
You must be signed in to change notification settings - Fork 7
Configuring llama.cpp server
Rickard Edén edited this page Oct 5, 2024
·
4 revisions
Llama.cpp works works simply using the server:
- Start the llama.cpp server with the model and settings you would like to use with
server -m ...
- Open
llm_config.yaml
in the /tale folder and change the following fields:
-
URL: "http://127.0.0.1:8080"
or wherever you server is running
or if you want to use OAI like interface(legacy):
- Start the llama.cpp server with the model and settings you would like to use with
server -m ...
- Start an 'OAI-like' server interface, ie run
python examples/server/api_like_OAI.py
- Open
llm_config.yaml
in the /tale folder and change the following fields:
-
URL: "http://127.0.0.1:8081"
or wherever you server is running ENDPOINT: "/v1/chat/completions"
BACKEND: "openai"
The game will be using OPENAI_BODY
if you want to change any message settings, not DEFAULT_BODY