'Everything' regarding the configuration of the LLM, inference settings, prompts, etc are found in 'llm_config.yaml' in the root folder of the project.

There are three types of settings sent to the LLM, depending on what it's supposed to do. These have been decided empirically and there might be more optimal ones depending on your model. DEFAULT_BODY is used in most cases. It is always used for OpenAI. GENERATION_BODY is used when creating things, whether it's items, npcs or locations. Since the advent of JSON grammar, this is less relevant, but kept for now. ANALYSIS_BODY is used when the dialogue is analyzed. It's an old setting, and may also be obsolete now.

Since v0.35.0 USER_START and USER_END are deprecated in favor of using the backend's default chat template. They are still available, however, if your backend doesn't support automatic chat templates. If so, add them and configure them according to the model you want to use. For example:

Alpaca: USER_START: '### Instruction:\n' USER_END: '### Response:\n'

Llama3 requires a bit more:

SYSTEM_START: '<|begin_of_text|><|eot_id|><|start_header_id|>system<|end_header_id|>\n\n'

USER_START: '<|eot_id|><|start_header_id|>user<|end_header_id|>\n\n'

USER_END: <|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n'

PROMPT_END: '<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n'

But the recommend way is to omit these and set up the backend to provide the template.