Add multi-language support for prompt templates

[sladinji]

Introduce a centralized Jinja2 prompt system with multi-language capabilities, including English and French templates. Enhance configuration options for prompts directory and language selection, and update documentation to reflect these changes. Refactor existing prompt handling to utilize the new template-based approach.

To fix this vulnerability, autoescaping should be enabled when constructing the Jinja2 Environment. The recommended and most robust solution is to use select_autoescape, which will automatically enable escaping for templates that are commonly rendered as HTML or XML (files ending in .html, .htm, .xml, etc.)—and possibly leave others, like pure text templates, unescaped. However, if you know that all of your templates are either HTML or XML, you can use autoescape=True. To maximize safety and future flexibility, replacing autoescape=False with autoescape=select_autoescape(['html', 'xml', 'j2']) ensures anything with .j2 (or the relevant extensions you use) is escaped. Since your templates use the .j2 extension, including 'j2' in the extension list is prudent. This does not change any application logic and retains existing behavior, with the reliability benefit of escaping.

The only changes needed are:

• Import the select_autoescape method from jinja2 (to allow its use).
• Replace the autoescape=False argument in the Environment instantiation with autoescape=select_autoescape(['j2', 'html', 'xml']).

These changes need to be made in api/utils/prompt_loader.py—specifically, to the import statement and the environment construction.

---

To mitigate path traversal, the user-controlled language value (as well as related inputs derived from it) must be validated before constructing candidate filenames. The best fix, given filenames are supposed to follow the format <lang>.j2 or <module>.<lang>.j2, is to allow only valid language codes (e.g. two-letter alphabetical lowercase strings or an explicitly allow-listed set) for the filename position.

The recommended approach:

• Validate language strictly in the PromptRenderer initializer. Accept only language codes matching a regex such as ^[a-z]{2}$ or a set of known codes.
• If the value does not match, either default to the fallback DEFAULT_LANGUAGE or raise an exception.
• Additionally, in _candidate_templates, ensure that only valid candidate filenames get constructed (derived from a safe language string).
• This validation can be encapsulated in a helper function, and performed immediately on initialization.

Files to edit: api/utils/prompt_loader.py (in the constructor and associated candidate template logic).
Dependencies: For regex, Python's builtin re is sufficient.

---

To mitigate log injection, user-provided values that are included in log messages should be sanitized. For plain-text logs, the most important step is to strip or replace newline and carriage return characters from the language value before logging. The simplest fix is to wrap the self.language reference in a helper that removes problematic characters (e.g., replacing \n and \r with an empty string) in all logging calls where user-provided language codes could end up in log output. Only the logging on line 106 needs to be changed, as subsequent uses do not directly log the value.

The fix should focus only on the code shown in api/utils/prompt_loader.py. A good approach is to use a local variable (e.g., safe_language) constructed by sanitizing self.language just before logging, and using that variable in place of self.language in the log statement.

No new imports are required, as only native string replacement (str.replace) is needed.

---

The best way to fix this problem is to sanitize any user-controlled input that is written to log files. In this specific case, before logging the candidate filenames joined together, each candidate should be sanitized to remove potentially malicious characters such as carriage returns and line feeds (\r, \n). This can be done by mapping a sanitize function over the list of candidate filenames before joining them for logging.

The only file that requires editing is api/utils/prompt_loader.py, specifically in the _resolve_template_for_module method where the log uses ", ".join(candidates). To implement this:

• Define a helper function (locally or as a static/class method) to sanitize user-supplied or user-influenced strings before logging.
• Use this function to clean all candidate strings before joining and logging them.

No external dependencies need to be added; built-in Python string operations are sufficient.

---

To fix this issue, we should instantiate the Jinja2 Environment with autoescape enabled in a way that is correct for what is being templated. If the templates might generate HTML or XML, use select_autoescape(['html', 'xml'])—this is Jinja2's best practice. This approach will enable autoescaping for HTML/XML templates but not for others (like plain .txt files). If all templates are, for example, .j2 files but exclusively render non-HTML, the more precise control is to explicitly enable or disable autoescape based on downstream needs. The safest general fix is to replace autoescape=False with autoescape=select_autoescape(['html', 'xml']).

Therefore, in api/utils/prompt_loader.py, replace the instantiation:

env = Environment(loader=FileSystemLoader([base]), autoescape=False, trim_blocks=True, lstrip_blocks=True)

with

env = Environment(
    loader=FileSystemLoader([base]),
    autoescape=select_autoescape(['html', 'xml']),
    trim_blocks=True,
    lstrip_blocks=True,
)

and add an import for select_autoescape from jinja2 at the top of the file.

To fix the issue, delete the assignment to the unused global variable branch_labels in api/alembic/versions/2025_09_10_1143-095feb42bc54_user_name_optional.py. Make sure not to remove any right-hand side code that has side effects, but since the assignment is simply None, safe to fully remove the line. No other changes are necessary because Alembic does not require this variable to be present unless the migration is part of a branch.

---

To fix the problem, we should remove the definition of the unused global variable depends_on on line 19 of api/alembic/versions/2025_09_10_1143-095feb42bc54_user_name_optional.py. As its value is simply None and there are no side effects associated with its assignment, it's safe to delete this line without impacting any functionality or documentation. No imports or other changes are necessary.