Re-enable passing chat template to model

[engelmi]

Fixes: #1202

Re-enable passing the chat template to running/serving the model as the issue where using the template lowers the quality of the output seems to be solved.

Summary by Sourcery

Enhancements:

• Restore chat template retrieval and injection in the llama_serve execution arguments

[sourcery-ai]

Reviewer's Guide

Restored the logic to pass a chat template file to the Llama serving command by uncommenting and integrating the existing code path.

Sequence diagram for passing chat template to model serving

sequenceDiagram
    participant User
    participant Model
    participant LlamaServeProcess
    User->>Model: Call llama_serve(args)
    Model->>Model: _get_chat_template_path(args.container, args.generate, args.dryrun)
    alt chat_template_path is not None
        Model->>LlamaServeProcess: Start with --chat-template-file <path>
    else chat_template_path is None
        Model->>LlamaServeProcess: Start without --chat-template-file
    end

Loading

Class diagram for llama_serve method update

classDiagram
    class Model {
        llama_serve(args)
        _get_chat_template_path(container, generate, dryrun)
    }
    Model : +llama_serve(args)
    Model : +_get_chat_template_path(container, generate, dryrun)
    Model : llama_serve now calls _get_chat_template_path and may add --chat-template-file to exec_args

Loading

File-Level Changes

Change	Details	Files
Re-enable chat template parameter for model serving	Removed the TODO and commented-out logic blocking template usage Uncommented and reactivated _get_chat_template_path invocation Added condition to append --chat-template-file to exec_args when a template path is found	ramalama/model.py

Possibly linked issues

• chat template file from model causes failure #1202: The PR re-enables passing the chat template, which was the cause of the failures described in issue 1202, as the problem is now resolved.
• chat template file from model causes failure #1202: The PR re-enables chat templates, addressing a specific aspect of the original issue 1202.
• chat template file from model causes failure #1202: The PR re-enables passing the chat template to the model, which fixes the issue with the jinja flag not being enabled for custom chat formats.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[gemini-code-assist]

Summary of Changes

Hello @engelmi, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request restores a previously disabled feature that allows passing a chat template to the model serving process. The re-enablement is based on the resolution of an earlier issue where the template negatively impacted model output quality, aiming to improve or stabilize model behavior with templates.

Highlights

• Feature Re-enablement: Re-enabled the functionality to pass a chat template file to the model serving process. This was previously disabled due to an issue where using the template lowered output quality, which is now reported as resolved.
• Issue Resolution: This change directly addresses and resolves the issue tracked in https://github.com/containers/ramalama/issues/1202 by restoring the chat template functionality.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments or fill out our survey to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[sourcery-ai]

Hey @engelmi - I've reviewed your changes and they look great!

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[gemini-code-assist]

Code Review

This pull request re-enables passing a chat template to the model. The change is simple and just uncomments a few lines of code. However, I've found a potential issue in the re-enabled code related to handling dry_run mode, which could pass an invalid empty string as a file path. I've provided a suggestion to make the check more robust.

[rhatdan]

LGTM