Add ramalama chat command #1531

rhatdan · 2025-06-15T19:57:07Z

For now we will just add the chat command, next PR will remove the external chat command and just use this internal one.

Summary by Sourcery

Introduce a new internal "ramalama chat" CLI command to enable OpenAI chat over REST API and update documentation accordingly.

New Features:

Add "chat" subcommand and associated CLI logic (chat_parser and chat_cli) for interacting with OpenAI via REST API.

Documentation:

Add ramalama-chat(1) man page and integrate it into the docs index.
Update README and main man page to list the new "ramalama chat" command.

For now we will just add the chat command, next PR will remove the external chat command and just use this internal one. Signed-off-by: Daniel J Walsh <[email protected]>

gemini-code-assist · 2025-06-15T19:57:10Z

Important

Installation incomplete: to start using Gemini Code Assist, please ask the organization owner(s) to visit the Gemini Code Assist Admin Console and sign the Terms of Services.

sourcery-ai · 2025-06-15T19:57:19Z

Reviewer's Guide

This PR adds an internal “chat” subcommand by wiring up a new parser and handler in the CLI and brings documentation in sync by creating a dedicated man page and adjusting existing docs to reference the new command and unify option descriptions.

Sequence Diagram for 'ramalama chat' command execution

sequenceDiagram
    actor User
    participant CLI as "ramalama CLI"
    participant Parser as "chat_parser()"
    participant Handler as "chat_cli(args)"
    participant Shell as "RamaLamaShell"
    participant OpenAI_API as "OpenAI REST API (via --url)"

    User->>CLI: Executes `ramalama chat [ARGS]`
    CLI->>Parser: Is called during subcommand configuration
    Parser-->>CLI: Registers 'chat' command with chat_cli as handler
    CLI->>Handler: Invokes chat_cli(args) with parsed arguments
    Handler->>Shell: new RamaLamaShell(args)
    Handler->>Shell: shell.handle_args()
    alt ARGS provided (one-off mode)
        Shell-->>Handler: true (request sent, output shown)
        Handler-->>CLI: return
    else No ARGS (interactive mode)
        Shell-->>Handler: false
        Handler->>Shell: shell.loop()
        loop Interactive Chat
            User->>Shell: Enters prompt
            Shell->>OpenAI_API: Sends request with prompt
            OpenAI_API-->>Shell: Returns response
            Shell->>User: Displays response
        end
        Handler->>Shell: shell.kills() (cleanup)
        Handler-->>CLI: return
    end

Sequence Diagram for 'ramalama chat' command execution

sequenceDiagram
    actor User
    participant CLI as "ramalama CLI"
    participant Parser as "chat_parser()"
    participant Handler as "chat_cli(args)"
    participant Shell as "RamaLamaShell"
    participant OpenAI_API as "OpenAI REST API (via --url)"

    User->>CLI: Executes `ramalama chat [ARGS]`
    CLI->>Parser: Is called during subcommand configuration
    Parser-->>CLI: Registers 'chat' command with chat_cli as handler
    CLI->>Handler: Invokes chat_cli(args) with parsed arguments
    Handler->>Shell: new RamaLamaShell(args)
    Handler->>Shell: shell.handle_args()
    alt ARGS provided (one-off mode)
        Shell-->>Handler: true (request sent, output shown)
        Handler-->>CLI: return
    else No ARGS (interactive mode)
        Shell-->>Handler: false
        Handler->>Shell: shell.loop()
        loop Interactive Chat
            User->>Shell: Enters prompt
            Shell->>OpenAI_API: Sends request with prompt
            OpenAI_API-->>Shell: Returns response
            Shell->>User: Displays response
        end
        Handler->>Shell: shell.kills() (cleanup)
        Handler-->>CLI: return
    end

File-Level Changes

Change	Details	Files
Introduce the internal “chat” CLI subcommand	Add chat_parser with color, prefix, url and ARGS options Register chat_parser in configure_subcommands Implement chat_cli to delegate to RamaLamaShell	`ramalama/cli.py`
Add and link dedicated ramalama-chat man page	Create docs/ramalama-chat.1.md with synopsis, options, examples Insert ramalama-chat entry in README shortnames table Add ramalama-chat to command index in docs/ramalama.1.md	`docs/ramalama-chat.1.md` `README.md` `docs/ramalama.1.md`
Unify capitalization in serve documentation	Capitalize option descriptions for --api, --authfile, --ctx-size in docs	`docs/ramalama-serve.1.md`

Tips and commands

Interacting with Sourcery

Trigger a new review: Comment @sourcery-ai review on the pull request.
Continue discussions: Reply directly to Sourcery's review comments.
Generate a GitHub issue from a review comment: Ask Sourcery to create an
issue from a review comment by replying to it. You can also reply to a
review comment with @sourcery-ai issue to create an issue from it.
Generate a pull request title: Write @sourcery-ai anywhere in the pull
request title to generate a title at any time. You can also comment
@sourcery-ai title on the pull request to (re-)generate the title at any time.
Generate a pull request summary: Write @sourcery-ai summary anywhere in
the pull request body to generate a PR summary at any time exactly where you
want it. You can also comment @sourcery-ai summary on the pull request to
(re-)generate the summary at any time.
Generate reviewer's guide: Comment @sourcery-ai guide on the pull
request to (re-)generate the reviewer's guide at any time.
Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
pull request to resolve all Sourcery comments. Useful if you've already
addressed all the comments and don't want to see them anymore.
Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
request to dismiss all existing Sourcery reviews. Especially useful if you
want to start fresh with a new review - don't forget to comment
@sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

Enable or disable review features such as the Sourcery-generated pull request
summary, the reviewer's guide, and others.
Change the review language.
Add, remove or edit custom review instructions.
Adjust other review settings.

Getting Help

Contact our support team for questions or feedback.
Visit our documentation for detailed guides and information.
Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

sourcery-ai

Hey @rhatdan - I've reviewed your changes - here's some feedback:

The DESCRIPTION in docs/ramalama-chat.1.md still references model removal; please update it to accurately describe the chat functionality.
Your examples treat the REST API URL as a positional argument, but --url is defined as an option; either make URL positional in the parser or update the docs to match.
Consider having chat_cli return an explicit exit code (e.g. from shell.handle_args() or shell.kills()) so scripts can detect failures reliably.

Prompt for AI Agents

Please address the comments from this code review:
## Overall Comments
- The DESCRIPTION in docs/ramalama-chat.1.md still references model removal; please update it to accurately describe the chat functionality.
- Your examples treat the REST API URL as a positional argument, but --url is defined as an option; either make URL positional in the parser or update the docs to match.
- Consider having chat_cli return an explicit exit code (e.g. from shell.handle_args() or shell.kills()) so scripts can detect failures reliably.

## Individual Comments

### Comment 1
<location> `README.md:1044` </location>
<code_context>
 | ------------------------------------------------------ | ---------------------------------------------------------- |
 | [ramalama(1)](https://github.com/containers/ramalama/blob/main/docs/ramalama.1.md)                      | primary RamaLama man page                                  |
 | [ramalama-bench(1)](https://github.com/containers/ramalama/blob/main/docs/ramalama-bench.1.md)| benchmark specified AI Model                                         |
+| [ramalama-chat(1)](https://github.com/containers/ramalama/blob/main/docs/ramalama-chat.1.md)| chat with specified OpenAI RESTAPI                         |
 | [ramalama-containers(1)](https://github.com/containers/ramalama/blob/main/docs/ramalama-containers.1.md)| list all RamaLama containers                               |
 | [ramalama-convert(1)](https://github.com/containers/ramalama/blob/main/docs/ramalama-convert.1.md)      | convert AI Model from local storage to OCI Image           |
</code_context>

<issue_to_address>
Consider using "REST API" instead of "RESTAPI".

This aligns with standard terminology and improves clarity.
</issue_to_address>

<suggested_fix>
<<<<<<< SEARCH
| [ramalama-chat(1)](https://github.com/containers/ramalama/blob/main/docs/ramalama-chat.1.md)| chat with specified OpenAI RESTAPI                         |
=======
| [ramalama-chat(1)](https://github.com/containers/ramalama/blob/main/docs/ramalama-chat.1.md)| chat with specified OpenAI REST API                        |
>>>>>>> REPLACE

</suggested_fix>

### Comment 2
<location> `docs/ramalama-chat.1.md:4` </location>
<code_context>
+% ramalama-chat 1
+
+## NAME
+ramalama\-chat - OpenAI chat with the specified RESTAPI URL
+
+## SYNOPSIS
</code_context>

<issue_to_address>
Consider using "REST API" instead of "RESTAPI" in the NAME section.

This aligns with standard terminology.
</issue_to_address>

<suggested_fix>
<<<<<<< SEARCH
ramalama\-chat - OpenAI chat with the specified RESTAPI URL
=======
ramalama\-chat - OpenAI chat with the specified REST API URL
>>>>>>> REPLACE

</suggested_fix>

### Comment 3
<location> `docs/ramalama-chat.1.md:14` </location>
<code_context>
+                        returned without entering the chatbot
+
+## DESCRIPTION
+Specify one or more AI Models to be removed from local storage
+
+## OPTIONS
</code_context>

<issue_to_address>
The DESCRIPTION section appears to be incorrect for `ramalama-chat`.

Update the DESCRIPTION to accurately reflect that `ramalama-chat` is for chatting via the OpenAI REST API.
</issue_to_address>

### Comment 4
<location> `docs/ramalama-chat.1.md:37` </location>
<code_context>
+$ ramalama chat
+🦭 >
+
+$ ramalama chat http://localhost:1234
+🐋 >
+```
+
</code_context>

<issue_to_address>
The example `ramalama chat http://localhost:1234` and its behavior are unclear.

Please clarify what `http://localhost:1234` represents and how it affects the prompt and chatbot behavior. Also, explain the change in prompt icon. Consider using `--url` in examples for clarity, and ensure the example aligns with the documented `ARGS` usage.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

README.md

docs/ramalama-chat.1.md

sourcery-ai · 2025-06-15T19:59:58Z

docs/ramalama-chat.1.md

+                        returned without entering the chatbot
+
+## DESCRIPTION
+Specify one or more AI Models to be removed from local storage


issue: The DESCRIPTION section appears to be incorrect for ramalama-chat.

Update the DESCRIPTION to accurately reflect that ramalama-chat is for chatting via the OpenAI REST API.

sourcery-ai · 2025-06-15T19:59:58Z

docs/ramalama-chat.1.md

+$ ramalama chat http://localhost:1234
+🐋 >


question: The example ramalama chat http://localhost:1234 and its behavior are unclear.

Please clarify what http://localhost:1234 represents and how it affects the prompt and chatbot behavior. Also, explain the change in prompt icon. Consider using --url in examples for clarity, and ensure the example aligns with the documented ARGS usage.

gemini-code-assist · 2025-06-15T20:02:34Z

Important

Installation incomplete: to start using Gemini Code Assist, please ask the organization owner(s) to visit the Gemini Code Assist Admin Console and sign the Terms of Services.

gemini-code-assist · 2025-06-15T20:02:35Z

Important

Installation incomplete: to start using Gemini Code Assist, please ask the organization owner(s) to visit the Gemini Code Assist Admin Console and sign the Terms of Services.

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> Signed-off-by: Daniel J Walsh <[email protected]>

ericcurtin · 2025-06-15T20:24:31Z

It's a good idea we changed to --url

We probably want the positional arg to be model (maybe optionally) in future when we start supporting model swapping

ericcurtin · 2025-06-15T20:34:52Z

Be interesting if we can make --url work with:

--url unix://somesocketfile

rhatdan · 2025-06-15T20:35:11Z

Yes I agree. Next step is to remove libexec/ramalama-client-core.

I did turn on gemini for the repo as well, will see how well it does.

gemini-code-assist · 2025-06-15T20:35:14Z

Important

Installation incomplete: to start using Gemini Code Assist, please ask the organization owner(s) to visit the Gemini Code Assist Admin Console and sign the Terms of Services.

ericcurtin · 2025-06-15T20:36:43Z

copilot suggested do something like this, that could be complete junk though, never tested, at least this solution uses urllib.request so one doesn't have to re-implement HTTP (even though the basics of http we need, wouldn't be that hard to re-implement if needs be):

import urllib.request
import socket
import json

DOCKER_SOCKET = "/var/run/docker.sock"
DOCKER_PATH = "/images/json"

class UnixHTTPConnection(urllib.request.HTTPConnection):
    def __init__(self, unix_socket_path):
        super().__init__('localhost')
        self.unix_socket_path = unix_socket_path

    def connect(self):
        self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
        self.sock.connect(self.unix_socket_path)

class UnixHTTPHandler(urllib.request.AbstractHTTPHandler):
    def __init__(self, unix_socket_path):
        self.unix_socket_path = unix_socket_path
        super().__init__()

    def http_open(self, req):
        return self.do_open(lambda host, **_: UnixHTTPConnection(self.unix_socket_path), req)

def get_docker_images():
    opener = urllib.request.build_opener(UnixHTTPHandler(DOCKER_SOCKET))
    url = f"http://localhost{DOCKER_PATH}"
    with opener.open(url) as response:
        data = response.read()
        return json.loads(data)

if __name__ == "__main__":
    images = get_docker_images()
    print(json.dumps(images, indent=2))

ericcurtin · 2025-06-15T20:47:43Z

We will want to delete "ramalama client" also with the libexec, this is a copy of that with a different name

gemini-code-assist · 2025-06-15T20:47:50Z

Important

Installation incomplete: to start using Gemini Code Assist, please ask the organization owner(s) to visit the Gemini Code Assist Admin Console and sign the Terms of Services.

Add ramalama chat command

a21fa39

For now we will just add the chat command, next PR will remove the external chat command and just use this internal one. Signed-off-by: Daniel J Walsh <[email protected]>

rhatdan requested review from ericcurtin, bmahabirbu, maxamillion, swarajpande5, jhjaggars, cgruver, slp and engelmi as code owners June 15, 2025 19:57

sourcery-ai bot approved these changes Jun 15, 2025

View reviewed changes

rhatdan force-pushed the chat branch from b6bc0f0 to 8404302 Compare June 15, 2025 20:14

Apply suggestions from code review

3cd6a59

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> Signed-off-by: Daniel J Walsh <[email protected]>

rhatdan force-pushed the chat branch from 8404302 to 3cd6a59 Compare June 15, 2025 20:16

ericcurtin approved these changes Jun 15, 2025

View reviewed changes

ericcurtin merged commit 2fe2e51 into containers:main Jun 15, 2025
15 checks passed

Add ramalama chat command #1531

Add ramalama chat command #1531

Uh oh!

Conversation

rhatdan commented Jun 15, 2025 • edited by sourcery-ai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by Sourcery

Uh oh!

gemini-code-assist bot commented Jun 15, 2025

Uh oh!

sourcery-ai bot commented Jun 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviewer's Guide

Sequence Diagram for 'ramalama chat' command execution

Sequence Diagram for 'ramalama chat' command execution

File-Level Changes

Interacting with Sourcery

Customizing Your Experience

Getting Help

Uh oh!

sourcery-ai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

sourcery-ai bot Jun 15, 2025

Choose a reason for hiding this comment

Uh oh!

sourcery-ai bot Jun 15, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot commented Jun 15, 2025

Uh oh!

gemini-code-assist bot commented Jun 15, 2025

Uh oh!

ericcurtin commented Jun 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ericcurtin commented Jun 15, 2025

Uh oh!

rhatdan commented Jun 15, 2025

Uh oh!

gemini-code-assist bot commented Jun 15, 2025

Uh oh!

ericcurtin commented Jun 15, 2025

Uh oh!

ericcurtin commented Jun 15, 2025

Uh oh!

Uh oh!

gemini-code-assist bot commented Jun 15, 2025

Uh oh!

Uh oh!

rhatdan commented Jun 15, 2025 •

edited by sourcery-ai bot

Loading

sourcery-ai bot commented Jun 15, 2025 •

edited

Loading

ericcurtin commented Jun 15, 2025 •

edited

Loading