Skip to content

Readme improvements: Update model's name and improve CUDA_VISIBLE_DEVICES section #1691

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 14, 2025

Conversation

mbortoli
Copy link
Contributor

@mbortoli mbortoli commented Jul 13, 2025

  1. granite-moe3 is not found on Ollama, the name probably changed in the repository.
  2. It wasn't clear on how to use CPU instead of GPUs and what happened if CUDA_VISIBLE_DEVICES was set to empty.

Reproducing 1:

On bash: ramalama bench granite-moe3

Should show:

Error: Manifest for granite-moe3:latest was not found in the Ollama registry

Reproducing 2:

export CUDA_VISIBLE_DEVICES="" # or export CUDA_VISIBLE_DEVICES= 
ramalama bench granite3-moe

Will show:
ggml_cuda_init: failed to initialize CUDA: no CUDA-capable device is detected

And when executing unset CUDA_VISIBLE_DEVICES it returns to the standard behavior, using all GPUs.

Summary by Sourcery

Correct the model shortname in documentation and clarify CUDA_VISIBLE_DEVICES behavior.

Documentation:

  • Update all references of the model from granite-moe3 to granite3-moe in README and manpages
  • Add guidance that setting CUDA_VISIBLE_DEVICES to empty defaults to CPU and unsetting it restores GPU usage

Copy link
Contributor

sourcery-ai bot commented Jul 13, 2025

Reviewer's Guide

This PR refines documentation by replacing outdated model shortnames with the current one across examples and enriches the CUDA usage guide with explicit behavior and examples when CUDA_VISIBLE_DEVICES is empty.

File-Level Changes

Change Details Files
Correct model shortname in documentation examples
  • Replaced 'granite-moe3' with 'granite3-moe' in README usage examples
  • Updated benchmark command example in docs/ramalama-bench.1.md
  • Updated perplexity command example in docs/ramalama-perplexity.1.md
README.md
docs/ramalama-bench.1.md
docs/ramalama-perplexity.1.md
Clarify behavior when CUDA_VISIBLE_DEVICES is set empty
  • Inserted note that empty setting defaults to CPU
  • Added example commands showing export with empty value and unset
docs/ramalama-cuda.7.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary of Changes

Hello @mbortoli, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request focuses on enhancing the clarity and accuracy of the project's documentation. It addresses an outdated model name reference and provides essential guidance on configuring GPU/CPU device selection using the CUDA_VISIBLE_DEVICES environment variable, improving the user experience for ramalama users.

Highlights

  • Documentation Accuracy: Updated references to the granite-moe3 model to granite3-moe across the README.md and relevant man pages (ramalama-bench.1.md, ramalama-perplexity.1.md), reflecting a likely name change in the Ollama registry.
  • CUDA Device Management Clarity: Added detailed instructions and warnings in docs/ramalama-cuda.7.md regarding the behavior of the CUDA_VISIBLE_DEVICES environment variable. This clarifies that setting it to an empty string (export CUDA_VISIBLE_DEVICES="") forces CPU usage and provides guidance on how to unset it to revert to GPU usage.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in issue comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @mbortoli - I've reviewed your changes and they look great!


Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the model name from granite-moe3 to granite3-moe across the documentation. It also clarifies the behavior of the CUDA_VISIBLE_DEVICES environment variable, explaining how to force CPU usage and how to revert to using GPUs. The changes improve the documentation's accuracy and clarity.

@rhatdan
Copy link
Member

rhatdan commented Jul 14, 2025

You need to sign your commits.
Also please squash your commits.

git rebase -i origin
git push --force

@mbortoli mbortoli force-pushed the readme-improvements branch from 81849c5 to 3e89975 Compare July 14, 2025 11:41
@mbortoli
Copy link
Contributor Author

@rhatdan Sorry about that, first time ever contributing to a project.
Hope everything is alright now.

@rhatdan
Copy link
Member

rhatdan commented Jul 14, 2025

@mbortoli no problem and thanks. you still have not signed your commit.

git commit -a --amend -s
git push --force

@mbortoli mbortoli force-pushed the readme-improvements branch 2 times, most recently from 36c54b5 to f628815 Compare July 14, 2025 12:37
- Corrected the model name under the Benchmark section; previous name was not available in Ollama's registry.

- Added instructions to switch between CPU-only mode and using all available GPUs via CUDA_VISIBLE_DEVICES.

Signed-off-by: Mario Antonio Bortoli Filho <[email protected]>
@mbortoli mbortoli force-pushed the readme-improvements branch from f628815 to b5826c9 Compare July 14, 2025 12:43
@mbortoli
Copy link
Contributor Author

mbortoli commented Jul 14, 2025

@rhatdan done! my github email was set to private and I didn't know they needed to match

@rhatdan
Copy link
Member

rhatdan commented Jul 14, 2025

Thanks @mbortoli
LGTM

@rhatdan rhatdan merged commit eba46c8 into containers:main Jul 14, 2025
7 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants