Generic LLM Interface (#13)

* new generic llm interface

* ollama refactor

* minor fixes

* fix ollama num_ctx

* fix invalid parameters in openai

* remove redundant and unused code

* update poetry and build dependencies

* update depracated poetry

Author: DavidKoleczek

.github/_typos.toml

@@ -1,2 +1,5 @@
 [default.extend-identifiers]
-arange = "arange" # np.arange
+arange = "arange" # np.arange
+
+[files]
+extend-exclude = ["*.ipynb"]

.github/workflows/pages.yml

@@ -8,21 +8,21 @@ on:
     branches: [main]
 
 env:
-  POETRY_VERSION: "1.8.3"
-  PYTHON_VERSION: "3.11"
+  POETRY_VERSION: "2.0.1"
+  PYTHON_VERSION: "3.12"
 
 jobs:
   deploy:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
       - name: Install poetry
         run: pipx install poetry==${POETRY_VERSION}
-      - uses: actions/setup-python@v4
+      - uses: actions/setup-python@v5
         with:
           python-version: ${{ env.PYTHON_VERSION }}
           cache: poetry
       - name: Install Nox and doc dependencies
-        run: poetry install --only nox,docs
+        run: poetry sync --only nox,docs
       - name: Publish Docs
         run: poetry run nox -s docs_github_pages

.github/workflows/python.yml

@@ -8,36 +8,36 @@ on:
     branches: ["main"]
 
 env:
-  POETRY_VERSION: "1.8.3"
+  POETRY_VERSION: "2.0.1"
   PYTHON_VERSION: "3.12"
 
 jobs:
   test:
-    runs-on: ubuntu-22.04
+    runs-on: ubuntu-24.04
     strategy:
       matrix:
         python-version: ["3.11", "3.12" ]
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
       - name: Install poetry
         run: pipx install poetry==${POETRY_VERSION}
       - name: Install poetry-plugin-export
         run: pipx inject poetry poetry-plugin-export
       - name: Set up Python ${{ matrix.python-version }}
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
           cache: poetry
       - name: Install Nox
-        run: poetry install --only nox
+        run: poetry sync --only nox
       - name: Test with Nox
         env:
           OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
           OPENAI_ORG_ID: ${{ secrets.OPENAI_ORG_ID }}
-          SKIP_TESTS_NAAI: "tests/llm tests/local_llm tests/data"
+          SKIP_TESTS_NAAI: "tests/llm tests/data"
         run: poetry run nox -s test-${{ matrix.python-version }}
   quality:
-    runs-on: ubuntu-22.04
+    runs-on: ubuntu-24.04
     strategy:
       matrix:
         job:
@@ -46,27 +46,27 @@ jobs:
           - { nox-session: type_check, poetry-groups: "type_check,main,test" }
           - { nox-session: docs, poetry-groups: "docs" }
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
       - name: Install poetry
         run: pipx install poetry==${POETRY_VERSION}
       - name: Set up Python
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
         with:
           python-version: ${{ env.PYTHON_VERSION }}
           cache: poetry
       - name: Install Nox and group dependencies
-        run: poetry install --all-extras
+        run: poetry sync --all-extras
       - name: Test with Nox
         run: poetry run nox -s ${{ matrix.job.nox-session }}
   poetry-check:
-    runs-on: ubuntu-22.04
+    runs-on: ubuntu-24.04
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
       - name: Install poetry
         run: pipx install poetry==${POETRY_VERSION}
       - name: Set up Python
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
         with:
           python-version: ${{ env.PYTHON_VERSION }}
       - name: Validate Poetry Configuration
-        run: poetry check --lock
+        run: poetry check --lock --strict

.vscode/settings.json

@@ -17,6 +17,6 @@
     },
     "python.testing.unittestEnabled": false,
     "python.testing.pytestEnabled": true,
-    "python.testing.pytestArgs": [],
+    "python.testing.pytestArgs": ["-s"],
     "markdown.extension.orderedList.marker": "one",
 }

README.md

@@ -24,11 +24,9 @@ Requires: Python 3.11, or 3.12
 Install the entire package from [PyPI](https://pypi.org/project/not-again-ai/) with: 
 
 ```bash
-$ pip install not_again_ai[llm,local_llm,statistics,viz]
+$ pip install not_again_ai[data,llm,statistics,viz]
 ```
 
-Note that local LLM requires separate installations and will not work out of the box due to how hardware dependent it is. Be sure to check the [notebooks](notebooks/local_llm/) for more details.
-
 The package is split into subpackages, so you can install only the parts you need.
 
 ### Base
@@ -49,16 +47,7 @@ The package is split into subpackages, so you can install only the parts you nee
    1. Using AOAI requires using Entra ID authentication. See https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/managed-identity for how to set this up for your AOAI deployment.
       * Requires the correct role assigned to your user account and being signed into the Azure CLI.
    1. (Optional) Set the `AZURE_OPENAI_ENDPOINT` environment variable.
-1. Setup GitHub Models
-   1. Get a Personal Access Token from https://github.com/settings/tokens and set the `GITHUB_TOKEN` environment variable. The token does not need any permissions.
-   1. Check the [Github Marketplace](https://github.com/marketplace/models) to see which models are available.
-
-
-### Local LLM
- 1. `pip install not_again_ai[llm,local_llm]`
- 1. Some HuggingFace transformers tokenizers are gated behind access requests. If you wish to use these, you will need to request access from HuggingFace on the model card.
-    * Then set the `HF_TOKEN` environment variable to your HuggingFace API token which can be found here: https://huggingface.co/settings/tokens
- 1. If you wish to use Ollama:
+1. If you wish to use Ollama:
      1. Follow the instructions at https://github.com/ollama/ollama to install Ollama for your system. 
      1. (Optional) [Add Ollama as a startup service (recommended)](https://github.com/ollama/ollama/blob/main/docs/linux.md#adding-ollama-as-a-startup-service-recommended)
      1. (Optional) To make the Ollama service accessible on your local network from a Linux server, add the following to the `/etc/systemd/system/ollama.service` file which will make Ollama available at `http://<local_address>:11434`:
@@ -68,7 +57,6 @@ The package is split into subpackages, so you can install only the parts you nee
          Environment="OLLAMA_HOST=0.0.0.0"
          ```
      1. It is recommended to always have the latest version of Ollama. To update Ollama check the [docs](https://github.com/ollama/ollama/blob/main/docs/). The command for Linux is: `curl -fsSL https://ollama.com/install.sh | sh`
- 1. HuggingFace transformers and other requirements are hardware dependent so for providers other than Ollama, this only installs some generic dependencies. Check the [notebooks](notebooks/local_llm/) for more details on what is available and how to install it.
 
 
 ### Statistics
@@ -112,10 +100,8 @@ $ poetry update
 
 To install all dependencies (with all extra dependencies) into an isolated virtual environment:
 
-> Append `--sync` to uninstall dependencies that are no longer in use from the virtual environment.
-
 ```bash
-$ poetry install --all-extras
+$ poetry sync --all-extras
 ```
 
 To [activate](https://python-poetry.org/docs/basic-usage#activating-the-virtual-environment) the
@@ -171,7 +157,7 @@ Automated code quality checks are performed using
 environments and run commands based on [`noxfile.py`](./noxfile.py) for unit testing, PEP 8 style
 guide checking, type checking and documentation generation.
 
-> Note: `nox` is installed into the virtual environment automatically by the `poetry install`
+> Note: `nox` is installed into the virtual environment automatically by the `poetry sync`
 > command above. Run `poetry shell` to activate the virtual environment.
 
 To run all default sessions: