Try and compare some of the available js/ts llm libraries

[justyns]

This issue is mostly so I don't forget about this and can write my findings somewhere.

Currently, we have interfaces for different LLMs from scratch in https://github.com/justyns/silverbullet-ai/tree/main/src/providers

This is fine right now because the only functionality really being used is chat completions and embedding generation. However, if we add multi-modal (image) support, tool support, etc, each provider could have its own weird differences that turn into a pain to maintain.

I'm hoping to find a good javascript/typescript library to abstract most of that complexity out. These are some of the options:

• langchain.js
  • opinionated / over-engineered / complex
• multi-llm-ts
  • large bundle size, depends on every providers individual sdk instead of directly using apis
• hopfield
  • repo recently archived
• huggingface.js
  • huggingface models only?
• llm-chain
  • easy to extend and add new providers
• token.js
  • looks easy to use
  • large bundle size, depends on every providers individual sdk instead of directly using apis
• LLM.js
• LlamaIndexTS
• ???

Other than using a library, another option talked about before is to only support openai/ollama and then rely on an external proxy/router like litellm that does the translation to other llm provider apis.