Currently natively supports: OpenAI, Anthropic, Gemini, XAI/Grok, Ollama, Groq, DeepSeek (deepseek.com & Groq), Cohere (more to come)

Also allows a custom URL with ServiceTargetResolver (see examples/c06-target-resolver.rs)

Provides a single, ergonomic API to many generative AI providers, such as Anthropic, OpenAI, Gemini, xAI, Ollama, Groq, and more.

NOTE: Big update with v0.4.x - More adapters, PDF and image support, embeddings, custom headers, and transparent support for the OpenAI Responses API (gpt-5-codex)

• What's new:
  • PDF and Images support (thanks to Andrew Rademacher)
  • Embedding support (thanks to Jesus Santander)
  • Custom Headers support (for AWS Bedrock, Vertex, etc.) (thanks to Adrien/Julien Chaumond)
  • Simpler, flatter MessageContent multi-part format (API change) (thanks to Andrew Rademacher for insights)
  • Raw body capture with ChatOptions::with_capture_raw_body(true) (thanks to 4t145)
  • Transparent gpt-5-codex support with the Responses API, even if gpt-5-codex uses a new API protocol (OpenAI Responses API)
• What's still awesome:
  • Normalized and ergonomic Chat API across all providers
  • Most providers built in (OpenAI, Gemini, Anthropic, xAI, Groq, Together.ai, Fireworks.ai, ...)
  • Native protocol support for Gemini and Anthropic protocols, for example allowing full budget controls with Gemini models
  • Can override auth, endpoint, and headers to connect to AWS Bedrock, Vertex AI, etc.

See: - migration from v0.3 to v0.4 - CHANGELOG

• Adrien Extra headers in requests, seed for chat requests, and fixes (with Julien Chaumond for extra headers)
• Andrew Rademacher for PDF support, Anthropic streamer, and insight on flattening the message content (e.g., ContentParts)
• Jesus Santander Embedding support PR #83
• 4t145 for raw body capture PR #68
• Vagmi Mudumbai exec_chat bug fix PR #86
• Maximilian Goisser Fix OpenAI adapter to use ServiceTarget
• ClanceyLu for Tool Use Streaming support, web configuration support, and fixes
• @SilasMarvin for fixing content/tools issues with some Ollama models PR #55
• @una-spirito for Gemini ReasoningEffort::Budget support
• @jBernavaPrah for adding tracing (it was long overdue). PR #45
• @GustavoWidman for the initial Gemini tool/function support! PR #41
• @AdamStrojek for initial image support PR #36
• @semtexzv for stop_sequences Anthropic support PR #34
• @omarshehab221 for de/serialize on structs PR #19
• @tusharmath for making webc::Error PR #12
• @giangndm for making stream Send PR #10
• @stargazing-dino for PR #2 - implement Groq completions

• Check out AIPACK, which wraps this genai library into an agentic runtime to run, build, and share AI Agent Packs. See pro@coder for a simple example of how I use AI PACK/genai for production coding.

Note: Feel free to send me a short description and a link to your application or library using genai.

• Native Multi-AI Provider/Model: OpenAI, Anthropic, Gemini, Ollama, Groq, xAI, DeepSeek (Direct chat and stream) (see examples/c00-readme.rs)
• DeepSeekR1 support, with reasoning_content (and stream support), plus DeepSeek Groq and Ollama support (and reasoning_content normalization)
• Image Analysis (for OpenAI, Gemini flash-2, Anthropic) (see examples/c07-image.rs)
• Custom Auth/API Key (see examples/c02-auth.rs)
• Model aliases (see examples/c05-model-names.rs)
• Custom endpoint, auth, and model identifier (see examples/c06-target-resolver.rs)

Examples | Thanks | Library Focus | Changelog | Provider Mapping: ChatOptions | Usage

examples/c00-readme.rs

//! Base examples demonstrating the core capabilities of genai

use genai::chat::printer::{print_chat_stream, PrintChatStreamOptions};
use genai::chat::{ChatMessage, ChatRequest};
use genai::Client;

const MODEL_OPENAI: &str = "gpt-4o-mini"; // o1-mini, gpt-4o-mini
const MODEL_ANTHROPIC: &str = "claude-3-haiku-20240307";
const MODEL_COHERE: &str = "command-light";
const MODEL_GEMINI: &str = "gemini-2.0-flash";
const MODEL_GROQ: &str = "llama-3.1-8b-instant";
const MODEL_OLLAMA: &str = "gemma:2b"; // sh: `ollama pull gemma:2b`
const MODEL_XAI: &str = "grok-beta";
const MODEL_DEEPSEEK: &str = "deepseek-chat";

// NOTE: These are the default environment keys for each AI Adapter Type.
//       They can be customized; see `examples/c02-auth.rs`
const MODEL_AND_KEY_ENV_NAME_LIST: &[(&str, &str)] = &[
	// -- De/activate models/providers
	(MODEL_OPENAI, "OPENAI_API_KEY"),
	(MODEL_ANTHROPIC, "ANTHROPIC_API_KEY"),
	(MODEL_COHERE, "COHERE_API_KEY"),
	(MODEL_GEMINI, "GEMINI_API_KEY"),
	(MODEL_GROQ, "GROQ_API_KEY"),
	(MODEL_XAI, "XAI_API_KEY"),
	(MODEL_DEEPSEEK, "DEEPSEEK_API_KEY"),
	(MODEL_OLLAMA, ""),
];

// NOTE: Model to AdapterKind (AI Provider) type mapping rule
//  - starts_with "gpt"      -> OpenAI
//  - starts_with "claude"   -> Anthropic
//  - starts_with "command"  -> Cohere
//  - starts_with "gemini"   -> Gemini
//  - model in Groq models   -> Groq
//  - For anything else      -> Ollama
//
// This can be customized; see `examples/c03-mapper.rs`

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
	let question = "Why is the sky red?";

	let chat_req = ChatRequest::new(vec![
		// -- Messages (de/activate to see the differences)
		ChatMessage::system("Answer in one sentence"),
		ChatMessage::user(question),
	]);

	let client = Client::default();

	let print_options = PrintChatStreamOptions::from_print_events(false);

	for (model, env_name) in MODEL_AND_KEY_ENV_NAME_LIST {
		// Skip if the environment name is not set
		if !env_name.is_empty() && std::env::var(env_name).is_err() {
			println!("===== Skipping model: {model} (env var not set: {env_name})");
			continue;
		}

		let adapter_kind = client.resolve_service_target(model).await?.model.adapter_kind;

		println!("\n===== MODEL: {model} ({adapter_kind}) =====");

		println!("\n--- Question:\n{question}");

		println!("\n--- Answer:");
		let chat_res = client.exec_chat(model, chat_req.clone(), None).await?;
		println!("{}", chat_res.first_text().unwrap_or("NO ANSWER"));

		println!("\n--- Answer: (streaming)");
		let chat_res = client.exec_chat_stream(model, chat_req.clone(), None).await?;
		print_chat_stream(chat_res, Some(&print_options)).await?;

		println!();
	}

	Ok(())
}

• examples/c00-readme.rs - Quick overview code with multiple providers and streaming.
• examples/c01-conv.rs - Shows how to build a conversation flow.
• examples/c02-auth.rs - Demonstrates how to provide a custom AuthResolver to provide auth data (i.e., for api_key) per adapter kind.
• examples/c03-mapper.rs - Demonstrates how to provide a custom AdapterKindResolver to customize the "model name" to "adapter kind" mapping.
• examples/c04-chat-options.rs - Demonstrates how to set chat generation options such as temperature and max_tokens at the client level (for all requests) and per-request level.
• examples/c05-model-names.rs - Shows how to get model names per AdapterKind.
• examples/c06-target-resolver.rs - For custom auth, endpoint, and model.
• examples/c07-image.rs - Image analysis support

• genai ModelMapper code demo (v0.1.7)

• genai introduction (v0.1.0)

• genai live coding, code design, & best practices

  • Adding Gemini Structured Output (vid-0060)
  • Adding OpenAI Structured Output (vid-0059)
  • Splitting the json value extension trait to its own public crate value-ext value-ext
  • (part 1/3) Module, Error, constructors/builders
  • (part 2/3) Extension Traits, Project Files, Versioning
  • (part 3/3) When to Async? Project Files, Versioning strategy

• Focuses on standardizing chat completion APIs across major AI services.

• Native implementation, meaning no per-service SDKs.

  • Reason: While there are some variations across the various APIs, they all follow the same pattern and high-level flow and constructs. Managing the differences at a lower layer is actually simpler and more cumulative across services than doing SDK gymnastics.

• Prioritizes ergonomics and commonality, with depth being secondary. (If you require a complete client API, consider using async-openai and ollama-rs; they are both excellent and easy to use.)

• Initially, this library will mostly focus on text chat APIs, with images and function calling coming later.

• (1) - OpenAI-compatible notes
  • Models: OpenAI, DeepSeek, Groq, Ollama, xAI

• (1) - OpenAI-compatible notes

  • Models: OpenAI, DeepSeek, Groq, Ollama, xAI
  • For Groq, the property x_groq.usage.
  • At this point, Ollama does not emit input/output tokens when streaming due to the Ollama OpenAI compatibility layer limitation. (see ollama #4448 - Streaming Chat Completion via OpenAI API should support stream option to include Usage)
  • prompt_tokens_details and completion_tokens_details will have the value sent by the compatible provider (or None)

• (2): Gemini tokens

  • Right now, with the Gemini Stream API, it's not clear whether usage for each event is cumulative or must be summed. It appears to be cumulative, meaning the last message shows the total amount of input, output, and total tokens, so that is the current assumption. See possible tweet answer for more info.

• Will add more data on ChatResponse and ChatStream, especially metadata about usage.
• Add vision/image support to chat messages and responses.
• Add function calling support to chat messages and responses.
• Add embed and embed_batch.
• Add the AWS Bedrock variants (e.g., Mistral and Anthropic). Most of the work will be on the "interesting" token signature scheme; trying to avoid bringing in large SDKs, this might be a lower-priority feature.
• Add the Google Vertex AI variants.
• May add the Azure OpenAI variant (not sure yet).

• crates.io: crates.io/crates/genai
• GitHub: github.com/jeremychone/rust-genai
• Sponsored by BriteSnow (Jeremy Chone's consulting company)