multillm

A unified async interface for multiple LLM providers with agentic capabilities. Switch between providers with a single line change.

Features

🔄 Unified API - Same interface for OpenAI, Anthropic, Google, and more
🤖 Agentic by default - Automatic tool execution and multi-turn conversations
🛠️ Interactive tools - AI can ask you questions during execution
📦 Provider flexibility - Switch providers without changing code
🎯 Simple CLI - Quick testing and experimentation
⚡ Async-first - Built on asyncio for performance

Quick Start

CLI

# Install
pip install multillm-cli multillm-openai

# Simple query
multillm -m openai/gpt-4o -p "What is 2+2?"

# With interactive tools
multillm -m openai/gpt-4o -p "Ask me about my preferences" --use-tools ask_user

# With other tools
multillm -m openai/gpt-4o -p "What's the weather in Tokyo?" --use-tools get_weather

See multillm-cli for full CLI documentation.

Python API

pip install multillm multillm-openai multillm-anthropic

Simple query:

import asyncio
import multillm

async def main():
    client = multillm.Client()

    # Agentic API - works with any provider
    async for msg in client.run("agentwrap/openai/gpt-4o", "Hello!"):
        if msg.type == "text":
            print(msg.content)

asyncio.run(main())

With tools:

import asyncio
import multillm

# Define a tool
calculate = multillm.Tool(
    name="calculate",
    description="Perform a calculation",
    parameters={
        "type": "object",
        "properties": {
            "expression": {"type": "string"}
        },
        "required": ["expression"]
    },
    handler=lambda args: {"result": eval(args["expression"])}
)

async def main():
    client = multillm.Client()

    # AI can use tools automatically
    async for msg in client.run(
        "agentwrap/openai/gpt-4o",
        "What's 25 * 4?",
        tools=[calculate]
    ):
        if msg.type == "text":
            print(msg.content)
        elif msg.type == "tool_use":
            print(f"Using tool: {msg.tool_name}")

asyncio.run(main())

Interactive tools:

import asyncio
import multillm

# Define interactive tool
ask_user = multillm.Tool(
    name="ask_user",
    description="Ask the user a question",
    parameters={
        "type": "object",
        "properties": {
            "question": {"type": "string"}
        },
        "required": ["question"]
    },
    handler=lambda args: {
        "answer": input(f"\n{args['question']}\nYour answer: ")
    }
)

async def main():
    client = multillm.Client()

    # AI can ask you questions!
    async for msg in client.run(
        "agentwrap/openai/gpt-4o",
        "Help me plan a project by asking about my requirements",
        tools=[ask_user]
    ):
        if msg.type == "text":
            print(msg.content)

asyncio.run(main())

Packages

Package	Description
multillm	Core library with unified client
multillm-cli	Command-line interface
Chat Providers
multillm-openai	OpenAI GPT models
multillm-anthropic	Anthropic Claude chat API
multillm-gemini	Google Gemini
multillm-openrouter	OpenRouter (access to 100+ models)
Agent Providers
multillm-agentwrap	Wrap chat providers with agentic capabilities
multillm-claude	Claude native agent with built-in tools

How It Works

The Agentic API

All providers use the same agentic API powered by run():

async for msg in client.run(model, prompt, tools=tools):
    # Process messages

Message types: - system - Session started - text - Text response from AI - tool_use - AI is calling a tool - tool_result - Tool execution result - result - Final result

Provider Format

Chat providers with agentwrap:

"agentwrap/openai/gpt-4o"
"agentwrap/google/gemini-pro"
"agentwrap/anthropic/claude-3-5-sonnet-20241022"

Native agent providers:

"claude/default"
"claude/claude-sonnet-4-20250514"

What is agentwrap?

agentwrap wraps standard chat providers (OpenAI, Google, etc.) with agentic capabilities: - ✅ Automatic tool execution - ✅ Multi-turn conversations - ✅ Tool calling loop - ✅ Conversation history management

This means any chat model can work like an agent!

Interactive Tools

AI models can ask you questions during execution:

CLI:

# Chat providers
multillm -m openai/gpt-4o -p "Ask me about my project" --use-tools ask_user

# Claude agent
multillm -m claude/default -p "Ask me about my project" \
  --allowed-tools AskUserQuestion --permission-mode acceptEdits

Python:

ask_user_tool = multillm.Tool(
    name="ask_user",
    description="Ask the user a question",
    parameters={"type": "object", "properties": {"question": {"type": "string"}}},
    handler=lambda args: {"answer": input(f"{args['question']}\nYour answer: ")}
)

async for msg in client.run("agentwrap/openai/gpt-4o", "Ask me questions", tools=[ask_user_tool]):
    if msg.type == "text":
        print(msg.content)

When the AI calls the tool, you'll see:

======================================================================
❓ QUESTION FROM ASSISTANT
======================================================================

What is your favorite programming language?

Your answer: _

See CLI documentation for more interactive tool examples.

Configuration

Environment Variables

export OPENAI_API_KEY=sk-...
export ANTHROPIC_API_KEY=sk-ant-...
export GOOGLE_API_KEY=...

Programmatic

client = multillm.Client(config={
    "openai": {"api_key": "sk-..."},
    "anthropic": {"api_key": "sk-ant-..."},
})

Config Files

Create ~/.config/multillm/providers/<provider>.json:

{
  "api_key": "sk-..."
}

See provider-specific documentation for all options: - OpenAI configuration - Anthropic configuration - Google Gemini configuration - Claude Agent configuration

Examples

Chat with Different Providers

import asyncio
import multillm

async def chat(model: str, prompt: str):
    client = multillm.Client()
    async for msg in client.run(model, prompt):
        if msg.type == "text":
            print(msg.content)

# All use the same API!
asyncio.run(chat("agentwrap/openai/gpt-4o", "Hello"))
asyncio.run(chat("agentwrap/google/gemini-pro", "Hello"))
asyncio.run(chat("agentwrap/anthropic/claude-3-5-sonnet-20241022", "Hello"))
asyncio.run(chat("claude/default", "Hello"))

Custom Tools

import asyncio
import multillm
from datetime import datetime

# Define custom tools
get_time = multillm.Tool(
    name="get_current_time",
    description="Get the current time",
    parameters={"type": "object", "properties": {}},
    handler=lambda args: {"time": datetime.now().isoformat()}
)

weather = multillm.Tool(
    name="get_weather",
    description="Get weather for a location",
    parameters={
        "type": "object",
        "properties": {"location": {"type": "string"}},
        "required": ["location"]
    },
    handler=lambda args: {"temp": 72, "condition": "sunny"}
)

async def main():
    client = multillm.Client()

    async for msg in client.run(
        "agentwrap/openai/gpt-4o",
        "What time is it and what's the weather in Tokyo?",
        tools=[get_time, weather]
    ):
        if msg.type == "text":
            print(msg.content)

asyncio.run(main())

Agent Options

import asyncio
import multillm

async def main():
    client = multillm.Client()

    options = multillm.AgentOptions(
        max_turns=10,  # Max tool execution iterations
        extra={
            "temperature": 0.7,
            "max_tokens": 2000
        }
    )

    async for msg in client.run(
        "agentwrap/openai/gpt-4o",
        "Complex task requiring multiple steps",
        options=options
    ):
        if msg.type == "text":
            print(msg.content)

asyncio.run(main())

Claude Native Agent

Claude has a native agent provider with built-in tools:

import asyncio
import multillm

async def main():
    client = multillm.Client()

    async for msg in client.run(
        "claude/default",
        "List Python files in current directory",
        options=multillm.AgentOptions(
            allowed_tools=["Bash", "Glob"],
            permission_mode="acceptEdits",
            max_turns=5
        )
    ):
        if msg.type == "text":
            print(msg.content)

asyncio.run(main())

Built-in tools: Bash, Read, Write, Edit, Glob, Grep, Task, WebFetch, WebSearch, and more.

See Claude Agent documentation for details.

Development

This is a uv workspace:

# Install
uv sync

# Run examples
uv run python examples/test-agentwrap.py
uv run python examples/test-interactive-tools.py

# Run CLI
uv run multillm -m openai/gpt-4o -p "Hello"

Documentation

Getting Started Guide - Understand the API design
CLI Documentation - Command-line usage
Agentwrap Provider - Wrapping chat models
Claude Agent Provider - Native agent capabilities
Interactive Tools Guide - Building interactive agents
Migration Guide - Updating from older versions

Migration from single()

If you're using the deprecated single() API:

Old:

result = await client.single("openai/gpt-4o", "Hello")
print(result.text)

New:

async for msg in client.run("agentwrap/openai/gpt-4o", "Hello"):
    if msg.type == "text":
        print(msg.content)

See Migration Guide for details.

License

MIT