path: root/README.md

# multillm

A unified async interface for multiple LLM providers with **agentic capabilities**. Switch between providers with a single line change.

## Features

- 🔄 **Unified API** - Same interface for OpenAI, Anthropic, Google, and more
- 🤖 **Agentic by default** - Automatic tool execution and multi-turn conversations
- 🛠️ **Interactive tools** - AI can ask you questions during execution
- 📦 **Provider flexibility** - Switch providers without changing code
- 🎯 **Simple CLI** - Quick testing and experimentation
- ⚡ **Async-first** - Built on asyncio for performance

## Quick Start

### CLI

```bash
# Install
pip install multillm-cli multillm-openai

# Simple query
multillm -m openai/gpt-4o -p "What is 2+2?"

# With interactive tools
multillm -m openai/gpt-4o -p "Ask me about my preferences" --use-tools ask_user

# With other tools
multillm -m openai/gpt-4o -p "What's the weather in Tokyo?" --use-tools get_weather
```

See [multillm-cli](packages/multillm-cli) for full CLI documentation.

### Python API

```bash
pip install multillm multillm-openai multillm-anthropic
```

**Simple query:**
```python
import asyncio
import multillm

async def main():
    client = multillm.Client()

    # Agentic API - works with any provider
    async for msg in client.run("agentwrap/openai/gpt-4o", "Hello!"):
        if msg.type == "text":
            print(msg.content)

asyncio.run(main())
```

**With tools:**
```python
import asyncio
import multillm

# Define a tool
calculate = multillm.Tool(
    name="calculate",
    description="Perform a calculation",
    parameters={
        "type": "object",
        "properties": {
            "expression": {"type": "string"}
        },
        "required": ["expression"]
    },
    handler=lambda args: {"result": eval(args["expression"])}
)

async def main():
    client = multillm.Client()

    # AI can use tools automatically
    async for msg in client.run(
        "agentwrap/openai/gpt-4o",
        "What's 25 * 4?",
        tools=[calculate]
    ):
        if msg.type == "text":
            print(msg.content)
        elif msg.type == "tool_use":
            print(f"Using tool: {msg.tool_name}")

asyncio.run(main())
```

**Interactive tools:**
```python
import asyncio
import multillm

# Define interactive tool
ask_user = multillm.Tool(
    name="ask_user",
    description="Ask the user a question",
    parameters={
        "type": "object",
        "properties": {
            "question": {"type": "string"}
        },
        "required": ["question"]
    },
    handler=lambda args: {
        "answer": input(f"\n{args['question']}\nYour answer: ")
    }
)

async def main():
    client = multillm.Client()

    # AI can ask you questions!
    async for msg in client.run(
        "agentwrap/openai/gpt-4o",
        "Help me plan a project by asking about my requirements",
        tools=[ask_user]
    ):
        if msg.type == "text":
            print(msg.content)

asyncio.run(main())
```

## Packages

| Package | Description |
|---------|-------------|
| [multillm](packages/multillm) | Core library with unified client |
| [multillm-cli](packages/multillm-cli) | Command-line interface |
| **Chat Providers** | |
| [multillm-openai](packages/multillm-openai) | OpenAI GPT models |
| [multillm-anthropic](packages/multillm-anthropic) | Anthropic Claude chat API |
| [multillm-gemini](packages/multillm-gemini) | Google Gemini |
| [multillm-openrouter](packages/multillm-openrouter) | OpenRouter (access to 100+ models) |
| **Agent Providers** | |
| [multillm-agentwrap](packages/multillm-agentwrap) | Wrap chat providers with agentic capabilities |
| [multillm-claude](packages/multillm-claude) | Claude native agent with built-in tools |

## How It Works

### The Agentic API

All providers use the same **agentic API** powered by `run()`:

```python
async for msg in client.run(model, prompt, tools=tools):
    # Process messages
```

**Message types:**
- `system` - Session started
- `text` - Text response from AI
- `tool_use` - AI is calling a tool
- `tool_result` - Tool execution result
- `result` - Final result

### Provider Format

**Chat providers with agentwrap:**
```python
"agentwrap/openai/gpt-4o"
"agentwrap/google/gemini-pro"
"agentwrap/anthropic/claude-3-5-sonnet-20241022"
```

**Native agent providers:**
```python
"claude/default"
"claude/claude-sonnet-4-20250514"
```

### What is agentwrap?

`agentwrap` wraps standard chat providers (OpenAI, Google, etc.) with agentic capabilities:
- ✅ Automatic tool execution
- ✅ Multi-turn conversations
- ✅ Tool calling loop
- ✅ Conversation history management

This means **any chat model** can work like an agent!

## Interactive Tools

AI models can ask you questions during execution:

**CLI:**
```bash
# Chat providers
multillm -m openai/gpt-4o -p "Ask me about my project" --use-tools ask_user

# Claude agent
multillm -m claude/default -p "Ask me about my project" \
  --allowed-tools AskUserQuestion --permission-mode acceptEdits
```

**Python:**
```python
ask_user_tool = multillm.Tool(
    name="ask_user",
    description="Ask the user a question",
    parameters={"type": "object", "properties": {"question": {"type": "string"}}},
    handler=lambda args: {"answer": input(f"{args['question']}\nYour answer: ")}
)

async for msg in client.run("agentwrap/openai/gpt-4o", "Ask me questions", tools=[ask_user_tool]):
    if msg.type == "text":
        print(msg.content)
```

When the AI calls the tool, you'll see:
```
======================================================================
❓ QUESTION FROM ASSISTANT
======================================================================

What is your favorite programming language?

Your answer: _
```

See [CLI documentation](packages/multillm-cli) for more interactive tool examples.

## Configuration

### Environment Variables

```bash
export OPENAI_API_KEY=sk-...
export ANTHROPIC_API_KEY=sk-ant-...
export GOOGLE_API_KEY=...
```

### Programmatic

```python
client = multillm.Client(config={
    "openai": {"api_key": "sk-..."},
    "anthropic": {"api_key": "sk-ant-..."},
})
```

### Config Files

Create `~/.config/multillm/providers/<provider>.json`:

```json
{
  "api_key": "sk-..."
}
```

See provider-specific documentation for all options:
- [OpenAI configuration](packages/multillm-openai)
- [Anthropic configuration](packages/multillm-anthropic)
- [Google Gemini configuration](packages/multillm-gemini)
- [Claude Agent configuration](packages/multillm-claude)

## Examples

### Chat with Different Providers

```python
import asyncio
import multillm

async def chat(model: str, prompt: str):
    client = multillm.Client()
    async for msg in client.run(model, prompt):
        if msg.type == "text":
            print(msg.content)

# All use the same API!
asyncio.run(chat("agentwrap/openai/gpt-4o", "Hello"))
asyncio.run(chat("agentwrap/google/gemini-pro", "Hello"))
asyncio.run(chat("agentwrap/anthropic/claude-3-5-sonnet-20241022", "Hello"))
asyncio.run(chat("claude/default", "Hello"))
```

### Custom Tools

```python
import asyncio
import multillm
from datetime import datetime

# Define custom tools
get_time = multillm.Tool(
    name="get_current_time",
    description="Get the current time",
    parameters={"type": "object", "properties": {}},
    handler=lambda args: {"time": datetime.now().isoformat()}
)

weather = multillm.Tool(
    name="get_weather",
    description="Get weather for a location",
    parameters={
        "type": "object",
        "properties": {"location": {"type": "string"}},
        "required": ["location"]
    },
    handler=lambda args: {"temp": 72, "condition": "sunny"}
)

async def main():
    client = multillm.Client()

    async for msg in client.run(
        "agentwrap/openai/gpt-4o",
        "What time is it and what's the weather in Tokyo?",
        tools=[get_time, weather]
    ):
        if msg.type == "text":
            print(msg.content)

asyncio.run(main())
```

### Agent Options

```python
import asyncio
import multillm

async def main():
    client = multillm.Client()

    options = multillm.AgentOptions(
        max_turns=10,  # Max tool execution iterations
        extra={
            "temperature": 0.7,
            "max_tokens": 2000
        }
    )

    async for msg in client.run(
        "agentwrap/openai/gpt-4o",
        "Complex task requiring multiple steps",
        options=options
    ):
        if msg.type == "text":
            print(msg.content)

asyncio.run(main())
```

## Claude Native Agent

Claude has a native agent provider with built-in tools:

```python
import asyncio
import multillm

async def main():
    client = multillm.Client()

    async for msg in client.run(
        "claude/default",
        "List Python files in current directory",
        options=multillm.AgentOptions(
            allowed_tools=["Bash", "Glob"],
            permission_mode="acceptEdits",
            max_turns=5
        )
    ):
        if msg.type == "text":
            print(msg.content)

asyncio.run(main())
```

**Built-in tools:** Bash, Read, Write, Edit, Glob, Grep, Task, WebFetch, WebSearch, and more.

See [Claude Agent documentation](packages/multillm-claude) for details.

## Development

This is a uv workspace:

```bash
# Install
uv sync

# Run examples
uv run python examples/test-agentwrap.py
uv run python examples/test-interactive-tools.py

# Run CLI
uv run multillm -m openai/gpt-4o -p "Hello"
```

## Documentation

- [Getting Started Guide](INTERFACE_CONCEPTS.md) - Understand the API design
- [CLI Documentation](packages/multillm-cli/README.md) - Command-line usage
- [Agentwrap Provider](packages/multillm-agentwrap/README.md) - Wrapping chat models
- [Claude Agent Provider](packages/multillm-claude/README.md) - Native agent capabilities
- [Interactive Tools Guide](INTERACTIVE_TOOLS_IMPLEMENTATION.md) - Building interactive agents
- [Migration Guide](MIGRATION_SINGLE_TO_AGENTWRAP.md) - Updating from older versions

## Migration from single()

If you're using the deprecated `single()` API:

**Old:**
```python
result = await client.single("openai/gpt-4o", "Hello")
print(result.text)
```

**New:**
```python
async for msg in client.run("agentwrap/openai/gpt-4o", "Hello"):
    if msg.type == "text":
        print(msg.content)
```

See [Migration Guide](MIGRATION_SINGLE_TO_AGENTWRAP.md) for details.

## License

MIT