Tool Calling with OpenAI & Anthropic

Tool Calling: The Mechanism That Makes Agents Real

Without tool calling, an LLM can only generate text. With tool calling, it can:

• Search the web and return real-time information
• Query databases and return structured data
• Call APIs and trigger side effects
• Execute code and return results

The mechanism: you declare a set of tools (as JSON schemas). When the model decides to use one, instead of generating free-form text, it generates a structured JSON object matching the tool’s schema. Your code intercepts that, calls the real function, and sends the result back. The model then continues its reasoning with the actual result in context.

Tool Calling with the OpenAI API

1. Define Your Tools

Tools are declared as a JSON schema list. The model uses the description fields to decide when and how to call each tool, write them like documentation for a smart colleague, not a computer.

# src/agents/tools.py
import json
import httpx
from datetime import datetime

# OpenAI tool schema format: each entry is a dict with "type" and "function" keys
tools = [
    {
        "type": "function",   # always "function" for tool calls
        "function": {
            "name": "get_current_weather",
            # description is what the model reads to decide WHEN to call this tool: be specific
            "description": "Get the current weather for a city. Use this when the user asks about weather.",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {
                        "type": "string",
                        "description": "The city name, e.g. 'London' or 'New York'"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],  # enum constrains the model to valid values only
                        "description": "Temperature unit"
                    }
                },
                "required": ["city"],          # model must always provide city; unit is optional
                "additionalProperties": False  # strict: model cannot invent extra keys
            },
            "strict": True  # OpenAI strict mode: guarantees the response JSON matches this schema exactly
        }
    },
    {
        "type": "function",
        "function": {
            "name": "search_web",
            "description": "Search the web for current information. Use when you need up-to-date facts.",
            "parameters": {
                "type": "object",
                "properties": {
                    "query": {
                        "type": "string",
                        "description": "The search query"
                    },
                    "num_results": {
                        "type": "integer",
                        "description": "Number of results to return (1-10)",
                        "default": 5
                    }
                },
                "required": ["query"],
                "additionalProperties": False
            },
            "strict": True
        }
    }
]

2. Implement the Python Functions

# Actual implementations: these run when the model calls the tool
async def get_current_weather(city: str, unit: str = "celsius") -> dict:
    # Real implementation would call a weather API like OpenWeatherMap or WeatherAPI
    return {
        "city": city,
        "temperature": 18,
        "unit": unit,
        "condition": "partly cloudy",
        "humidity": 65,
        "retrieved_at": datetime.utcnow().isoformat()  # include timestamp so model knows data freshness
    }

async def search_web(query: str, num_results: int = 5) -> list[dict]:
    # Real implementation would call Serper, Tavily, or Brave Search API
    return [
        {"title": f"Result for: {query}", "url": "https://example.com", "snippet": "..."},
    ]

# Dispatch table: maps function name string (from the model) → callable Python function
# This avoids eval() or getattr() hacks and makes the allowed set explicit
TOOL_FUNCTIONS = {
    "get_current_weather": get_current_weather,
    "search_web": search_web,
}

3. The Agent Loop

The core agent loop sends messages, checks whether the model wants to call a tool, executes it, feeds the result back, and repeats until the model produces a plain text response with no tool calls.

# src/agents/openai_agent.py
import asyncio
import json
from openai import AsyncOpenAI
from src.agents.config import settings
from src.agents.tools import tools, TOOL_FUNCTIONS

client = AsyncOpenAI(api_key=settings.openai_api_key)

async def run_agent(user_message: str) -> str:
    messages = [
        {"role": "system", "content": "You are a helpful assistant with access to weather and search tools."},
        {"role": "user", "content": user_message}
    ]
    
    while True:  # loop continues until the model returns a text response with no tool calls
        response = await client.chat.completions.create(
            model="gpt-5.6-luna",
            messages=messages,
            tools=tools,
            tool_choice="auto",       # "auto" lets the model decide; "required" forces a tool call
            parallel_tool_calls=True, # model can request multiple tools in one turn, reduces round trips
            temperature=0.0,          # deterministic outputs are critical for tool argument generation
        )
        
        message = response.choices[0].message
        messages.append(message)  # always append the assistant's message to maintain conversation history
        
        # Check if model wants to call tools
        if message.tool_calls:
            # asyncio.gather runs all tool calls concurrently: key for latency when tools are I/O-bound
            tool_results = await asyncio.gather(*[
                execute_tool_call(tc) for tc in message.tool_calls
            ])
            
            # All results must be sent back together before the model can continue
            for result in tool_results:
                messages.append(result)
            
            # Continue the loop: model will reason with the results
            continue
        
        # No tool calls = model has reached a final answer
        return message.content

async def execute_tool_call(tool_call) -> dict:
    func_name = tool_call.function.name
    func_args = json.loads(tool_call.function.arguments)  # model returns args as a JSON string
    
    print(f"  → Calling tool: {func_name}({func_args})")
    
    if func_name not in TOOL_FUNCTIONS:
        # Return error as a string rather than raising: the model can handle the error gracefully
        result = {"error": f"Unknown tool: {func_name}"}
    else:
        try:
            result = await TOOL_FUNCTIONS[func_name](**func_args)
        except Exception as e:
            result = {"error": str(e)}  # surface errors to the model so it can retry or explain
    
    return {
        "role": "tool",              # OpenAI uses role="tool" for tool results (Anthropic uses "user")
        "tool_call_id": tool_call.id, # must match the id from the model's tool_call, links result to request
        "content": json.dumps(result) # content must be a string, serialize dicts to JSON
    }

Tool Calling with the Anthropic API

Anthropic’s API uses slightly different terminology (tool_use blocks instead of tool_calls) but the same conceptual flow.

# src/agents/anthropic_agent.py
import asyncio
import json
from anthropic import AsyncAnthropic
from src.agents.config import settings

client = AsyncAnthropic(api_key=settings.anthropic_api_key)

# Anthropic tool schema format: note "input_schema" instead of OpenAI's "parameters"
anthropic_tools = [
    {
        "name": "get_current_weather",
        "description": "Get the current weather for a city.",
        "input_schema": {           # Anthropic uses "input_schema" where OpenAI uses "parameters"
            "type": "object",
            "properties": {
                "city": {"type": "string", "description": "City name"},
                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
            },
            "required": ["city"]
        }
    }
]

async def run_anthropic_agent(user_message: str) -> str:
    messages = [{"role": "user", "content": user_message}]
    
    while True:
        response = await client.messages.create(
            model="claude-sonnet-5",
            max_tokens=4096,  # Anthropic requires an explicit max_tokens, no default
            system="You are a helpful assistant with weather and search tools.",
            messages=messages,
            tools=anthropic_tools,
        )
        
        # Anthropic returns response.content as a list of blocks (text, tool_use, etc.)
        messages.append({"role": "assistant", "content": response.content})
        
        # Filter for tool_use blocks: the model may return a mix of text and tool_use in one response
        tool_use_blocks = [b for b in response.content if b.type == "tool_use"]
        
        if not tool_use_blocks:
            # No tool calls: extract the final text response
            text_blocks = [b for b in response.content if b.type == "text"]
            return text_blocks[0].text if text_blocks else ""
        
        # Execute tools and collect results
        tool_results = []
        for block in tool_use_blocks:
            result = await TOOL_FUNCTIONS[block.name](**block.input)  # block.input is already a dict
            tool_results.append({
                "type": "tool_result",
                "tool_use_id": block.id,       # must match the block.id from the response
                "content": json.dumps(result)
            })
        
        # Anthropic tool results go back as role="user" with a content list: different from OpenAI's role="tool"
        messages.append({"role": "user", "content": tool_results})

Key Differences: OpenAI vs Anthropic Tool Calling

Testing Tool Calls in Isolation

Never test tool calling with a live model when unit testing. Mock the LLM responses:

# tests/test_openai_agent.py
import pytest
from unittest.mock import AsyncMock, MagicMock
from src.agents.openai_agent import run_agent

@pytest.mark.asyncio
async def test_weather_tool_called(monkeypatch):
    # Build a fake tool-calling response that mimics what the OpenAI API returns
    mock_tool_call = MagicMock()
    mock_tool_call.id = "call_abc123"
    mock_tool_call.function.name = "get_current_weather"
    mock_tool_call.function.arguments = '{"city": "London", "unit": "celsius"}'
    
    # First response: model requests the tool
    first_response = MagicMock()
    first_response.choices[0].message.tool_calls = [mock_tool_call]
    first_response.choices[0].message.content = None  # no text when calling a tool
    
    # Second response: model produces final answer after seeing the tool result
    second_response = MagicMock()
    second_response.choices[0].message.tool_calls = None
    second_response.choices[0].message.content = "London is 18°C and partly cloudy."
    
    # side_effect returns responses in sequence: simulates the two-turn tool loop
    mock_create = AsyncMock(side_effect=[first_response, second_response])
    monkeypatch.setattr("src.agents.openai_agent.client.chat.completions.create", mock_create)
    
    result = await run_agent("What's the weather in London?")
    assert "London" in result
    assert mock_create.call_count == 2  # verifies the loop ran exactly two turns

Mocking the LLM at the API client level lets you test the full tool execution loop, schema parsing, function dispatch, result formatting, without spending API credits or depending on model behavior.