Skip to main content
Tool calling (also called function calling) lets an AI model request that your application execute a specific function and return the result. The model decides when to use a tool, what arguments to pass, and how to incorporate the result into its final response.
Routeway supports the OpenAI tool calling format. Any code that works with OpenAI function calling works with Routeway unchanged — just swap the base_url.

​
How It Works

  1. Define your tools in the request as JSON schemas.
  2. The model returns a tool_calls array when it wants to call a function.
  3. Your app executes the function and sends back a role: "tool" message.
  4. The model generates its final answer using the result.

​
Basic Example

import os
import json
from openai import OpenAI

client = OpenAI(
    base_url="https://api.routeway.ai/v1",
    api_key=os.getenv("ROUTEWAY_API_KEY")
)

# Step 1: Define tools
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a given city.",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {
                        "type": "string",
                        "description": "The city name, e.g. 'London'"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "Temperature unit"
                    }
                },
                "required": ["city"],
                "additionalProperties": False
            },
            "strict": True
        }
    }
]

messages = [{"role": "user", "content": "What's the weather like in Paris right now?"}]

# Step 2: Send request with tools
response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    tools=tools,
    tool_choice="auto",
)

message = response.choices[0].message

# Step 3: Check if model wants to call a tool
if message.tool_calls:
    for tool_call in message.tool_calls:
        args = json.loads(tool_call.function.arguments)
        print(f"Model wants to call: {tool_call.function.name}({args})")

        # Step 4: Execute the function (your logic here)
        result = {"temperature": 18, "conditions": "Partly cloudy", "city": args["city"]}

        # Step 5: Send result back
        messages.append(message)
        messages.append({
            "role": "tool",
            "tool_call_id": tool_call.id,
            "content": json.dumps(result)
        })

    # Step 6: Get final answer
    final = client.chat.completions.create(
        model="gpt-4o",
        messages=messages,
        tools=tools,
    )
    print(final.choices[0].message.content)

​
Controlling Tool Use

Use tool_choice to control how the model selects tools.
ValueBehavior
"auto"Model decides whether to call a tool (default)
"none"Model will not call any tools
"required"Model must call at least one tool
{"type": "function", "function": {"name": "..."}}Force a specific tool
{
  "tool_choice": {
    "type": "function",
    "function": {"name": "get_weather"}
  }
}

​
Parallel Tool Calls

Models may request multiple tools simultaneously. Iterate over message.tool_calls, execute each in parallel, and send all results back before requesting the final answer. 
import asyncio
import json

# After receiving a response with multiple tool_calls:
async def execute_all_tools(tool_calls):
    results = await asyncio.gather(*[run_tool(tc) for tc in tool_calls])
    return results

async def run_tool(tool_call):
    args = json.loads(tool_call.function.arguments)
    # dispatch to the right function...
    return {"tool_call_id": tool_call.id, "result": "..."}
Send all role: "tool" messages in the same follow-up request. The model needs all results together to compose its final answer.

​
Designing Good Tool Schemas

Tool names should clearly describe the action. The model uses the name and description to decide when to call the tool.
// Good
"name": "search_knowledge_base"

// Too vague
"name": "search"
Descriptions directly influence the model’s decision to call a tool. Be specific about what the tool does and when to use it.
"description": "Search the internal knowledge base for articles about a topic. Use this when the user asks about product features, pricing, or support issues."
Avoid free-form string fields when enums or typed fields work. Tight schemas reduce parsing errors and improve reliability.
// Good
"status": {"type": "string", "enum": ["open", "closed", "pending"]}

// Fragile
"status": {"type": "string", "description": "open, closed, or pending"}
Return JSON objects from your functions instead of plain strings. The model can reason over structured data more reliably.
// Good
{"temperature": 18, "unit": "celsius", "conditions": "Partly cloudy"}

// Weaker
"It's 18°C and partly cloudy"

​
Common Mistakes

Don’t expose tools with dangerous side effects without application-level guards. The model can call tools in unexpected sequences. Always validate arguments before executing writes, deletions, or external API calls.
  • Forgetting to replay the assistant message — the messages array must include the assistant turn containing tool_calls before any role: "tool" messages.
  • Parsing arguments before the stream ends — when streaming, accumulate delta.tool_calls fragments before calling JSON.parse.
  • Assuming a final text response always follows — on the first turn, the model may return only tool calls with no content. Handle both cases.