Tool Calling

Tool calling (also called function calling) lets an AI model request that your application execute a specific function and return the result. The model decides when to use a tool, what arguments to pass, and how to incorporate the result into its final response.

Routeway supports the OpenAI tool calling format. Any code that works with OpenAI function calling works with Routeway unchanged — just swap the base_url.

How It Works

Define your tools in the request as JSON schemas.
The model returns a tool_calls array when it wants to call a function.
Your app executes the function and sends back a role: "tool" message.
The model generates its final answer using the result.

Basic Example

Python
Node.js
cURL

import os
import json
from openai import OpenAI

client = OpenAI(
    base_url="https://api.routeway.ai/v1",
    api_key=os.getenv("ROUTEWAY_API_KEY")
)

# Step 1: Define tools
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a given city.",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {
                        "type": "string",
                        "description": "The city name, e.g. 'London'"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "Temperature unit"
                    }
                },
                "required": ["city"],
                "additionalProperties": False
            },
            "strict": True
        }
    }
]

messages = [{"role": "user", "content": "What's the weather like in Paris right now?"}]

# Step 2: Send request with tools
response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    tools=tools,
    tool_choice="auto",
)

message = response.choices[0].message

# Step 3: Check if model wants to call a tool
if message.tool_calls:
    for tool_call in message.tool_calls:
        args = json.loads(tool_call.function.arguments)
        print(f"Model wants to call: {tool_call.function.name}({args})")

        # Step 4: Execute the function (your logic here)
        result = {"temperature": 18, "conditions": "Partly cloudy", "city": args["city"]}

        # Step 5: Send result back
        messages.append(message)
        messages.append({
            "role": "tool",
            "tool_call_id": tool_call.id,
            "content": json.dumps(result)
        })

    # Step 6: Get final answer
    final = client.chat.completions.create(
        model="gpt-4o",
        messages=messages,
        tools=tools,
    )
    print(final.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.routeway.ai/v1",
  apiKey: process.env.ROUTEWAY_API_KEY,
});

const tools = [
  {
    type: "function",
    function: {
      name: "get_weather",
      description: "Get the current weather for a given city.",
      parameters: {
        type: "object",
        properties: {
          city: { type: "string", description: "The city name" },
          unit: { type: "string", enum: ["celsius", "fahrenheit"] },
        },
        required: ["city"],
        additionalProperties: false,
      },
      strict: true,
    },
  },
];

async function main() {
  const messages = [
    { role: "user", content: "What's the weather like in Paris right now?" },
  ];

  const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages,
    tools,
    tool_choice: "auto",
  });

  const message = response.choices[0].message;

  if (message.tool_calls) {
    for (const toolCall of message.tool_calls) {
      const args = JSON.parse(toolCall.function.arguments);
      console.log(`Calling: ${toolCall.function.name}`, args);

      // Your function logic here
      const result = { temperature: 18, conditions: "Partly cloudy", city: args.city };

      messages.push(message);
      messages.push({
        role: "tool",
        tool_call_id: toolCall.id,
        content: JSON.stringify(result),
      });
    }

    const final = await client.chat.completions.create({
      model: "gpt-4o",
      messages,
      tools,
    });

    console.log(final.choices[0].message.content);
  }
}

main().catch(console.error);

# Step 1 — send request with tools
curl https://api.routeway.ai/v1/chat/completions \
  -H "Authorization: Bearer $ROUTEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {"role": "user", "content": "What is the weather in Paris?"}
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_weather",
          "description": "Get current weather for a city.",
          "parameters": {
            "type": "object",
            "properties": {
              "city": {"type": "string"}
            },
            "required": ["city"]
          }
        }
      }
    ],
    "tool_choice": "auto"
  }'

Controlling Tool Use

Use tool_choice to control how the model selects tools.

Value	Behavior
`"auto"`	Model decides whether to call a tool (default)
`"none"`	Model will not call any tools
`"required"`	Model must call at least one tool
`{"type": "function", "function": {"name": "..."}}`	Force a specific tool

{
  "tool_choice": {
    "type": "function",
    "function": {"name": "get_weather"}
  }
}

Parallel Tool Calls

Models may request multiple tools simultaneously. Iterate over message.tool_calls, execute each in parallel, and send all results back before requesting the final answer.

import asyncio
import json

# After receiving a response with multiple tool_calls:
async def execute_all_tools(tool_calls):
    results = await asyncio.gather(*[run_tool(tc) for tc in tool_calls])
    return results

async def run_tool(tool_call):
    args = json.loads(tool_call.function.arguments)
    # dispatch to the right function...
    return {"tool_call_id": tool_call.id, "result": "..."}

Send all role: "tool" messages in the same follow-up request. The model needs all results together to compose its final answer.

Designing Good Tool Schemas

Use explicit, stable names

Tool names should clearly describe the action. The model uses the name and description to decide when to call the tool.

// Good
"name": "search_knowledge_base"

// Too vague
"name": "search"

Write clear descriptions

Descriptions directly influence the model’s decision to call a tool. Be specific about what the tool does and when to use it.

"description": "Search the internal knowledge base for articles about a topic. Use this when the user asks about product features, pricing, or support issues."

Keep schemas narrow

Avoid free-form string fields when enums or typed fields work. Tight schemas reduce parsing errors and improve reliability.

// Good
"status": {"type": "string", "enum": ["open", "closed", "pending"]}

// Fragile
"status": {"type": "string", "description": "open, closed, or pending"}

Return structured results

Return JSON objects from your functions instead of plain strings. The model can reason over structured data more reliably.

// Good
{"temperature": 18, "unit": "celsius", "conditions": "Partly cloudy"}

// Weaker
"It's 18°C and partly cloudy"

Common Mistakes

Don’t expose tools with dangerous side effects without application-level guards. The model can call tools in unexpected sequences. Always validate arguments before executing writes, deletions, or external API calls.

Forgetting to replay the assistant message — the messages array must include the assistant turn containing tool_calls before any role: "tool" messages.
Parsing arguments before the stream ends — when streaming, accumulate delta.tool_calls fragments before calling JSON.parse.
Assuming a final text response always follows — on the first turn, the model may return only tool calls with no content. Handle both cases.

Get Started

Platform Details

Help & Resources

Core Concepts

Account Management

Guides

How It Works

Basic Example

Controlling Tool Use

Parallel Tool Calls

Designing Good Tool Schemas

Common Mistakes

​How It Works

​Basic Example

​Controlling Tool Use

​Parallel Tool Calls

​Designing Good Tool Schemas

​Common Mistakes

How It Works

Basic Example

Controlling Tool Use

Parallel Tool Calls

Designing Good Tool Schemas

Common Mistakes