> ## Documentation Index
> Fetch the complete documentation index at: https://docs.routeway.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Tool Calling

> Let models invoke your functions to fetch live data and take actions.

Tool calling (also called function calling) lets an AI model request that your application execute a specific function and return the result. The model decides when to use a tool, what arguments to pass, and how to incorporate the result into its final response.

> [!NOTE]
>
> Routeway supports the OpenAI tool calling format. Any code that works with OpenAI function calling works with Routeway unchanged — just swap the `base_url`.

## How It Works

```mermaid theme={null}
sequenceDiagram
    participant App
    participant Routeway
    participant Model

    App->>Routeway: Request + tools[]
    Routeway->>Model: Forward request
    Model-->>Routeway: tool_calls (name + arguments)
    Routeway-->>App: Response with tool_calls
    App->>App: Execute function locally
    App->>Routeway: New request with tool result
    Routeway->>Model: Continue generation
    Model-->>Routeway: Final answer
    Routeway-->>App: Final response
```

1. Define your tools in the request as JSON schemas.
2. The model returns a `tool_calls` array when it wants to call a function.
3. Your app executes the function and sends back a `role: "tool"` message.
4. The model generates its final answer using the result.

***

## Basic Example

> [tab] Python
>
> ```python theme={null}
> import os
> import json
> from openai import OpenAI
>
> client = OpenAI(
>     base_url="https://api.routeway.ai/v1",
>     api_key=os.getenv("ROUTEWAY_API_KEY")
> )
>
> # Step 1: Define tools
> tools = [
>     {
>         "type": "function",
>         "function": {
>             "name": "get_weather",
>             "description": "Get the current weather for a given city.",
>             "parameters": {
>                 "type": "object",
>                 "properties": {
>                     "city": {
>                         "type": "string",
>                         "description": "The city name, e.g. 'London'"
>                     },
>                     "unit": {
>                         "type": "string",
>                         "enum": ["celsius", "fahrenheit"],
>                         "description": "Temperature unit"
>                     }
>                 },
>                 "required": ["city"],
>                 "additionalProperties": False
>             },
>             "strict": True
>         }
>     }
> ]
>
> messages = [{"role": "user", "content": "What's the weather like in Paris right now?"}]
>
> # Step 2: Send request with tools
> response = client.chat.completions.create(
>     model="gpt-4o",
>     messages=messages,
>     tools=tools,
>     tool_choice="auto",
> )
>
> message = response.choices[0].message
>
> # Step 3: Check if model wants to call a tool
> if message.tool_calls:
>     for tool_call in message.tool_calls:
>         args = json.loads(tool_call.function.arguments)
>         print(f"Model wants to call: {tool_call.function.name}({args})")
>
>         # Step 4: Execute the function (your logic here)
>         result = {"temperature": 18, "conditions": "Partly cloudy", "city": args["city"]}
>
>         # Step 5: Send result back
>         messages.append(message)
>         messages.append({
>             "role": "tool",
>             "tool_call_id": tool_call.id,
>             "content": json.dumps(result)
>         })
>
>     # Step 6: Get final answer
>     final = client.chat.completions.create(
>         model="gpt-4o",
>         messages=messages,
>         tools=tools,
>     )
>     print(final.choices[0].message.content)
> ```

> [tab] Node.js
>
> ```javascript theme={null}
> import OpenAI from "openai";
>
> const client = new OpenAI({
>   baseURL: "https://api.routeway.ai/v1",
>   apiKey: process.env.ROUTEWAY_API_KEY,
> });
>
> const tools = [
>   {
>     type: "function",
>     function: {
>       name: "get_weather",
>       description: "Get the current weather for a given city.",
>       parameters: {
>         type: "object",
>         properties: {
>           city: { type: "string", description: "The city name" },
>           unit: { type: "string", enum: ["celsius", "fahrenheit"] },
>         },
>         required: ["city"],
>         additionalProperties: false,
>       },
>       strict: true,
>     },
>   },
> ];
>
> async function main() {
>   const messages = [
>     { role: "user", content: "What's the weather like in Paris right now?" },
>   ];
>
>   const response = await client.chat.completions.create({
>     model: "gpt-4o",
>     messages,
>     tools,
>     tool_choice: "auto",
>   });
>
>   const message = response.choices[0].message;
>
>   if (message.tool_calls) {
>     for (const toolCall of message.tool_calls) {
>       const args = JSON.parse(toolCall.function.arguments);
>       console.log(`Calling: ${toolCall.function.name}`, args);
>
>       // Your function logic here
>       const result = { temperature: 18, conditions: "Partly cloudy", city: args.city };
>
>       messages.push(message);
>       messages.push({
>         role: "tool",
>         tool_call_id: toolCall.id,
>         content: JSON.stringify(result),
>       });
>     }
>
>     const final = await client.chat.completions.create({
>       model: "gpt-4o",
>       messages,
>       tools,
>     });
>
>     console.log(final.choices[0].message.content);
>   }
> }
>
> main().catch(console.error);
> ```

> [tab] cURL
>
> ```bash theme={null}
> # Step 1 — send request with tools
> curl https://api.routeway.ai/v1/chat/completions \
>   -H "Authorization: Bearer $ROUTEWAY_API_KEY" \
>   -H "Content-Type: application/json" \
>   -d '{
>     "model": "gpt-4o",
>     "messages": [
>       {"role": "user", "content": "What is the weather in Paris?"}
>     ],
>     "tools": [
>       {
>         "type": "function",
>         "function": {
>           "name": "get_weather",
>           "description": "Get current weather for a city.",
>           "parameters": {
>             "type": "object",
>             "properties": {
>               "city": {"type": "string"}
>             },
>             "required": ["city"]
>           }
>         }
>       }
>     ],
>     "tool_choice": "auto"
>   }'
> ```

***

## Controlling Tool Use

Use `tool_choice` to control how the model selects tools.

| Value | Behavior |
| - | - |
| `"auto"` | Model decides whether to call a tool (default) |
| `"none"` | Model will not call any tools |
| `"required"` | Model must call at least one tool |
| `{"type": "function", "function": {"name": "..."}}` | Force a specific tool |

```json theme={null}
{
  "tool_choice": {
    "type": "function",
    "function": {"name": "get_weather"}
  }
}
```

***

## Parallel Tool Calls

Models may request multiple tools simultaneously. Iterate over `message.tool_calls`, execute each in parallel, and send all results back before requesting the final answer.

```python theme={null}
import asyncio
import json

# After receiving a response with multiple tool_calls:
async def execute_all_tools(tool_calls):
    results = await asyncio.gather(*[run_tool(tc) for tc in tool_calls])
    return results

async def run_tool(tool_call):
    args = json.loads(tool_call.function.arguments)
    # dispatch to the right function...
    return {"tool_call_id": tool_call.id, "result": "..."}
```

> [!TIP]
>
> Send all `role: "tool"` messages in the same follow-up request. The model needs all results together to compose its final answer.

***

## Designing Good Tool Schemas

> [accordion] Use explicit, stable names
>
> Tool names should clearly describe the action. The model uses the name and description to decide when to call the tool.
>
> ```json theme={null}
> // Good
> "name": "search_knowledge_base"
>
> // Too vague
> "name": "search"
> ```

> [accordion] Write clear descriptions
>
> Descriptions directly influence the model's decision to call a tool. Be specific about what the tool does and when to use it.
>
> ```json theme={null}
> "description": "Search the internal knowledge base for articles about a topic. Use this when the user asks about product features, pricing, or support issues."
> ```

> [accordion] Keep schemas narrow
>
> Avoid free-form string fields when enums or typed fields work. Tight schemas reduce parsing errors and improve reliability.
>
> ```json theme={null}
> // Good
> "status": {"type": "string", "enum": ["open", "closed", "pending"]}
>
> // Fragile
> "status": {"type": "string", "description": "open, closed, or pending"}
> ```

> [accordion] Return structured results
>
> Return JSON objects from your functions instead of plain strings. The model can reason over structured data more reliably.
>
> ```json theme={null}
> // Good
> {"temperature": 18, "unit": "celsius", "conditions": "Partly cloudy"}
>
> // Weaker
> "It's 18°C and partly cloudy"
> ```

***

## Common Mistakes

> [!WARNING]
>
> **Don't expose tools with dangerous side effects without application-level guards.** The model can call tools in unexpected sequences. Always validate arguments before executing writes, deletions, or external API calls.

* **Forgetting to replay the assistant message** — the `messages` array must include the assistant turn containing `tool_calls` before any `role: "tool"` messages.
* **Parsing arguments before the stream ends** — when streaming, accumulate `delta.tool_calls` fragments before calling `JSON.parse`.
* **Assuming a final text response always follows** — on the first turn, the model may return *only* tool calls with no `content`. Handle both cases.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.