# API Key Management
Source: https://docs.routeway.ai/account-management/api-key-management
Create, list, update, and delete API keys programmatically.
Routeway's key management endpoints let you create and configure API keys without visiting the dashboard. This is essential for automating key rotation, provisioning keys for new services, and enforcing per-key spending limits.
All endpoints on this page require a **management key**. Create one from [Dashboard → Management Keys](https://routeway.ai/dashboard/management-keys).
The full key value is only returned once — when you create it. After that, the key is masked in list and detail responses. Store it securely immediately after creation.
## List All Keys
`GET /v1/account/keys` returns all API keys for your account with masked values.
```python theme={null}
import os
import requests
headers = {"Authorization": f"Bearer {os.getenv('ROUTEWAY_MANAGEMENT_KEY')}"}
response = requests.get("https://api.routeway.ai/v1/account/keys", headers=headers)
keys = response.json()["data"]
for key in keys:
print(f"ID: {key['api_key_id']} | Label: {key['label']} | Key: {key['key']}")
```
```javascript theme={null}
const response = await fetch("https://api.routeway.ai/v1/account/keys", {
headers: { Authorization: `Bearer ${process.env.ROUTEWAY_MANAGEMENT_KEY}` },
});
const { data: keys } = await response.json();
for (const key of keys) {
console.log(`ID: ${key.api_key_id} | Label: ${key.label} | Key: ${key.key}`);
}
```
```bash theme={null}
curl https://api.routeway.ai/v1/account/keys \
-H "Authorization: Bearer $ROUTEWAY_MANAGEMENT_KEY"
```
**Response:**
```json theme={null}
{
"data": [
{
"api_key_id": 1,
"key": "rw-abc1234...5678",
"label": "production",
"daily_limit": 1000,
"created_at": "2025-06-01T00:00:00Z"
},
{
"api_key_id": 2,
"key": "rw-def5678...9012",
"label": "staging",
"daily_limit": 100,
"created_at": "2025-06-10T00:00:00Z"
}
]
}
```
***
## Create a Key
`POST /v1/account/keys` creates a new API key with optional configuration.
### Parameters
| Parameter | Type | Description |
| ------------------ | --------- | ------------------------------------------------------- |
| `label` | string | Human-readable name (e.g. `"production"`, `"staging"`) |
| `daily_limit` | integer | Maximum spend or requests per day |
| `minute_limit` | integer | Maximum requests per minute |
| `allowed_models` | string\[] | Whitelist of model IDs this key can access |
| `balance` | number | Pre-allocated balance (in dollars) attached to this key |
| `use_subs_first` | boolean | Consume subscription quota before PAYG credit |
| `fallback_to_payg` | boolean | Fall back to PAYG when subscription quota is exhausted |
```python theme={null}
import os
import requests
headers = {
"Authorization": f"Bearer {os.getenv('ROUTEWAY_MANAGEMENT_KEY')}",
"Content-Type": "application/json"
}
response = requests.post(
"https://api.routeway.ai/v1/account/keys",
headers=headers,
json={
"label": "backend-service",
"daily_limit": 500,
"minute_limit": 30,
"allowed_models": ["gpt-4o", "gpt-4o-mini", "claude-sonnet-4-20250514"]
}
)
new_key = response.json()["data"]
print(f"Key created: {new_key['key']}") # Save this! Only shown once.
print(f"ID: {new_key['api_key_id']}")
```
```javascript theme={null}
const response = await fetch("https://api.routeway.ai/v1/account/keys", {
method: "POST",
headers: {
Authorization: `Bearer ${process.env.ROUTEWAY_MANAGEMENT_KEY}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
label: "backend-service",
daily_limit: 500,
minute_limit: 30,
allowed_models: ["gpt-4o", "gpt-4o-mini", "claude-sonnet-4-20250514"],
}),
});
const { data: newKey } = await response.json();
console.log(`Key created: ${newKey.key}`); // Save this! Only shown once.
console.log(`ID: ${newKey.api_key_id}`);
```
```bash theme={null}
curl https://api.routeway.ai/v1/account/keys \
-H "Authorization: Bearer $ROUTEWAY_MANAGEMENT_KEY" \
-H "Content-Type: application/json" \
-d '{
"label": "backend-service",
"daily_limit": 500,
"minute_limit": 30,
"allowed_models": ["gpt-4o", "gpt-4o-mini", "claude-sonnet-4-20250514"]
}'
```
The full API key value is only returned in the creation response. Copy and store it in a secrets manager immediately — you cannot retrieve it again.
***
## Get Key Details
`GET /v1/account/keys/{key_id}` returns detailed information about a specific key, including current usage.
```python theme={null}
key_id = 1
response = requests.get(
f"https://api.routeway.ai/v1/account/keys/{key_id}",
headers=headers
)
detail = response.json()["data"]
print(f"Label: {detail['label']}")
print(f"Usage today: ${detail['usage_today']:.2f}")
print(f"Daily limit: {detail['daily_limit']}")
print(f"Last used: {detail['last_used_at']}")
```
```bash theme={null}
curl https://api.routeway.ai/v1/account/keys/1 \
-H "Authorization: Bearer $ROUTEWAY_MANAGEMENT_KEY"
```
**Response:**
```json theme={null}
{
"data": {
"api_key_id": 1,
"key": "rw-abc1234...5678",
"label": "production",
"daily_limit": 1000,
"minute_limit": 60,
"balance": 10.00,
"usage_today": 3.21,
"usage_minute": 0,
"created_at": "2025-06-01T00:00:00Z",
"last_used_at": "2025-06-15T11:00:00Z",
"use_subs_first": true,
"fallback_to_payg": false
}
}
```
### Key Detail Fields
| Field | Description |
| ------------------ | ----------------------------------------------- |
| `api_key_id` | Unique identifier for the key |
| `key` | Masked key value |
| `label` | Human-readable name |
| `daily_limit` | Maximum spend/requests per day |
| `minute_limit` | Maximum requests per minute |
| `balance` | Pre-allocated balance on this key |
| `usage_today` | Amount spent today |
| `usage_minute` | Requests made in the current minute |
| `created_at` | When the key was created |
| `last_used_at` | When the key was last used for a request |
| `use_subs_first` | Whether subscription quota is used before PAYG |
| `fallback_to_payg` | Whether PAYG is used when subscription runs out |
***
## Update a Key
`PATCH /v1/account/keys/{key_id}` updates one or more settings on an existing key. Only include the fields you want to change.
```python theme={null}
key_id = 1
response = requests.patch(
f"https://api.routeway.ai/v1/account/keys/{key_id}",
headers=headers,
json={
"label": "production-v2",
"daily_limit": 2000,
"allowed_models": ["gpt-4o", "gpt-4o-mini"]
}
)
updated = response.json()["data"]
print(f"Updated key {updated['api_key_id']}: {updated['label']}")
```
```bash theme={null}
curl -X PATCH https://api.routeway.ai/v1/account/keys/1 \
-H "Authorization: Bearer $ROUTEWAY_MANAGEMENT_KEY" \
-H "Content-Type: application/json" \
-d '{
"label": "production-v2",
"daily_limit": 2000
}'
```
Use updates to adjust limits dynamically — for example, increase a key's `daily_limit` during a traffic spike, or tighten `allowed_models` for a service that should only use cheap models.
***
## Delete a Key
`DELETE /v1/account/keys/{key_id}` permanently deletes a key. Any requests using this key will immediately start returning `401`.
```python theme={null}
key_id = 2
response = requests.delete(
f"https://api.routeway.ai/v1/account/keys/{key_id}",
headers=headers
)
print(response.json()["message"]) # "API key successfully deleted."
```
```bash theme={null}
curl -X DELETE https://api.routeway.ai/v1/account/keys/2 \
-H "Authorization: Bearer $ROUTEWAY_MANAGEMENT_KEY"
```
**Response:**
```json theme={null}
{
"message": "API key successfully deleted."
}
```
Deletion is permanent and immediate. Any application using this key will lose access instantly. Make sure to rotate keys gracefully — create the new key first, update your services, then delete the old one.
***
## Key Rotation Pattern
A safe key rotation workflow:
```python theme={null}
import os
import requests
headers = {
"Authorization": f"Bearer {os.getenv('ROUTEWAY_MANAGEMENT_KEY')}",
"Content-Type": "application/json"
}
# 1. Create new key with same settings
response = requests.post(
"https://api.routeway.ai/v1/account/keys",
headers=headers,
json={"label": "production-rotated", "daily_limit": 1000}
)
new_key = response.json()["data"]["key"]
print(f"New key: {new_key}")
# 2. Deploy new key to your services
# ... update secrets manager, redeploy, etc.
# 3. Verify the new key is working
# ... check activity logs for requests using the new key
# 4. Delete the old key
old_key_id = 1
requests.delete(
f"https://api.routeway.ai/v1/account/keys/{old_key_id}",
headers=headers
)
print(f"Old key {old_key_id} deleted.")
```
***
## Best Practices
Name keys after the service or environment they belong to (e.g. `"prod-backend"`, `"staging-worker"`). This makes it easy to audit usage.
Always configure a `daily_limit` on production keys to prevent runaway costs from bugs or attacks.
Use `allowed_models` to limit expensive models to keys that actually need them. A chatbot key probably doesn't need access to image generation models.
Rotate keys periodically (e.g. monthly) and immediately if a key may have been exposed.
# Balance & Usage
Source: https://docs.routeway.ai/account-management/balance-and-usage
Monitor your credit balance, review request history, and query usage analytics.
Routeway's account endpoints let you programmatically track spending and usage without visiting the dashboard. This is useful for building internal dashboards, setting up alerts, or auditing costs.
All endpoints on this page require a **management key**. Create one from [Dashboard → Management Keys](https://routeway.ai/dashboard/management-keys).
## Check Your Balance
`GET /v1/account/balance` returns your current credit balance in dollars.
```python theme={null}
import os
import requests
headers = {"Authorization": f"Bearer {os.getenv('ROUTEWAY_MANAGEMENT_KEY')}"}
response = requests.get("https://api.routeway.ai/v1/account/balance", headers=headers)
data = response.json()
print(f"Current balance: ${data['balance']:.2f}")
```
```javascript theme={null}
const response = await fetch("https://api.routeway.ai/v1/account/balance", {
headers: { Authorization: `Bearer ${process.env.ROUTEWAY_MANAGEMENT_KEY}` },
});
const { balance } = await response.json();
console.log(`Current balance: $${balance.toFixed(2)}`);
```
```bash theme={null}
curl https://api.routeway.ai/v1/account/balance \
-H "Authorization: Bearer $ROUTEWAY_MANAGEMENT_KEY"
```
**Response:**
```json theme={null}
{
"balance": 4.20
}
```
Use this endpoint to implement low-balance alerts. Poll periodically and notify your team when the balance drops below a threshold.
***
## Request Activity
`GET /v1/account/activity` returns a paginated list of individual request logs, including the model used, cost, and timestamp.
### Parameters
| Parameter | Type | Default | Description |
| --------- | ------- | ------- | ----------------------------------- |
| `limit` | integer | `50` | Number of records to return (1–200) |
| `offset` | integer | `0` | Offset for pagination |
```python theme={null}
import os
import requests
headers = {"Authorization": f"Bearer {os.getenv('ROUTEWAY_MANAGEMENT_KEY')}"}
response = requests.get(
"https://api.routeway.ai/v1/account/activity",
headers=headers,
params={"limit": 10, "offset": 0}
)
data = response.json()
for entry in data["data"]:
print(f"{entry['created_at']} | {entry['model']:20s} | ${entry['cost']:.5f}")
print(f"\nShowing {len(data['data'])} of results (offset: {data['offset']})")
```
```javascript theme={null}
const params = new URLSearchParams({ limit: "10", offset: "0" });
const response = await fetch(
`https://api.routeway.ai/v1/account/activity?${params}`,
{ headers: { Authorization: `Bearer ${process.env.ROUTEWAY_MANAGEMENT_KEY}` } }
);
const { data, limit, offset } = await response.json();
for (const entry of data) {
console.log(`${entry.created_at} | ${entry.model.padEnd(20)} | $${entry.cost.toFixed(5)}`);
}
```
```bash theme={null}
curl "https://api.routeway.ai/v1/account/activity?limit=10&offset=0" \
-H "Authorization: Bearer $ROUTEWAY_MANAGEMENT_KEY"
```
**Response:**
```json theme={null}
{
"data": [
{
"model": "gpt-4o",
"cost": 0.00042,
"created_at": "2025-06-15T12:00:00Z"
},
{
"model": "claude-sonnet-4-20250514",
"cost": 0.00128,
"created_at": "2025-06-15T11:58:30Z"
}
],
"limit": 10,
"offset": 0
}
```
### Pagination
To fetch all activity, increment `offset` by `limit` until the returned `data` array is empty:
```python theme={null}
all_activity = []
offset = 0
limit = 200
while True:
response = requests.get(
"https://api.routeway.ai/v1/account/activity",
headers=headers,
params={"limit": limit, "offset": offset}
)
page = response.json()
all_activity.extend(page["data"])
if len(page["data"]) < limit:
break
offset += limit
print(f"Total requests: {len(all_activity)}")
```
***
## Usage Analytics
`GET /v1/account/analytics` returns aggregated usage metrics over a time window. Results are cached server-side for 5 minutes.
### Parameters
| Parameter | Type | Default | Description |
| --------- | ------ | ------- | ------------------------------------------------- |
| `period` | string | `"30d"` | Time window: `"24h"`, `"7d"`, `"30d"`, or `"all"` |
| `model` | string | — | Filter metrics to a specific model ID |
```python theme={null}
import os
import requests
headers = {"Authorization": f"Bearer {os.getenv('ROUTEWAY_MANAGEMENT_KEY')}"}
# Get last 7 days of usage
response = requests.get(
"https://api.routeway.ai/v1/account/analytics",
headers=headers,
params={"period": "7d"}
)
# Check cache status
cache_status = response.headers.get("X-Cache", "MISS")
print(f"Cache: {cache_status}")
analytics = response.json()
print(analytics)
```
```javascript theme={null}
const response = await fetch(
"https://api.routeway.ai/v1/account/analytics?period=7d",
{ headers: { Authorization: `Bearer ${process.env.ROUTEWAY_MANAGEMENT_KEY}` } }
);
const cacheStatus = response.headers.get("X-Cache") ?? "MISS";
console.log(`Cache: ${cacheStatus}`);
const analytics = await response.json();
console.log(analytics);
```
```bash theme={null}
# Last 7 days, all models
curl "https://api.routeway.ai/v1/account/analytics?period=7d" \
-H "Authorization: Bearer $ROUTEWAY_MANAGEMENT_KEY"
# Last 24 hours, specific model
curl "https://api.routeway.ai/v1/account/analytics?period=24h&model=gpt-4o" \
-H "Authorization: Bearer $ROUTEWAY_MANAGEMENT_KEY"
```
The `X-Cache` response header tells you whether the result was served from cache (`HIT`) or freshly computed (`MISS`). Analytics are cached for 5 minutes, so rapid repeated calls will return the same data without extra processing.
***
## Rate Limits
You can find more information about rate limits [here](/account-management/overview).
***
## Common Use Cases
Poll `/v1/account/balance` every few minutes and send a Slack/email alert when balance drops below a threshold.
Use `/v1/account/activity` to break down costs by model and time period for internal billing or chargeback.
Feed `/v1/account/analytics` into Grafana, Datadog, or a custom dashboard to visualize trends.
Combine balance checks with key management to auto-disable keys when a budget threshold is reached.
# Overview
Source: https://docs.routeway.ai/account-management/overview
Monitor usage, check balances, and manage API keys programmatically.
Routeway provides a set of account endpoints that let you monitor spending, review request history, and manage API keys — all without leaving your codebase or touching the dashboard.
Account endpoints require a **management key**, not a regular API key. Management keys are separate credentials with access to account operations. Create one from [Dashboard → Management Keys](https://routeway.ai/dashboard/management-keys).
## Available Endpoints
### Usage & Billing
| Method | Path | Description |
| ------------------ | --------------------------------------------------------- | -------------------------------------------- |
| GET | [/v1/account/balance](/api-reference/Account/balance) | Current credit balance |
| GET | [/v1/account/activity](/api-reference/Account/activity) | Paginated request history with cost per call |
| GET | [/v1/account/analytics](/api-reference/Account/analytics) | Aggregated usage metrics over time |
### API Keys
| Method | Path | Description |
| --------------------- | ---------------------------------------------------------- | ------------------------------ |
| GET | [/v1/account/keys](/api-reference/Account/keys-list) | List all API keys |
| POST | [/v1/account/keys](/api-reference/Account/keys-create) | Create a new API key |
| GET | [/v1/account/keys/:id](/api-reference/Account/keys-get) | Get details for a specific key |
| PATCH | [/v1/account/keys/:id](/api-reference/Account/keys-update) | Update key settings |
| DELETE | [/v1/account/keys/:id](/api-reference/Account/keys-delete) | Delete an API key |
***
## Rate Limits
Routeway applies rate limits per endpoint to ensure system stability and fair usage across all users.
Each endpoint is governed by two limits:
* **Burst limit** — maximum number of requests allowed within a 10-second window
* **Sustained limit** — maximum number of requests allowed per minute
When a limit is exceeded, the API returns a `429 Too Many Requests` response. Clients should implement retry logic with exponential backoff.
| Method | Endpoint | Burst (10s) | Per Minute |
| -------------------- | ---------------------------------------------------------- | ----------: | ---------: |
| GET | [/v1/account/balance](/api-reference/Account/balance) | 10 | 120 |
| GET | [/v1/account/activity](/api-reference/Account/activity) | 5 | 60 |
| GET | [/v1/account/analytics](/api-reference/Account/analytics) | 2 | 25 |
| GET | [/v1/account/keys](/api-reference/Account/keys-list) | 5 | 60 |
| GET | [/v1/account/keys/:id](/api-reference/Account/keys-get) | 5 | 60 |
| POST | [/v1/account/keys](/api-reference/Account/keys-create) | 2 | 12 |
| PATCH | [/v1/account/keys/:id](/api-reference/Account/keys-update) | 2 | 15 |
## Quick Example — Check Your Balance
```python theme={null}
import os
import requests
# Account endpoints require a management key
headers = {"Authorization": f"Bearer {os.getenv('ROUTEWAY_MANAGEMENT_KEY')}"}
response = requests.get("https://api.routeway.ai/v1/account/balance", headers=headers)
data = response.json()
print(f"Balance: ${data['balance']:.2f}")
```
```javascript theme={null}
const response = await fetch("https://api.routeway.ai/v1/account/balance", {
headers: {
Authorization: `Bearer ${process.env.ROUTEWAY_MANAGEMENT_KEY}`,
},
});
const data = await response.json();
console.log(`Balance: $${data.balance.toFixed(2)}`);
```
```bash theme={null}
curl https://api.routeway.ai/v1/account/balance \
-H "Authorization: Bearer $ROUTEWAY_MANAGEMENT_KEY"
```
```json theme={null}
{
"balance": 4.20
}
```
***
## What's in This Section
Check your credit balance, review request activity logs, and query aggregated analytics to understand spending patterns.
Create, list, update, and delete API keys programmatically. Configure daily limits, minute limits, model whitelists, and subscription settings per key.
Add Tavily, Exa, or Valyu keys in Settings and use your own search provider with `@online` chat requests.
***
## Authentication
Account endpoints require a **management key** passed as a Bearer token:
```
Authorization: Bearer YOUR_MANAGEMENT_KEY
```
Management keys are separate from regular API keys and can only be created from the [Dashboard → Management Keys](https://routeway.ai/dashboard/management-keys) page.
Regular API keys (used for model requests) cannot access account endpoints. Always use a management key for these operations.
# Get Activity
Source: https://docs.routeway.ai/api-reference/Account/activity
GET /v1/account/activity
Fetch account activity logs. Requires scope: account:activity
For a detailed walkthrough with examples, see the [Balance & Usage guide](/account-management/balance-and-usage).
# Get Analytics
Source: https://docs.routeway.ai/api-reference/Account/analytics
GET /v1/account/analytics
Fetch analytics dashboard. Requires scope: account:read
For a detailed walkthrough with examples, see the [Balance & Usage guide](/account-management/balance-and-usage).
# Get Balance
Source: https://docs.routeway.ai/api-reference/Account/balance
GET /v1/account/balance
Check account balance. Requires scope: account:balance
For a detailed walkthrough with examples, see the [Balance & Usage guide](/account-management/balance-and-usage).
# Create Key
Source: https://docs.routeway.ai/api-reference/Account/keys-create
POST /v1/account/keys
Create a new general API key. Requires scope: keys:create
For a detailed walkthrough with examples, see the [API Key Management guide](/account-management/api-key-management).
# Delete Key
Source: https://docs.routeway.ai/api-reference/Account/keys-delete
DELETE /v1/account/keys/{key_id}
Delete an API key. Requires scope: keys:delete
Deleting a key is permanent. Any application using it will immediately lose API access. Rotate to a new key before deleting the old one.
For a detailed walkthrough with examples, see the [API Key Management guide](/account-management/api-key-management).
# Get Key
Source: https://docs.routeway.ai/api-reference/Account/keys-get
GET /v1/account/keys/{key_id}
Get details of a specific API key. Requires scope: keys:read
For a detailed walkthrough with examples, see the [API Key Management guide](/account-management/api-key-management).
# List Keys
Source: https://docs.routeway.ai/api-reference/Account/keys-list
GET /v1/account/keys
List all general API keys for the account. Requires scope: keys:read
For a detailed walkthrough with examples, see the [API Key Management guide](/account-management/api-key-management).
# Update Key
Source: https://docs.routeway.ai/api-reference/Account/keys-update
PATCH /v1/account/keys/{key_id}
Update an existing API key. Requires scope: keys:update
For a detailed walkthrough with examples, see the [API Key Management guide](/account-management/api-key-management).
# Chat Completions
Source: https://docs.routeway.ai/api-reference/ChatCompletions/completions
POST /v1/chat/completions
Generate a model response for a given chat conversation. Supports streaming and non-streaming responses.
This API endpoint allows you to send text and images as inputs, and the model will generate the next message in the conversation.
## Create Chat Completion
To create a chat completion, use the following endpoint:
`POST /v1/chat/completions`
### Request Body
The model id to use for completion ("gpt-4o", "gpt-4o-mini", "deepseek-r",
[...](/getting-started/models))
An array of message objects that form the conversation
The maximum number of tokens to generate
Sampling temperature, a value between 0 and 2.
Whether to enable streaming responses
Tells the model not to repeat a word that has already been used multiple times
in the conversation.
Prevents the model from repeating a word, even if it's only been used once
Up to 4 sequences where the API will stop generating further tokens. The
returned text will not contain the stop sequence.
```python Python theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("Routeway_API_KEY")
)
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "user", "content": "Write a short story about a robot and a cat."}
]
)
print(response.choices[0].message.content)
```
```javascript Node.js theme={null}
const axios = require('axios')
const apiKey = 'your_routeway_api_key'
const url = 'https://api.routeway.ai/v1/chat/completions'
const data = {
model: 'gpt-4o',
messages: [
{
role: 'user',
content: 'Say this is a test!',
},
],
}
const headers = {
'Content-Type': 'application/json',
Authorization: `Bearer ${apiKey}`,
}
axios
.post(url, data, { headers })
.then(response => {
console.log(response.data)
})
.catch(error => {
console.error('Error:', error)
})
```
```bash cURL theme={null}
curl https://api.routeway.ai/v1/chat/completions \
-H "Authorization: Bearer $Routeway_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o-mini",
"messages": [
{"role": "user", "content": "Write a short story about a robot and a cat"}
]
}'
```
# Image Edits
Source: https://docs.routeway.ai/api-reference/ImageEdits/edits
POST /v1/images/edits
This API endpoint allows you to edit images using AI models. You can provide one or more base64-encoded images along with a text prompt describing the desired edit.
The Image Edits API supports both single and multiple image editing, with optional mask support for targeted edits.
Images stored via URL are only available for 1 hour after generation. Make sure to download and save important images locally.
## Create Image Edit
To edit an image, use the following endpoint:
`POST /v1/images/edits`
### Request Body
Base64-encoded images (PNG format, \< 4MB each)
Base64-encoded mask image (PNG format, \< 4MB) for targeted edits
Text description of the desired edit
Model to use for image editing
Number of images to generate (1-10, default: 1)
Image size (options: "256x256", "512x512", "1024x1024")
Response format (options: "url" or "b64\_json", default: "url")
Transformation strength (0.0-1.0)
Guidance scale for the edit (0.0-20.0)
Number of inference steps (1-100)
Random seed for reproducible results
Kontext max mode configuration
```python Python theme={null}
import os
import base64
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("Routeway_API_KEY")
)
# Read and encode image
with open("input.png", "rb") as image_file:
encoded_image = base64.b64encode(image_file.read()).decode('utf-8')
response = client.images.edit(
images=[encoded_image],
prompt="Add a sunset in the background",
size="1024x1024",
n=1
)
print(response.data[0].url)
```
```javascript Node.js theme={null}
const axios = require('axios')
const fs = require('fs')
const apiKey = 'your_routeway_api_key'
const url = 'https://api.routeway.ai/v1/images/edits'
// Read and encode image
const imageBuffer = fs.readFileSync('input.png')
const encodedImage = imageBuffer.toString('base64')
const data = {
images: [encodedImage],
prompt: 'Add a sunset in the background',
size: '1024x1024',
n: 1
}
const headers = {
'Content-Type': 'application/json',
Authorization: `Bearer ${apiKey}`,
}
axios
.post(url, data, { headers })
.then(response => {
console.log(response.data)
})
.catch(error => {
console.error('Error:', error)
})
```
```bash cURL theme={null}
# First encode your image to base64
ENCODED_IMAGE=$(base64 -i input.png)
curl https://api.routeway.ai/v1/images/edits \
-H "Authorization: Bearer $Routeway_API_KEY" \
-H "Content-Type: application/json" \
-d "{
\"images\": [\"$ENCODED_IMAGE\"],
\"prompt\": \"Add a sunset in the background\",
\"size\": \"1024x1024\",
\"n\": 1
}"
```
### Response
The response includes the edited image(s) with either a URL or base64-encoded data, depending on the `response_format` parameter.
```json theme={null}
{
"created": 1234567890,
"data": [
{
"url": "https://example.com/image.png",
"revised_prompt": "Add a sunset in the background"
}
]
}
```
When `response_format` is set to `b64_json`, the response will include base64-encoded image data instead of a URL:
```json theme={null}
{
"created": 1234567890,
"data": [
{
"b64_json": "iVBORw0KGgoAAAANSUhEUgAA...",
"revised_prompt": "Add a sunset in the background"
}
]
}
```
### Error Response
```json theme={null}
{
"error": {
"message": "Invalid image format",
"type": "invalid_request_error",
"code": 400
}
}
```
# Image Generations
Source: https://docs.routeway.ai/api-reference/ImageGenerations/generations
POST /v1/images/generations
Generate one or more images from a text prompt. Supports both synchronous and asynchronous modes.
This API endpoint allows you to generate images from text prompts using various image generation models.
Images stored via URL are only available for 1 hour after generation. Make sure to download and save important images locally.
## Create Image Generation
To generate an image, use the following endpoint:
`POST /v1/images/generations`
### Request Body
The image generation model to use (e.g., "flux-1-schnell")
The text prompt for image generation
Provider-specific quality preset
Target image resolution (e.g., "1024x1024")
Number of images to generate (default: 1)
```python Python theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("Routeway_API_KEY")
)
response = client.images.generate(
model="flux-1-schnell",
prompt="A beautiful sunset over a mountain lake",
size="1024x1024",
n=1,
quality="standard"
)
print(response.data[0].url)
```
```javascript Node.js theme={null}
const axios = require('axios')
const apiKey = 'your_routeway_api_key'
const url = 'https://api.routeway.ai/v1/images/generations'
const data = {
model: 'flux-1-schnell',
prompt: 'A futuristic city with flying cars',
size: '1024x1024',
n: 1
}
const headers = {
'Content-Type': 'application/json',
Authorization: `Bearer ${apiKey}`,
}
axios
.post(url, data, { headers })
.then(response => {
console.log(response.data)
})
.catch(error => {
console.error('Error:', error)
})
```
```bash cURL theme={null}
curl https://api.routeway.ai/v1/images/generations \
-H "Authorization: Bearer $Routeway_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "flux-1-schnell",
"prompt": "A serene garden with cherry blossoms",
"size": "1024x1024",
"n": 1
}'
```
# Anthropic Messages
Source: https://docs.routeway.ai/api-reference/Messages/messages
POST /v1/messages
Send a message to the Anthropic models. Supports streaming and non-streaming responses.
Send a message to Anthropic models using the native Anthropic Messages API format. Supports streaming and non-streaming responses.
## Create Anthropic Message
To create an Anthropic message, use the following endpoint:
`POST /v1/messages`
This endpoint is compatible with the Anthropic SDK (`anthropic` Python/Node.js package). Point
your SDK at `https://api.routeway.ai` and replace your Anthropic API key with your Routeway
key — no other changes needed.
### Request Body
The Anthropic model ID to use (e.g. `"claude-opus-4-5"`, `"claude-sonnet-4-5"`). See
[Models](/getting-started/models) for the full list of supported Anthropic models.
An array of message objects forming the conversation. Each object must have a `role`
(`"user"` or `"assistant"`) and a `content` field (string or array of content blocks).
The maximum number of tokens to generate before stopping.
A system prompt that provides context and instructions to the model. Can be a plain string
or an array of content blocks.
Sampling temperature between 0 and 1. Higher values produce more varied outputs.
Whether to stream the response as Server-Sent Events (SSE).
Custom text sequences that will cause the model to stop generating.
A list of tools the model may call. Each tool must have a `name`, optional `description`,
and an `input_schema` (JSON Schema object).
Controls tool use. Can be `{"type": "auto"}`, `{"type": "any"}`, or
`{"type": "tool", "name": "..."}`.
Nucleus sampling parameter. Use `temperature` or `top_p`, but not both.
Only sample from the top K options for each subsequent token.
```python Python theme={null}
import os
import anthropic
client = anthropic.Anthropic(
base_url="https://api.routeway.ai",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
message = client.messages.create(
model="claude-opus-4-5",
max_tokens=1024,
messages=[
{"role": "user", "content": "Hello, Claude!"}
]
)
print(message.content[0].text)
```
```javascript Node.js theme={null}
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic({
baseURL: "https://api.routeway.ai",
apiKey: process.env.ROUTEWAY_API_KEY,
});
const message = await client.messages.create({
model: "claude-opus-4-5",
max_tokens: 1024,
messages: [{ role: "user", content: "Hello, Claude!" }],
});
console.log(message.content[0].text);
```
```bash cURL theme={null}
curl https://api.routeway.ai/v1/messages \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "claude-opus-4-5",
"max_tokens": 1024,
"messages": [
{"role": "user", "content": "Hello, Claude!"}
]
}'
```
# Responses
Source: https://docs.routeway.ai/api-reference/Responses/responses
POST /v1/responses
Generate a unified response block from various input configurations.
Generate a unified response block from various input configurations. The Responses API is a higher-level alternative to Chat Completions with built-in tools and optional server-side conversation state.
## Create Response
To create a response, use the following endpoint:
`POST /v1/responses`
### Request Body
The model ID to use for the response (e.g. `"gpt-4o"`, `"gpt-4o-mini"`). Only models that
advertise `/v1/responses` in their `endpoints` array are supported. See
[Models](/getting-started/models) for the full list.
The input for the model. Can be a plain text string or an array of input items (text,
image, file, etc.).
A system-level instruction that guides the model's behavior throughout the response.
Equivalent to the `system` message in Chat Completions.
The maximum number of tokens to generate in the output.
Sampling temperature between 0 and 2. Higher values make output more random; lower values
make it more deterministic.
Whether to stream the response back as Server-Sent Events (SSE).
The ID of a previous response to continue from. Enables multi-turn conversations without
resending full message history.
A list of custom tools the model may call.
Controls whether and how the model calls tools. Can be `"auto"`, `"none"`, `"required"`,
or an object specifying a particular tool.
Whether to store the response server-side so it can be referenced by future requests via
`previous_response_id`. Defaults to `true`.
```python Python theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
response = client.responses.create(
model="gpt-4o",
input="What is the capital of France?"
)
print(response.output_text)
```
```javascript Node.js theme={null}
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.routeway.ai/v1",
apiKey: process.env.ROUTEWAY_API_KEY,
});
const response = await client.responses.create({
model: "gpt-4o",
input: "What is the capital of France?",
});
console.log(response.output_text);
```
```bash cURL theme={null}
curl https://api.routeway.ai/v1/responses \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o",
"input": "What is the capital of France?"
}'
```
# Overview
Source: https://docs.routeway.ai/api-reference/overview
A quick map of every endpoint in the Routeway API.
The Routeway API is fully OpenAI-compatible. Any SDK or tool that works against `api.openai.com` works against `api.routeway.ai` with only a base URL and key change.
All requests must include an `Authorization: Bearer ` header. Obtain your key from the [Dashboard](https://routeway.ai/dashboard/keys).
```
Base URL: https://api.routeway.ai
```
***
## Endpoints
### Inference
| Method | Path | Description |
| ------------------- | --------------------------------------------------------------------- | ------------------------------------------------------------------------------------- |
| POST | [/v1/chat/completions](/api-reference/ChatCompletions/completions) | Text and vision completions — the recommended default for apps, agents, and pipelines |
| POST | [/v1/images/generations](/api-reference/ImageGenerations/generations) | Generate images from a text prompt |
| POST | [/v1/images/edits](/api-reference/ImageEdits/edits) | Edit an existing image with a text prompt and optional mask |
### Account & Keys
| Method | Path | Description |
| --------------------- | ---------------------------------------------------------- | ------------------------------------ |
| GET | [/v1/account/balance](/api-reference/Account/balance) | Retrieve your current credit balance |
| GET | [/v1/account/activity](/api-reference/Account/activity) | Paginated log of usage events |
| GET | [/v1/account/analytics](/api-reference/Account/analytics) | Aggregated usage analytics |
| GET | [/v1/account/keys](/api-reference/Account/keys-list) | List all API keys on your account |
| POST | [/v1/account/keys](/api-reference/Account/keys-create) | Create a new API key |
| GET | [/v1/account/keys/:id](/api-reference/Account/keys-get) | Retrieve a single API key |
| PATCH | [/v1/account/keys/:id](/api-reference/Account/keys-update) | Update an API key |
| DELETE | [/v1/account/keys/:id](/api-reference/Account/keys-delete) | Delete an API key |
***
## Authentication
All endpoints (except a public status check) require a bearer token:
```bash theme={null}
Authorization: Bearer YOUR_ROUTEWAY_API_KEY
```
Keys can be scoped and rate-limited — see [Authentication](/getting-started/authentication) for details.
***
## OpenAI Compatibility
Drop in your existing OpenAI SDK code by changing two values:
```python theme={null}
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1", # changed
api_key="YOUR_ROUTEWAY_API_KEY", # changed
)
```
```javascript theme={null}
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.routeway.ai/v1", // changed
apiKey: process.env.ROUTEWAY_API_KEY, // changed
});
```
Everything else — request shapes, response objects, streaming, tool calls, structured outputs — stays identical.
# Messages & Roles
Source: https://docs.routeway.ai/core-concepts/chat-completions/messages-and-roles
Understand how the message array and roles shape every model response.
Every request to Routeway is built around a `messages` array — an ordered list of turns that represents the full conversation context. The model reads the entire array on each call and generates the next turn.
## Message Structure
Each message object has two required fields:
```json theme={null}
{
"role": "user",
"content": "What is the capital of France?"
}
```
| Field | Type | Description |
| --------- | --------------- | ------------------------------------------------------------ |
| `role` | string | Who sent the message: `system`, `user`, or `assistant` |
| `content` | string \| array | The message text, or a mixed array of text and media objects |
***
## Roles
### `system`
Sets the model's behavior, persona, and constraints for the entire conversation. Processed before any user message, giving it the highest priority.
```json theme={null}
{
"role": "system",
"content": "You are a concise technical assistant. Reply in plain text only, no markdown."
}
```
Use `system` to:
* Define the model's persona or tone
* Restrict the model to a specific domain
* Provide background context or instructions that apply globally
Put your `system` message first and keep it stable across turns. The model treats it as ground truth for the entire session.
### `user`
A message from the end user (or your application acting as the user). This is the primary way to send instructions and content to the model.
```json theme={null}
{
"role": "user",
"content": "Explain how HTTPS works in one paragraph."
}
```
### `assistant`
A message generated by the model. When building multi-turn conversations, append the model's previous responses as `assistant` messages so it can refer back to them.
```json theme={null}
{
"role": "assistant",
"content": "HTTPS uses TLS to encrypt data between a client and server..."
}
```
You can also inject `assistant` messages manually — for example, to prime the model with a specific tone or continue a previous conversation without replaying all prior turns.
***
## Multi-turn Conversations
To maintain context across turns, include all previous messages in each new request. The model has no memory between API calls — the `messages` array *is* the memory.
```python theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
messages = [
{"role": "system", "content": "You are a helpful assistant."}
]
# Turn 1
messages.append({"role": "user", "content": "What's the boiling point of water?"})
response = client.chat.completions.create(model="gpt-4o-mini", messages=messages)
reply = response.choices[0].message.content
messages.append({"role": "assistant", "content": reply})
print(reply)
# Turn 2 — model remembers the previous question
messages.append({"role": "user", "content": "And at high altitude?"})
response = client.chat.completions.create(model="gpt-4o-mini", messages=messages)
print(response.choices[0].message.content)
```
```javascript theme={null}
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.routeway.ai/v1",
apiKey: process.env.ROUTEWAY_API_KEY,
});
const messages = [
{ role: "system", content: "You are a helpful assistant." },
];
async function chat(userMessage) {
messages.push({ role: "user", content: userMessage });
const response = await client.chat.completions.create({
model: "gpt-4o-mini",
messages,
});
const reply = response.choices[0].message.content;
messages.push({ role: "assistant", content: reply });
return reply;
}
console.log(await chat("What's the boiling point of water?"));
console.log(await chat("And at high altitude?"));
```
```bash theme={null}
curl https://api.routeway.ai/v1/chat/completions \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o-mini",
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is the boiling point of water?"},
{"role": "assistant", "content": "Water boils at 100°C (212°F) at sea level."},
{"role": "user", "content": "And at high altitude?"}
]
}'
```
***
## Rich Content Messages
The `content` field can be an array of typed objects instead of a plain string. This is how you pass images, PDFs, and mixed media alongside text.
```json theme={null}
{
"role": "user",
"content": [
{ "type": "text", "text": "What does this chart show?" },
{ "type": "image_url", "image_url": { "url": "https://example.com/chart.png" } }
]
}
```
See [Multimodal Inputs](/guides/chat-completions/multimodal-inputs) for the full reference.
***
## Common Patterns
Give the model a name and personality via the `system` message. Consistent persona instructions reduce drift in long conversations.
```json theme={null}
{
"role": "system",
"content": "You are Aria, a friendly customer support agent for Acme Corp. Always greet users by name when known."
}
```
Prepend example `user`/`assistant` pairs to show the model the exact format you want.
```json theme={null}
[
{ "role": "user", "content": "Classify: 'I love this product!'" },
{ "role": "assistant", "content": "positive" },
{ "role": "user", "content": "Classify: 'Broke after one day.'" },
{ "role": "assistant", "content": "negative" },
{ "role": "user", "content": "Classify: 'It is okay I guess.'" }
]
```
Inject retrieved documents or data into a `user` or `system` message before the question. This is the foundation of RAG (retrieval-augmented generation).
```python theme={null}
context = fetch_relevant_docs(user_query)
messages = [
{"role": "system", "content": "Answer only using the provided context."},
{"role": "user", "content": f"Context:\n{context}\n\nQuestion: {user_query}"}
]
```
***
## Context Window Limits
The `messages` array is bounded by the model's **context window** — the maximum number of tokens it can process in a single request. Exceeding this limit causes a `400` error.
For long conversations, trim old messages from the middle of the array (keeping the `system` message and the most recent turns) or use [Prompt Caching](/guides/chat-completions/caching) to reduce costs when the prefix is stable.
See [Tokens & Context](/core-concepts/chat-completions/tokens-and-context) for more detail.
# Models & Providers
Source: https://docs.routeway.ai/core-concepts/chat-completions/models-and-providers
How Routeway routes requests across providers and how to pick the right model.
Routeway gives you access to 200+ models from providers like OpenAI, Anthropic, Google, Mistral, DeepSeek, xAI, and more — all through a single API key and a single base URL.
## How Routing Works
When you send a request, Routeway inspects the `model` field, resolves the correct upstream provider, forwards your request using that provider's credentials, and returns a normalized response in the standard OpenAI format.
```mermaid theme={null}
flowchart LR
A[Your App] -->|model: claude-opus-4-5| B[Routeway API]
B --> C{Route}
C -->|OpenAI models| D[OpenAI]
C -->|Claude models| E[Anthropic]
C -->|Gemini models| F[Google]
C -->|Other models| G[...]
D & E & F & G -->|Normalized response| B
B --> A
```
You never manage provider credentials, SDKs, or per-provider quirks. Routeway handles it.
***
## Model IDs
Use the provider-native model ID as the `model` value. Routeway maps it automatically.
```python theme={null}
# OpenAI model
client.chat.completions.create(model="gpt-4o", ...)
# Anthropic model — same client, same endpoint
client.chat.completions.create(model="claude-opus-4-5", ...)
# Google model
client.chat.completions.create(model="gemini-2.5-pro", ...)
# DeepSeek model
client.chat.completions.create(model="deepseek-chat", ...)
```
The full list of supported model IDs is on the [Models](/getting-started/models) page.
***
## Choosing a Model
No single model is best for every task. Use this table as a starting point:
| Use case | Recommended models | Why |
| ----------------------------- | --------------------------------------------- | ---------------------------------- |
| General chat & Q\&A | `gpt-4o-mini`, `gemini-2.0-flash` | Fast, cheap, high quality |
| Complex reasoning | `o3`, `o4-mini`, `claude-opus-4-5` | Extended thinking, better accuracy |
| Code generation | `gpt-4o`, `kimi-k2.7-code`, `claude-opus-4-5` | Strong code benchmarks |
| Long documents | `gemini-2.5-pro`, `claude-opus-4-5` | 1M+ token context windows |
| Image understanding | `gpt-4o`, `gemini-2.5-pro` | Best vision capabilities |
| Low latency / high throughput | `gpt-4o-mini`, `gemini-2.0-flash` | Sub-second TTFT at scale |
| Cost-sensitive pipelines | `deepseek-chat`, `gpt-4o-mini` | Lowest cost per token |
When evaluating models, run the same set of representative prompts through several candidates and compare quality, latency, and cost. The best model is the one that meets your quality bar at the lowest cost.
***
## Model Capabilities at a Glance
All models. The core capability — generating coherent, contextual text from a prompt.
`gpt-4o`, `gpt-4o-mini`, `claude-opus-4-5`, `gemini-2.5-pro`. Send images and PDFs alongside text.
`gpt-4o`, `gpt-4o-mini`, `claude-*`, `gemini-*` and most frontier models. Let models invoke your functions.
`gpt-4o`, `gpt-4o-mini`. Guarantees JSON schema conformance.
`o3`, `o4-mini`, `claude-opus-4-5` (extended thinking), `deepseek-r1`. Extended deliberation before answering.
`gpt-4o`, `claude-*`, `gemini-*`. Reuse processed prefix tokens to cut cost and latency.
***
## Switching Models
Because Routeway uses the OpenAI-compatible format, switching models is a one-line change:
```python theme={null}
# Before
response = client.chat.completions.create(model="gpt-4o", messages=messages)
# After — same client, same code, different model
response = client.chat.completions.create(model="claude-opus-4-5", messages=messages)
```
Some parameters are model-specific. For example, `reasoning_effort` applies to OpenAI `o`-series models, while Anthropic's extended thinking uses a `thinking` block. Routeway normalizes responses but does not translate provider-specific request parameters.
***
## Model Versioning
Frontier models are regularly updated. To avoid unexpected behavior changes in production, pin to a specific model snapshot:
```python theme={null}
# Pinned — behavior is stable
model = "gpt-4o-2024-11-20"
# Floating alias — always the latest, may change
model = "gpt-4o"
```
Use floating aliases during development to get the latest improvements, then pin to a snapshot before deploying to production.
***
## Pricing
Each model has its own input and output token pricing. Costs are deducted from your Routeway balance per request.
* **Input tokens** — tokens in your `messages` array (including the system prompt)
* **Output tokens** — tokens generated by the model
* **Reasoning tokens** — internal thinking tokens on reasoning models (billed separately)
* **Cached tokens** — prompt tokens served from cache (discounted on supported models)
See the [Models](/getting-started/models) page for per-model pricing and the [Billing](/getting-started/billing) page for balance and usage details.
# Request & Response
Source: https://docs.routeway.ai/core-concepts/chat-completions/request-and-response
Anatomy of a chat completion request and response object.
Every interaction with Routeway follows the same request/response cycle. Understanding the shape of these objects makes it easier to integrate the API, handle edge cases, and debug issues.
## Request Object
A minimal chat completion request looks like this:
```json theme={null}
{
"model": "gpt-4o-mini",
"messages": [
{ "role": "user", "content": "What is 2 + 2?" }
]
}
```
### Common Parameters
| Parameter | Type | Default | Description |
| ------------------ | --------------- | ------------- | ------------------------------------------------------------- |
| `model` | string | required | Model ID to use (e.g. `"gpt-4o"`, `"claude-opus-4-5"`) |
| `messages` | array | required | Ordered list of conversation turns |
| `max_tokens` | integer | model default | Maximum output tokens to generate |
| `temperature` | number | `1.0` | Sampling temperature (`0` = deterministic, `2` = very random) |
| `top_p` | number | `1.0` | Nucleus sampling threshold |
| `stream` | boolean | `false` | Stream tokens as SSE chunks |
| `tools` | array | — | Function schemas the model may call |
| `tool_choice` | string / object | `"auto"` | How the model selects tools |
| `response_format` | object | — | Force `json_object` or `json_schema` output |
| `reasoning_effort` | string | — | `"low"` / `"medium"` / `"high"` for OpenAI reasoning models |
`temperature` and `top_p` both control randomness. Use one or the other — not both. For deterministic tasks like data extraction, set `temperature: 0`. For creative tasks, try values between `0.7` and `1.2`.
***
## Full Request Example
```python theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
response = client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "system", "content": "You are a concise assistant."},
{"role": "user", "content": "Name the planets in our solar system."}
],
max_tokens=150,
temperature=0.3,
)
print(response.choices[0].message.content)
```
```javascript theme={null}
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.routeway.ai/v1",
apiKey: process.env.ROUTEWAY_API_KEY,
});
const response = await client.chat.completions.create({
model: "gpt-4o",
messages: [
{ role: "system", content: "You are a concise assistant." },
{ role: "user", content: "Name the planets in our solar system." },
],
max_tokens: 150,
temperature: 0.3,
});
console.log(response.choices[0].message.content);
```
```bash theme={null}
curl https://api.routeway.ai/v1/chat/completions \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o",
"messages": [
{"role": "system", "content": "You are a concise assistant."},
{"role": "user", "content": "Name the planets in our solar system."}
],
"max_tokens": 150,
"temperature": 0.3
}'
```
***
## Response Object
A successful response looks like this:
```json theme={null}
{
"id": "chatcmpl-abc123",
"object": "chat.completion",
"created": 1749052800,
"model": "gpt-4o-2024-11-20",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "The eight planets are Mercury, Venus, Earth, Mars, Jupiter, Saturn, Uranus, and Neptune."
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 28,
"completion_tokens": 22,
"total_tokens": 50
}
}
```
### Key Fields
| Field | Description |
| ---------------------------- | ------------------------------------------ |
| `id` | Unique identifier for this completion |
| `model` | The exact model snapshot that was used |
| `choices` | Array of generated responses (usually one) |
| `choices[0].message.content` | The model's text reply |
| `choices[0].finish_reason` | Why the model stopped generating |
| `usage` | Token counts for billing |
### `finish_reason` Values
| Value | Meaning |
| ------------------ | -------------------------------------------------- |
| `"stop"` | Model finished naturally |
| `"length"` | Hit `max_tokens` limit — response may be truncated |
| `"tool_calls"` | Model wants to call a function |
| `"content_filter"` | Response blocked by a content policy |
| `"null"` | Stream is still in progress (streaming only) |
Always check `finish_reason`. A value of `"length"` means the response was cut off. Handle it by either increasing `max_tokens` or detecting and requesting a continuation.
***
## Accessing the Response Text
The model's reply lives at `choices[0].message.content`:
```python theme={null}
text = response.choices[0].message.content
```
When `n > 1`, multiple completions are returned and you index into `choices[0]`, `choices[1]`, etc.
When the model calls a tool instead of replying with text, `content` may be `null` and `tool_calls` will be populated:
```python theme={null}
message = response.choices[0].message
if message.tool_calls:
# model wants to call a function
for tc in message.tool_calls:
print(tc.function.name, tc.function.arguments)
else:
# normal text response
print(message.content)
```
***
## Error Responses
When something goes wrong, Routeway returns a standard error object:
```json theme={null}
{
"error": {
"message": "Invalid model: 'gpt-999'.",
"type": "invalid_request_error",
"code": "model_not_found"
}
}
```
| HTTP status | Type | Common causes |
| ----------- | ----------------------- | -------------------------------------------------- |
| `400` | `invalid_request_error` | Bad parameters, context too long, invalid model ID |
| `401` | `authentication_error` | Missing or invalid API key |
| `429` | `rate_limit_error` | Too many requests or insufficient balance |
| `500` | `server_error` | Upstream provider error |
See the [Errors](/getting-started/errors) page for the full list and handling guidance.
***
## HTTP Headers
Every request must include:
```
Authorization: Bearer YOUR_ROUTEWAY_API_KEY
Content-Type: application/json
```
The response includes a `x-request-id` header you can use when contacting support about a specific call.
# Tokens & Context
Source: https://docs.routeway.ai/core-concepts/chat-completions/tokens-and-context
How tokenization works, what context windows mean, and how to manage both.
Every piece of text you send to or receive from a model is measured in **tokens**. Understanding tokens helps you estimate costs, avoid context-window errors, and build efficient pipelines.
## What Is a Token?
A token is a chunk of text — roughly 3–4 characters or about 0.75 words in English. Tokenization is not simply splitting on spaces; punctuation, subwords, and whitespace each contribute their own tokens.
| Text | Approximate tokens |
| --------------------------------- | ------------------ |
| `"Hello, world!"` | 4 |
| `"Explain quantum entanglement."` | 5 |
| A typical paragraph (100 words) | \~130 |
| A full A4 page of text | \~500–700 |
| A 10,000-word document | \~13,000 |
Token counts vary slightly by model because each provider uses a different tokenizer. The figures above are approximate. The `usage` field in every API response gives you the exact counts for that call.
***
## Token Types in a Response
The `usage` object returned with every completion breaks down token usage:
```json theme={null}
{
"usage": {
"prompt_tokens": 312,
"completion_tokens": 87,
"total_tokens": 399
}
}
```
| Field | What it counts |
| ------------------- | ------------------------------------------------------------- |
| `prompt_tokens` | All tokens in your `messages` array, including system prompts |
| `completion_tokens` | Tokens generated by the model in this response |
| `total_tokens` | Sum of the above |
On reasoning models, `completion_tokens` includes internal reasoning tokens. Some responses include a `completion_tokens_details` breakdown.
***
## Context Windows
A model's **context window** is the maximum number of tokens it can process in a single request — the sum of `prompt_tokens` and `completion_tokens` combined.
| Model | Context window |
| ----------------- | ---------------- |
| `gpt-4o-mini` | 128,000 tokens |
| `gpt-4o` | 128,000 tokens |
| `o3`, `o4-mini` | 200,000 tokens |
| `claude-opus-4-5` | 200,000 tokens |
| `gemini-2.5-pro` | 1,000,000 tokens |
Sending a request that exceeds the context window returns a `400` error. Always leave headroom for the model's output — if the window is 128K and your prompt is 127K tokens, the model has almost no room to respond.
***
## Controlling Output Length
Use `max_tokens` to cap how many tokens the model generates. This prevents runaway costs and enforces response length for your use case.
```python theme={null}
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Summarize the French Revolution."}],
max_tokens=200, # model will stop after ~200 output tokens
)
```
If the model hits `max_tokens` before finishing, `finish_reason` will be `"length"` instead of `"stop"`. The response is truncated — check `finish_reason` in production to detect this.
***
## Managing Long Conversations
Because the full `messages` array is sent on every request, conversation costs grow with each turn. For long sessions, use one of these strategies:
Keep only the last N turns in the messages array, always preserving the `system` message at the start.
```python theme={null}
MAX_TURNS = 10 # keep last 10 user+assistant pairs
def trim_messages(messages):
system = [m for m in messages if m["role"] == "system"]
rest = [m for m in messages if m["role"] != "system"]
return system + rest[-MAX_TURNS * 2:] # 2 messages per turn
```
When history gets long, ask the model to summarize it, then replace the old messages with the summary.
```python theme={null}
summary_response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": "Summarize this conversation in 3-5 sentences, preserving key facts."},
*old_messages
]
)
summary = summary_response.choices[0].message.content
# Replace history with summary
messages = [
{"role": "system", "content": system_prompt},
{"role": "assistant", "content": f"[Conversation summary: {summary}]"},
]
```
If your system prompt or context is large and stable across many requests, enable [Prompt Caching](/guides/chat-completions/caching). The first request processes and caches the prefix; subsequent requests with the same prefix pay a fraction of the cost.
***
## Estimating Costs Before Sending
You can estimate token usage before making a request by counting tokens locally. The `tiktoken` library implements OpenAI's tokenizer:
```python theme={null}
import tiktoken
enc = tiktoken.encoding_for_model("gpt-4o")
def count_tokens(messages):
total = 0
for msg in messages:
total += 4 # overhead per message
for value in msg.values():
total += len(enc.encode(str(value)))
total += 2 # priming tokens
return total
tokens = count_tokens(messages)
print(f"Estimated prompt tokens: {tokens}")
```
This is an approximation. For billing purposes, always use the `usage` values returned in the actual API response.
***
## Token Cost Summary
| Token type | Billing |
| -------------------- | -------------------------------------------- |
| Prompt tokens | Charged per model's input rate |
| Completion tokens | Charged per model's output rate |
| Cached prompt tokens | Discounted (typically 50–75% off input rate) |
| Reasoning tokens | Billed as output tokens on most models |
See the [Models](/getting-started/models) page for per-model rates and the [Billing](/getting-started/billing) page to check your current balance and usage.
# Editing Images
Source: https://docs.routeway.ai/core-concepts/image-generation/editing-images
Modify an existing image with a text prompt and optional inpainting mask.
The `/v1/images/edits` endpoint takes one or more existing images and a text description of what to change, then returns modified versions. You can target specific regions using a mask (inpainting), or apply changes to the whole image.
***
## Basic example
```python theme={null}
import os
import base64
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
with open("photo.png", "rb") as f:
encoded = base64.b64encode(f.read()).decode("utf-8")
response = client.images.edit(
model="flux-kontext-max",
image=encoded,
prompt="Replace the sky with a dramatic stormy sky, dark clouds and lightning",
size="1024x1024",
)
print(response.data[0].url)
```
```javascript theme={null}
import OpenAI from "openai";
import fs from "fs";
const client = new OpenAI({
baseURL: "https://api.routeway.ai/v1",
apiKey: process.env.ROUTEWAY_API_KEY,
});
const imageBuffer = fs.readFileSync("photo.png");
const encoded = imageBuffer.toString("base64");
const response = await client.images.edit({
model: "flux-kontext-max",
image: encoded,
prompt: "Replace the sky with a dramatic stormy sky, dark clouds and lightning",
size: "1024x1024",
});
console.log(response.data[0].url);
```
```bash theme={null}
ENCODED=$(base64 -i photo.png)
curl https://api.routeway.ai/v1/images/edits \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d "{
\"model\": \"flux-kontext-max\",
\"images\": [\"$ENCODED\"],
\"prompt\": \"Replace the sky with a dramatic stormy sky\",
\"size\": \"1024x1024\"
}"
```
***
## Request parameters
| Parameter | Type | Default | Description |
| --------------------- | ---------------- | ------------- | --------------------------------------------------------------------------------- |
| `images` | array of strings | required | Base64-encoded PNG images (max 4 MB each) |
| `prompt` | string | required | Text description of the desired change |
| `model` | string | — | Model to use for editing (e.g. `"flux-kontext-max"`) |
| `mask` | string | — | Base64-encoded PNG mask — white areas are edited, black areas are preserved |
| `n` | integer | `1` | Number of output images to generate (1–10) |
| `size` | string | `"1024x1024"` | Output resolution: `"256x256"`, `"512x512"`, or `"1024x1024"` |
| `response_format` | string | `"url"` | `"url"` or `"b64_json"` |
| `strength` | number | — | How strongly the edit is applied (0.0–1.0). Lower = more faithful to the original |
| `guidance_scale` | number | — | How closely the model follows the prompt (0.0–20.0). Higher = more literal |
| `num_inference_steps` | integer | — | Denoising steps (1–100). More steps = sharper result, slower generation |
| `seed` | integer | — | Fixed seed for reproducible outputs |
***
## Inpainting with a mask
A mask lets you edit only a specific region of the image and leave the rest untouched. The mask is a PNG of the same dimensions as the input:
* **White pixels** → the area you want the model to edit
* **Black pixels** → the area to preserve exactly
```python theme={null}
import os
import base64
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
with open("portrait.png", "rb") as f:
image_b64 = base64.b64encode(f.read()).decode("utf-8")
# Mask: white over the background, black over the subject
with open("mask_background.png", "rb") as f:
mask_b64 = base64.b64encode(f.read()).decode("utf-8")
response = client.images.edit(
model="flux-kontext-max",
image=image_b64,
mask=mask_b64,
prompt="Replace the background with a sunlit Tuscan vineyard",
size="1024x1024",
)
print(response.data[0].url)
```
Create masks in any image editor by painting white over the area to change and black over the area to keep, then export as PNG. The mask must be the same width and height as the source image.
***
## Editing multiple images
Pass more than one image to let the model combine or reference several source images in the edit.
```python theme={null}
with open("style_reference.png", "rb") as f:
style_b64 = base64.b64encode(f.read()).decode("utf-8")
with open("content_image.png", "rb") as f:
content_b64 = base64.b64encode(f.read()).decode("utf-8")
response = client.images.edit(
model="flux-kontext-max",
images=[content_b64, style_b64],
prompt="Apply the artistic style of the second image to the first image",
size="1024x1024",
)
```
***
## Controlling the edit with `strength`
`strength` controls how much the model is allowed to deviate from the original image:
| `strength` | Effect |
| ---------- | ----------------------------------------------------------- |
| `0.1–0.3` | Subtle changes — texture, colour, minor detail edits |
| `0.4–0.6` | Moderate changes — object swaps, lighting adjustments |
| `0.7–1.0` | Heavy changes — full style transfer, background replacement |
```python theme={null}
response = client.images.edit(
model="flux-kontext-max",
image=image_b64,
prompt="Make the image look like an oil painting",
strength=0.8, # high — allow major stylistic change
size="1024x1024",
)
```
***
## Reproducible results with `seed`
Pass a fixed integer `seed` to get the same output each time you send the same prompt and image. Useful for A/B testing or generating variants from a known baseline.
```python theme={null}
response = client.images.edit(
model="flux-kontext-max",
image=image_b64,
prompt="Add soft studio lighting",
seed=42,
size="1024x1024",
)
```
Not all models honour the `seed` parameter. Results may still vary slightly due to floating-point non-determinism in some backends.
***
## Response object
```json theme={null}
{
"created": 1749052800,
"data": [
{
"url": "https://cdn.routeway.ai/images/edited_abc123.png",
"revised_prompt": "Replace the sky with a dramatic stormy sky, dark clouds and lightning"
}
]
}
```
Same shape as the generations endpoint — `url` or `b64_json` per item, plus an optional `revised_prompt`.
***
## Image requirements
| Requirement | Value |
| ---------------------- | ------------------------------ |
| Format | PNG |
| Max file size | 4 MB per image |
| Encoding | Base64 string |
| Max images per request | 10 |
| Mask format | PNG, same dimensions as source |
***
## Best practices
Resize large photos before encoding. A 1024×1024 PNG is typically well under 2 MB. Oversized images return a `400` error.
```python theme={null}
from PIL import Image
img = Image.open("large_photo.jpg")
img = img.resize((1024, 1024))
img.save("resized.png", format="PNG")
```
The more specific the prompt, the better the edit respects the original. Mention what should stay unchanged.
```
# Vague — may alter the whole image
"Add snow"
# Specific — limits the change
"Add a light dusting of snow on the rooftops only, keep the people and street unchanged"
```
A `strength` of `0.3–0.5` works well for lighting corrections, colour grading, and texture changes without losing structural detail.
When the model isn't following the prompt closely enough, increase `guidance_scale` (try `7–12`). Very high values (`>15`) can introduce artefacts.
# Generating Images
Source: https://docs.routeway.ai/core-concepts/image-generation/generating-images
Turn a text prompt into an image using the /v1/images/generations endpoint.
The `/v1/images/generations` endpoint takes a text prompt and returns one or more images. It is compatible with the OpenAI `images.generate()` SDK method — only the base URL and key need to change.
***
## Basic example
```python theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
response = client.images.generate(
model="flux-1-schnell",
prompt="A serene Japanese garden at dawn, soft mist, cherry blossoms, photorealistic",
size="1024x1024",
n=1,
)
print(response.data[0].url)
```
```javascript theme={null}
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.routeway.ai/v1",
apiKey: process.env.ROUTEWAY_API_KEY,
});
const response = await client.images.generate({
model: "flux-1-schnell",
prompt: "A serene Japanese garden at dawn, soft mist, cherry blossoms, photorealistic",
size: "1024x1024",
n: 1,
});
console.log(response.data[0].url);
```
```bash theme={null}
curl https://api.routeway.ai/v1/images/generations \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "flux-1-schnell",
"prompt": "A serene Japanese garden at dawn, soft mist, cherry blossoms, photorealistic",
"size": "1024x1024",
"n": 1
}'
```
***
## Request parameters
| Parameter | Type | Default | Description |
| ----------------- | ------- | ------------- | ------------------------------------------------------ |
| `model` | string | required | Image model ID (e.g. `"flux-1-schnell"`, `"dall-e-3"`) |
| `prompt` | string | required | Text description of the image to generate |
| `n` | integer | `1` | Number of images to generate (1–10) |
| `size` | string | model default | Output resolution, e.g. `"1024x1024"`, `"1792x1024"` |
| `quality` | string | model default | Provider-specific preset, e.g. `"standard"`, `"hd"` |
| `response_format` | string | `"url"` | `"url"` or `"b64_json"` |
***
## Response object
```json theme={null}
{
"created": 1749052800,
"data": [
{
"url": "https://cdn.routeway.ai/images/abc123.png",
"revised_prompt": "A serene Japanese garden at dawn..."
}
]
}
```
| Field | Description |
| ----------------------- | --------------------------------------------------------------- |
| `data[].url` | Temporary URL to the generated image (expires in 1 hour) |
| `data[].b64_json` | Base64 PNG data, present when `response_format: "b64_json"` |
| `data[].revised_prompt` | The prompt as interpreted by the model, if the model revised it |
***
## Saving images locally
URLs expire after 1 hour. Download the image immediately if you need to keep it.
```python theme={null}
import os
import requests
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
response = client.images.generate(
model="flux-1-schnell",
prompt="A futuristic cityscape at night, neon lights, rain-slicked streets",
size="1024x1024",
)
image_url = response.data[0].url
image_bytes = requests.get(image_url).content
with open("output.png", "wb") as f:
f.write(image_bytes)
print("Saved to output.png")
```
```javascript theme={null}
import OpenAI from "openai";
import fs from "fs";
const client = new OpenAI({
baseURL: "https://api.routeway.ai/v1",
apiKey: process.env.ROUTEWAY_API_KEY,
});
const response = await client.images.generate({
model: "flux-1-schnell",
prompt: "A futuristic cityscape at night, neon lights, rain-slicked streets",
size: "1024x1024",
});
const imageUrl = response.data[0].url;
const imageResponse = await fetch(imageUrl);
const buffer = Buffer.from(await imageResponse.arrayBuffer());
fs.writeFileSync("output.png", buffer);
console.log("Saved to output.png");
```
Alternatively, request `response_format: "b64_json"` to receive the image bytes directly in the API response — no second HTTP request needed.
```python theme={null}
import base64
response = client.images.generate(
model="flux-1-schnell",
prompt="Abstract watercolour landscape",
response_format="b64_json",
)
image_bytes = base64.b64decode(response.data[0].b64_json)
with open("output.png", "wb") as f:
f.write(image_bytes)
```
***
## Generating multiple images
Set `n` to receive several variations in a single call. Each item in `data` is an independent generation.
```python theme={null}
response = client.images.generate(
model="flux-1-schnell",
prompt="A cosy mountain cabin in winter, warm light from the windows",
size="1024x1024",
n=4,
)
for i, image in enumerate(response.data):
print(f"Image {i + 1}: {image.url}")
```
***
## Choosing size and quality
Available sizes and quality presets vary by model. As a general guide:
| Size | Aspect ratio | Typical use |
| ----------- | ------------ | --------------------------------- |
| `1024x1024` | Square | Social media, avatars, thumbnails |
| `1792x1024` | Landscape | Banners, hero images |
| `1024x1792` | Portrait | Mobile wallpapers, posters |
| Quality | Trade-off |
| ---------- | ---------------------------------------- |
| `standard` | Faster, lower cost, slightly less detail |
| `hd` | Slower, higher cost, finer details |
Use `flux-1-schnell` for iteration and prototyping — it generates in seconds. Switch to `flux-1-dev` or `dall-e-3` when quality matters for production output.
***
## Writing effective prompts
Image models are sensitive to how prompts are written. A few techniques that consistently improve results:
Vague prompts produce generic results. Name a style, medium, or artist reference to anchor the output.
```
# Vague
"A forest"
# Specific
"A dense pine forest at dusk, oil painting, dramatic lighting, muted earth tones"
```
Tell the model where to place things and from what angle.
```
"Close-up portrait of an elderly sailor, weathered face, looking off-camera,
shallow depth of field, golden hour light"
```
Terms like `shallow depth of field`, `35mm lens`, `shot on film`, `bokeh`, and `golden hour` reliably improve photorealistic outputs.
Some models (like FLUX) accept a `negative_prompt` parameter to exclude unwanted elements.
```json theme={null}
{
"prompt": "A clean minimal product photo of a white ceramic mug",
"negative_prompt": "blurry, watermark, text, cluttered background, low quality"
}
```
***
## Error handling
```python theme={null}
from openai import BadRequestError
try:
response = client.images.generate(
model="flux-1-schnell",
prompt="...",
)
print(response.data[0].url)
except BadRequestError as e:
# Prompt was rejected (content policy, invalid parameters, etc.)
print(f"Request error: {e.message}")
except Exception as e:
print(f"Unexpected error: {e}")
```
Common errors:
| Status | Cause |
| ------ | ------------------------------------------------------- |
| `400` | Invalid parameters or prompt rejected by content policy |
| `401` | Missing or invalid API key |
| `429` | Rate limit or insufficient balance |
| `500` | Upstream model error — retry with exponential backoff |
# Image Generation
Source: https://docs.routeway.ai/core-concepts/image-generation/overview
Generate and edit images from text prompts using Routeway's image models.
Routeway exposes two image endpoints that mirror the OpenAI images API — so any code that calls `api.openai.com/v1/images/generations` works against Routeway with only a base URL swap.
## What's in this section
Turn a text prompt into an image. Covers models, sizes, quality presets, response formats, and saving images locally.
Modify an existing image with a prompt and optional mask. Covers inpainting, multi-image edits, and advanced parameters like `strength` and `guidance_scale`.
***
## Endpoints
| Method | Path | What it does |
| ------ | ----------------------------------------------------------------------- | ------------------------------------------------------- |
| `POST` | [`/v1/images/generations`](/api-reference/ImageGenerations/generations) | Generate one or more images from a text prompt |
| `POST` | [`/v1/images/edits`](/api-reference/ImageEdits/edits) | Edit an existing image using a prompt and optional mask |
Image models use dedicated endpoints — they are not available on `/v1/chat/completions`. Use the OpenAI SDK's `client.images.generate()` and `client.images.edit()` helpers, or call the endpoints directly with cURL.
***
## Response formats
Both endpoints support two ways to receive the result:
| `response_format` | What you get | When to use |
| ----------------- | ----------------------- | ---------------------------------------- |
| `"url"` (default) | A temporary HTTPS URL | Quick testing and previews |
| `"b64_json"` | Base64-encoded PNG data | Production — store the image immediately |
URLs returned by the API expire after **1 hour**. Download and store any image you want to keep — don't rely on the URL long-term.
***
## Supported models
Routeway routes image requests to the underlying provider based on the `model` field. A non-exhaustive selection:
| Model | Provider | Best for |
| ------------------ | ----------------- | --------------------------------------------- |
| `flux-1-schnell` | Black Forest Labs | Fast generation, low cost |
| `flux-1-dev` | Black Forest Labs | Higher quality, slower |
| `flux-kontext-max` | Black Forest Labs | Image editing with high fidelity |
| `gpt-image-1` | OpenAI | Photorealistic outputs, instruction-following |
| `dall-e-3` | OpenAI | Creative, stylised images |
| `dall-e-2` | OpenAI | Smaller images, lower cost |
Check the [Models catalog](https://routeway.ai/models) for the full list and current pricing.
# Overview
Source: https://docs.routeway.ai/core-concepts/overview
The fundamental building blocks behind every Routeway API call.
Before diving into advanced guides, it helps to understand a handful of concepts that underpin every request you make through Routeway. This section covers them in depth — read it once and the rest of the docs will make much more sense.
## What's in this section
### Chat Completions
How the `messages` array works, what `system`, `user`, and `assistant` roles do, and how to build multi-turn conversations correctly.
What tokens are, how context windows limit request size, and how to manage long conversations without hitting the limit or running up costs.
How Routeway routes a request to the right upstream provider, how to pick the right model for your use case, and how to pin versions for stable production behavior.
The full anatomy of a chat completion request and response — every key parameter, what `finish_reason` values mean, and how errors are shaped.
### Responses
The full anatomy of a Responses API request and response — input formats, output structure, and key parameters.
Use `previous_response_id` to build stateful conversations without resending full message history.
Leverage web search, file search, and code interpreter without defining custom function schemas.
### Image Generation
Turn a text prompt into an image. Covers models, sizes, quality presets, response formats, and saving images locally.
Modify an existing image with a prompt and optional mask. Covers inpainting, multi-image edits, and advanced parameters like `strength` and `guidance_scale`.
***
## Where to go next
Once you're comfortable with these fundamentals, the [Guides](/guides/chat-completions/overview) show you how to use streaming, tool calling, structured outputs, reasoning models, multimodal inputs, and caching — all of which build directly on what's covered here.
# Built-in Tools
Source: https://docs.routeway.ai/core-concepts/responses/built-in-tools
Use web search, file search, and code interpreter without defining custom schemas.
The Responses API includes built-in tools that the model can invoke automatically — no function schemas required. These tools extend the model's capabilities beyond its training data.
Built-in tools are handled server-side. Unlike custom function calling (where your code executes the function), built-in tools execute automatically and return results to the model within the same request.
## Available Built-in Tools
| Tool | Type value | Description |
| ---------------- | -------------------- | --------------------------------------------------- |
| Web Search | `web_search_preview` | Search the internet for current information |
| File Search | `file_search` | Search through uploaded files using semantic search |
| Code Interpreter | `code_interpreter` | Execute Python code in a sandbox |
***
## Web Search
Enable web search to let the model look up current information that isn't in its training data.
```python theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
response = client.responses.create(
model="gpt-4o",
tools=[{"type": "web_search_preview"}],
input="What were the top tech news stories today?"
)
print(response.output_text)
```
```javascript theme={null}
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.routeway.ai/v1",
apiKey: process.env.ROUTEWAY_API_KEY,
});
const response = await client.responses.create({
model: "gpt-4o",
tools: [{ type: "web_search_preview" }],
input: "What were the top tech news stories today?",
});
console.log(response.output_text);
```
```bash theme={null}
curl https://api.routeway.ai/v1/responses \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o",
"tools": [{"type": "web_search_preview"}],
"input": "What were the top tech news stories today?"
}'
```
When the model uses web search, the `output` array includes a `web_search_call` item showing what was searched:
```json theme={null}
{
"output": [
{
"type": "web_search_call",
"id": "ws_abc123",
"status": "completed"
},
{
"type": "message",
"role": "assistant",
"content": [
{
"type": "output_text",
"text": "Here are today's top tech stories..."
}
]
}
]
}
```
Web search is useful for questions about current events, recent releases, live data, or anything that may have changed since the model's training cutoff.
***
## Code Interpreter
Let the model write and execute Python code to solve math problems, analyze data, or generate charts.
```python theme={null}
response = client.responses.create(
model="gpt-4o",
tools=[{"type": "code_interpreter"}],
input="Calculate the first 20 Fibonacci numbers and show them in a formatted table."
)
print(response.output_text)
```
```javascript theme={null}
const response = await client.responses.create({
model: "gpt-4o",
tools: [{ type: "code_interpreter" }],
input: "Calculate the first 20 Fibonacci numbers and show them in a formatted table.",
});
console.log(response.output_text);
```
The `output` array will include a `code_interpreter_call` item with the code that was executed:
```json theme={null}
{
"output": [
{
"type": "code_interpreter_call",
"id": "ci_abc123",
"code": "fibs = [0, 1]\nfor i in range(18):\n fibs.append(fibs[-1] + fibs[-2])\nprint(fibs)",
"results": [
{
"type": "text",
"text": "[0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987, 1597, 2584, 4181]"
}
]
},
{
"type": "message",
"role": "assistant",
"content": [...]
}
]
}
```
***
## Combining Multiple Tools
You can enable multiple built-in tools in the same request. The model decides which to use based on the input.
```python theme={null}
response = client.responses.create(
model="gpt-4o",
tools=[
{"type": "web_search_preview"},
{"type": "code_interpreter"}
],
input="Look up the current population of the 10 largest countries, then create a bar chart comparing them."
)
print(response.output_text)
```
The model may:
1. Use web search to find current population data
2. Use code interpreter to generate the chart
3. Return the final answer with both tool calls visible in the `output` array
***
## Mixing Built-in and Custom Tools
You can combine built-in tools with your own custom function definitions:
```python theme={null}
response = client.responses.create(
model="gpt-4o",
tools=[
{"type": "web_search_preview"},
{
"type": "function",
"function": {
"name": "save_to_database",
"description": "Save structured data to the database.",
"parameters": {
"type": "object",
"properties": {
"data": {"type": "object"},
"collection": {"type": "string"}
},
"required": ["data", "collection"]
}
}
}
],
input="Find the current Bitcoin price and save it to the prices collection."
)
```
Built-in tools execute automatically on the server. Custom function calls still require your code to execute the function and submit results back. See [Tool Calling](/guides/chat-completions/tool-calling) for the full custom function workflow.
***
## Tool Choice
Control how the model selects tools with `tool_choice`:
| Value | Behavior |
| -------------------------------- | -------------------------------------------- |
| `"auto"` | Model decides whether to use tools (default) |
| `"required"` | Model must use at least one tool |
| `"none"` | Model cannot use any tools |
| `{"type": "web_search_preview"}` | Force a specific tool |
```python theme={null}
# Force the model to search the web
response = client.responses.create(
model="gpt-4o",
tools=[{"type": "web_search_preview"}],
tool_choice={"type": "web_search_preview"},
input="What is the current weather in Tokyo?"
)
```
***
## Best Practices
If your prompt clearly needs current data, enable web search. For questions answerable from training data, skip it to save latency.
Models are better at writing code than doing arithmetic. For math-heavy tasks, enable code interpreter for accurate results.
Inspect the `output` array to see which tools were used and verify the model's reasoning chain.
Enabling too many tools can confuse the model. Enable only the tools relevant to the task at hand.
# Multi-turn Conversations
Source: https://docs.routeway.ai/core-concepts/responses/multi-turn
Build stateful conversations using previous_response_id without resending message history.
The Responses API simplifies multi-turn conversations by letting you reference a previous response instead of manually maintaining and resending the full message history on every request.
## How It Works
Each response has a unique `id`. Pass it as `previous_response_id` in your next request, and the model automatically has access to the full conversation context up to that point.
```mermaid theme={null}
sequenceDiagram
participant App
participant Routeway
App->>Routeway: input: "What is Rust?"
Routeway-->>App: id: "resp_1", output_text: "Rust is a systems programming language..."
App->>Routeway: input: "How does its ownership model work?", previous_response_id: "resp_1"
Routeway-->>App: id: "resp_2", output_text: "Rust's ownership model ensures memory safety..."
App->>Routeway: input: "Give me a code example.", previous_response_id: "resp_2"
Routeway-->>App: id: "resp_3", output_text: "fn main() { let s = String::from(...)..."
```
You don't need to resend any messages from earlier turns. The server reconstructs the full conversation from the chain of response IDs.
***
## Basic Example
```python theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
# Turn 1
response = client.responses.create(
model="gpt-4o",
input="What is Rust?"
)
print(response.output_text)
# Turn 2 — references the first response
response = client.responses.create(
model="gpt-4o",
input="How does its ownership model work?",
previous_response_id=response.id
)
print(response.output_text)
# Turn 3 — references the second response (which includes turn 1)
response = client.responses.create(
model="gpt-4o",
input="Give me a simple code example.",
previous_response_id=response.id
)
print(response.output_text)
```
```javascript theme={null}
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.routeway.ai/v1",
apiKey: process.env.ROUTEWAY_API_KEY,
});
// Turn 1
let response = await client.responses.create({
model: "gpt-4o",
input: "What is Rust?",
});
console.log(response.output_text);
// Turn 2
response = await client.responses.create({
model: "gpt-4o",
input: "How does its ownership model work?",
previous_response_id: response.id,
});
console.log(response.output_text);
// Turn 3
response = await client.responses.create({
model: "gpt-4o",
input: "Give me a simple code example.",
previous_response_id: response.id,
});
console.log(response.output_text);
```
```bash theme={null}
# Turn 1
curl https://api.routeway.ai/v1/responses \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o",
"input": "What is Rust?"
}'
# Returns: {"id": "resp_abc123", ...}
# Turn 2 — reference the previous response
curl https://api.routeway.ai/v1/responses \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o",
"input": "How does its ownership model work?",
"previous_response_id": "resp_abc123"
}'
```
***
## Using Instructions Across Turns
The `instructions` parameter applies system-level guidance to every turn. When using `previous_response_id`, include `instructions` on each request to maintain consistent behavior.
```python theme={null}
instructions = "You are a Rust tutor. Give concise, beginner-friendly explanations."
response = client.responses.create(
model="gpt-4o",
instructions=instructions,
input="What are lifetimes in Rust?"
)
response = client.responses.create(
model="gpt-4o",
instructions=instructions,
input="Why are they needed?",
previous_response_id=response.id
)
```
Unlike Chat Completions where the system message is part of `messages`, the `instructions` parameter sits outside the conversation history. This makes it easy to change instructions between turns without modifying the stored context.
***
## Conversation Branching
Since each response has its own ID, you can branch conversations by referencing the same `previous_response_id` in multiple follow-up requests:
```python theme={null}
# Base question
base = client.responses.create(
model="gpt-4o",
input="Compare Python and Rust."
)
# Branch A — performance focus
branch_a = client.responses.create(
model="gpt-4o",
input="Focus on performance differences.",
previous_response_id=base.id
)
# Branch B — ecosystem focus (same parent)
branch_b = client.responses.create(
model="gpt-4o",
input="Focus on ecosystem and libraries.",
previous_response_id=base.id
)
```
***
## Disabling Storage
By default, responses are stored server-side so they can be referenced by future requests. If you don't need multi-turn and want to avoid storing data, set `store: false`:
```json theme={null}
{
"model": "gpt-4o",
"input": "One-off question, no need to remember this.",
"store": false
}
```
When `store` is `false`, the response ID cannot be used as a `previous_response_id` in future requests. Only disable storage for truly one-off queries.
***
## Comparison: Multi-turn Approaches
| Approach | When to use |
| --------------------------- | ----------------------------------------------------------------------------------------- |
| `previous_response_id` | Conversations where the server manages state — simpler client code |
| Manual `input` array | When you need full control over what context the model sees, or want to trim/edit history |
| Chat Completions `messages` | Maximum model coverage, non-OpenAI SDKs, or fine-grained message control |
***
## Token Usage in Multi-turn
Each turn in a `previous_response_id` chain re-processes the full conversation history as input tokens. The billing is the same as Chat Completions — longer conversations cost more per turn.
To manage costs:
* Keep conversations short when possible
* Use `instructions` instead of long system messages repeated in `input`
* Start a new conversation chain when the topic changes significantly
* Use [Prompt Caching](/guides/chat-completions/caching) on models that support it
Token usage reported in the `usage` field reflects the full context processed for that turn, including all prior messages from the chain.
# Responses API
Source: https://docs.routeway.ai/core-concepts/responses/overview
A higher-level API for text generation with built-in tools and multi-turn state.
The Responses API is a newer text generation endpoint that builds on top of Chat Completions with a simpler interface, built-in tools, and optional server-side conversation state. Routeway supports it at `POST /v1/responses` for all models that advertise this endpoint.
The Responses API uses the same OpenAI SDK methods (`client.responses.create`). Any code written for OpenAI's Responses API works with Routeway by changing `base_url` and `api_key`.
## Quick Example
```python theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
response = client.responses.create(
model="gpt-4o",
input="What is the capital of France?"
)
print(response.output_text)
```
```javascript theme={null}
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.routeway.ai/v1",
apiKey: process.env.ROUTEWAY_API_KEY,
});
const response = await client.responses.create({
model: "gpt-4o",
input: "What is the capital of France?",
});
console.log(response.output_text);
```
```bash theme={null}
curl https://api.routeway.ai/v1/responses \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o",
"input": "What is the capital of France?"
}'
```
***
## Which Models Support It?
Not every model on Routeway supports the Responses API. Check the model's `endpoints` array via `GET /v1/models`:
```json theme={null}
{
"id": "gpt-4o",
"endpoints": ["/v1/chat/completions", "/v1/responses"],
...
}
```
If `/v1/responses` is listed, the model is compatible. See [Models](/getting-started/models) for the full list.
***
## What's in This Section
The full anatomy of a Responses API request and response — input formats, output structure, and key parameters.
Use `previous_response_id` to build stateful conversations without resending full message history.
Leverage web search, file search, and code interpreter without defining custom function schemas.
# Request & Response
Source: https://docs.routeway.ai/core-concepts/responses/request-and-response
Anatomy of a Responses API request and response object.
The Responses API uses a streamlined request format and returns a richer response object compared to Chat Completions. This page covers every key field.
## Request Object
A minimal request needs only a `model` and `input`:
```json theme={null}
{
"model": "gpt-4o",
"input": "Explain photosynthesis in one sentence."
}
```
### Input Formats
The `input` field accepts multiple formats:
| Format | Example | Use case |
| ------------- | -------------------------------------- | -------------------------------------------- |
| Plain string | `"What is 2+2?"` | Simple single-turn questions |
| Message array | `[{"role": "user", "content": "..."}]` | Multi-role conversations with system prompts |
**String input:**
```json theme={null}
{
"model": "gpt-4o",
"input": "What is the speed of light?"
}
```
**Message array input (with system prompt):**
```json theme={null}
{
"model": "gpt-4o",
"input": [
{ "role": "system", "content": "You are a physics tutor." },
{ "role": "user", "content": "What is the speed of light?" }
]
}
```
***
### Common Parameters
| Parameter | Type | Default | Description |
| ---------------------- | ---------------- | ------------- | --------------------------------------------------------------------------- |
| `model` | string | required | Model ID to use (e.g. `"gpt-4o"`, `"gpt-4o-mini"`) |
| `input` | string \| array | required | The user input — a plain string or a messages array |
| `instructions` | string | — | System-level instructions (alternative to a system message in `input`) |
| `max_output_tokens` | integer | model default | Maximum tokens to generate |
| `temperature` | number | `1.0` | Sampling temperature (`0` = deterministic, `2` = very random) |
| `top_p` | number | `1.0` | Nucleus sampling threshold |
| `stream` | boolean | `false` | Stream output as Server-Sent Events |
| `tools` | array | — | Built-in tools or custom function schemas |
| `tool_choice` | string \| object | `"auto"` | How the model selects tools |
| `text` | object | — | Text output configuration (e.g. `{"format": {"type": "json_schema", ...}}`) |
| `previous_response_id` | string | — | Chain this response to a previous one for multi-turn |
| `reasoning` | object | — | Reasoning configuration for o-series models (e.g. `{"effort": "high"}`) |
| `store` | boolean | `true` | Whether to store the response for multi-turn use |
Use `instructions` as a cleaner alternative to embedding a system message inside `input`. It keeps your input focused on the user's question.
***
## Full Request Example
```python theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
response = client.responses.create(
model="gpt-4o",
instructions="You are a concise assistant. Reply in one paragraph.",
input="Explain how DNS works.",
max_output_tokens=200,
temperature=0.3,
)
print(response.output_text)
```
```javascript theme={null}
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.routeway.ai/v1",
apiKey: process.env.ROUTEWAY_API_KEY,
});
const response = await client.responses.create({
model: "gpt-4o",
instructions: "You are a concise assistant. Reply in one paragraph.",
input: "Explain how DNS works.",
max_output_tokens: 200,
temperature: 0.3,
});
console.log(response.output_text);
```
```bash theme={null}
curl https://api.routeway.ai/v1/responses \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o",
"instructions": "You are a concise assistant. Reply in one paragraph.",
"input": "Explain how DNS works.",
"max_output_tokens": 200,
"temperature": 0.3
}'
```
***
## Response Object
A successful response looks like this:
```json theme={null}
{
"id": "resp_abc123",
"object": "response",
"created_at": 1749052800,
"model": "gpt-4o-2024-11-20",
"status": "completed",
"output": [
{
"type": "message",
"id": "msg_abc123",
"role": "assistant",
"content": [
{
"type": "output_text",
"text": "DNS (Domain Name System) translates human-readable domain names into IP addresses..."
}
]
}
],
"output_text": "DNS (Domain Name System) translates human-readable domain names into IP addresses...",
"usage": {
"input_tokens": 35,
"output_tokens": 62,
"total_tokens": 97
}
}
```
### Key Fields
| Field | Description |
| ------------- | --------------------------------------------------------------------------- |
| `id` | Unique response identifier (used for multi-turn via `previous_response_id`) |
| `object` | Always `"response"` |
| `status` | `"completed"`, `"failed"`, `"in_progress"`, or `"cancelled"` |
| `output` | Array of output items (messages, tool calls, etc.) |
| `output_text` | Convenience field — the concatenated text content from all output messages |
| `model` | The exact model snapshot used |
| `usage` | Token counts for billing |
The `output_text` field is a shortcut. For simple text responses, use it directly instead of digging into `output[0].content[0].text`.
### Output Item Types
The `output` array can contain different item types:
| Type | Description |
| ----------------------- | ----------------------------------------------------- |
| `message` | A text response from the model |
| `function_call` | The model wants to call a custom function you defined |
| `web_search_call` | The model performed a web search (built-in tool) |
| `file_search_call` | The model searched uploaded files (built-in tool) |
| `code_interpreter_call` | The model executed code (built-in tool) |
***
## Status Values
| Status | Meaning |
| --------------- | ----------------------------------------------------------- |
| `"completed"` | Response generated successfully |
| `"failed"` | An error occurred during generation |
| `"in_progress"` | Response is still being generated (streaming or background) |
| `"cancelled"` | Response was cancelled before completion |
Always check the `status` field. A `"failed"` status means the response was not generated — check the `error` field on the response object for details.
***
## Accessing the Response Text
The simplest way to get the model's reply:
```python theme={null}
text = response.output_text
```
For more control, iterate over the `output` array:
```python theme={null}
for item in response.output:
if item.type == "message":
for content in item.content:
if content.type == "output_text":
print(content.text)
```
When the model calls a function, the output includes a `function_call` item:
```python theme={null}
for item in response.output:
if item.type == "function_call":
print(f"Function: {item.name}")
print(f"Arguments: {item.arguments}")
```
***
## Error Responses
Errors follow the same format as Chat Completions:
```json theme={null}
{
"error": {
"message": "Invalid model: 'gpt-999'.",
"type": "invalid_request_error",
"code": "model_not_found"
}
}
```
| HTTP status | Type | Common causes |
| ----------- | ----------------------- | ----------------------------------------------- |
| `400` | `invalid_request_error` | Bad parameters, invalid model, context too long |
| `401` | `authentication_error` | Missing or invalid API key |
| `429` | `rate_limit_error` | Too many requests or insufficient balance |
| `500` | `server_error` | Upstream provider error |
See the [Errors](/getting-started/errors) page for the full list and handling guidance.
# Authentication
Source: https://docs.routeway.ai/getting-started/authentication
Learn how to authenticate with the Routeway API using your API key.
All requests to the Routeway API must be authenticated using an API key. This key acts as your unique identifier and provides access to your account's usage and billing.
Never expose your API key publicly. Anyone with access to your key can use
your quota and rack up charges.
***
## Get Your API Key
New to APIs? You don't need any code to get started — just grab a key below and paste it into your chat app's settings. See our [Integrations Guide](/integrations/overview#chat-uis) or [Roleplay Platforms Guide](/integrations/overview#roleplay-platforms) for setup steps.
Go to the [Routeway dashboard](https://routeway.ai/dashboard) and sign in to your
account.
Once you're logged in, open the **API Keys** tab from the sidebar.
Click **Create API Key**, give it a name (e.g. `local-dev`, `production`), and
click **Confirm**.
Copy your new API key and store it securely — preferably in a `.env` file or a
secret manager like AWS Secrets Manager or HashiCorp Vault.
***
## Set Your API Key
You must make your API key available to your application. We recommend using environment variables to avoid hardcoding the key into your source code.
```bash theme={null}
export ROUTEWAY_API_KEY="your_key_here"
```
```bash theme={null}
setx ROUTEWAY_API_KEY "your_key_here"
```
Use `set` instead of `setx` if you only want to set the key for the current
session:
```bash theme={null}
set ROUTEWAY_API_KEY=your_key_here
```
***
## Authenticate in Code
```python theme={null}
import os
from openai import OpenAI
client = OpenAI(
api_key=os.getenv("ROUTEWAY_API_KEY"),
base_url="https://api.routeway.ai/v1"
)
```
```javascript theme={null}
import OpenAI from 'openai'
const openai = new OpenAI({
apiKey: process.env.ROUTEWAY_API_KEY,
baseURL: 'https://api.routeway.ai/v1',
})
```
```bash theme={null}
curl https://api.routeway.ai/v1/chat/completions \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o-mini",
"messages": [{"role": "user", "content": "Hello, world!"}]
}'
```
***
## API Key Format & Security
Routeway API keys usually begin with `sk-` and are 32+ characters long.
* Keep your keys secret.
* Never push them to GitHub (add `.env` to your `.gitignore`).
* Rotate keys regularly and revoke any you no longer use.
***
## Common Errors
```json theme={null}
{
"error": {
"message": "Invalid or missing API key",
"type": "error",
"tip": "Troubleshoot at: https://discord.gg/RjX2CpdPpd",
"code": 401
}
}
```
**Fix:** Make sure your `ROUTEWAY_API_KEY` is correctly set and passed to the client.
```json theme={null}
{
"error": {
"message": "API key does not have access to this resource",
"type": "authorization_error",
"code": "forbidden"
}
}
```
**Fix:** Check your key’s permissions in the dashboard. Some features may be restricted.
***
## Best Practices
Store API keys in `.env` files and use a secrets manager in production. Never commit them to version control.
Regularly rotate API keys and limit their scopes to the minimum required permissions.
Keep an eye on your API usage.
Never expose API keys in frontend code. Always make API calls from a backend service.
# How Billing Works
Source: https://docs.routeway.ai/getting-started/billing
Understand Routeway subscriptions, pay-as-you-go pricing, and payment methods.
Routeway offers two billing models: **Pay-As-You-Go** for flexible, usage-based access to every model, and **DeepSeek Subscriptions** for teams with consistent DeepSeek workloads.
Access every model. Pay only for what you use — no commitments, no minimums.
Reserved token budgets and bonus perks for consistent DeepSeek usage.
***
## Pay-As-You-Go
PAYG gives you access to every model on the Routeway API — OpenAI, Anthropic, Gemini, DeepSeek, Llama, and more. You're charged only for the tokens you process, with no monthly commitment required.
Every request is billed in two parts:
| Component | What it covers |
| :---------------- | :---------------------------------- |
| **Input tokens** | The prompt and any context you send |
| **Output tokens** | The text generated in the response |
Rates are quoted **per 1 million tokens** and vary by model. You can view live rates in the [Models Catalog](https://routeway.ai/models) or query them via the API.
Some models support **flex** and **priority** processing tiers in addition to the default tier. These are provider-side features from OpenAI and Gemini — Routeway passes your tier through to the upstream provider and bills you at the corresponding rate. Routeway does not run its own queue or priority routing.
| Tier | Cost | Trade-off |
| :----------- | :--------------- | :--------------------------------------------------- |
| **Default** | Standard pricing | Balanced TTFT |
| **Flex** | 50% off | Same output quality; TTFT may occasionally be higher |
| **Priority** | 2× standard | Lower, more consistent TTFT |
Set a tier via the `service_tier` request parameter, or append `:flex` / `:priority` to the model ID. Currently available on select OpenAI and Gemini models only — check `capabilities.service_tiers` in the model object.
See [Service Tiers](/guides/chat-completions/service-tiers) for full details.
Caching support varies by model. Check the `pricing.caching` field returned by the API (or the [Models Catalog](https://routeway.ai/models)) to see what a specific model supports.
| Tier | When you're charged | Billing key |
| :-------------------------- | :----------------------------------------------------- | :----------------- |
| **Cache read** | A cached prompt is reused (cheaper than a fresh input) | `caching.read` |
| **Cache write (5 min TTL)** | A prompt is written into cache, kept for 5 minutes | `caching.write.5m` |
| **Cache write (1 hr TTL)** | A prompt is written into cache, kept for 1 hour | `caching.write.1h` |
Not every model has every tier:
* **Read-only** (e.g. Kimi K2.7 Code) — cache hits are discounted; there is no explicit write charge.
* **Read + write** (e.g. Anthropic models) — both reads and writes are billed, with write rates varying by TTL.
* **No caching** — `pricing.caching` is absent from the model's pricing object.
***
## DeepSeek Subscriptions
Subscriptions include a reserved DeepSeek token budget and free daily model requests across all plans.
| | Biweekly | Monthly |
| :--------------------------- | :-------- | :-------- |
| **Price** | \$6.99 | \$15.99 |
| **Included DeepSeek tokens** | 15M | 35M |
| **Free model requests** | 250 / day | 250 / day |
| | Biweekly | Monthly |
| :--------------------------- | :-------- | :-------- |
| **Price** | \$12.99 | \$34.99 |
| **Included DeepSeek tokens** | 28M | 80M |
| **Free model requests** | 300 / day | 330 / day |
| **PAYG text discount** | 5% off | 5% off |
Core Pro includes an additional **5% discount** on all Pay-As-You-Go text usage.
Subscription tokens are consumed first. Once exhausted, usage automatically continues on PAYG (unless you disable key fallback in the dashboard).
***
## Cost Examples
Routeway calculates your cost using the standard token billing formula:
$\text{Total Cost} = (\text{Input Tokens} \times \text{Input Rate}) + (\text{Output Tokens} \times \text{Output Rate})$
**`deepseek-v3`** — $0.23/1M input · $0.64/1M output
| | Tokens | Cost |
| :--------------------- | :-------- | :------------ |
| Input (≈ 750 words) | 1,000 | \$0.00023 |
| Output (≈ 1,500 words) | 2,000 | \$0.00128 |
| **Total** | **3,000** | **\$0.00151** |
**`kimi-k2.7-code`** — $0.95/1M input · $4.00/1M output · \$0.20/1M cache read
Suppose 800 of your 1,000 input tokens were already cached:
| | Tokens | Rate | Cost |
| :---------- | :----- | :-------- | :------------ |
| Fresh input | 200 | \$0.95/1M | \$0.00019 |
| Cache read | 800 | \$0.20/1M | \$0.00016 |
| Output | 2,000 | \$4.00/1M | \$0.00800 |
| **Total** | | | **\$0.00835** |
Without caching the same request would cost **\$0.00950** — a \~12% saving.
**`claude-fable-5`** — $10.00/1M input · $50.00/1M output · $1.00/1M cache read · $12.50/1M cache write (5 min)
First request — prompt is written into cache:
| | Tokens | Rate | Cost |
| :------------------ | :----- | :--------- | :------------ |
| Cache write (5 min) | 1,000 | \$12.50/1M | \$0.01250 |
| Output | 500 | \$50.00/1M | \$0.02500 |
| **Total** | | | **\$0.03750** |
Subsequent requests within 5 minutes — same prompt served from cache:
| | Tokens | Rate | Cost |
| :--------- | :----- | :--------- | :------------ |
| Cache read | 1,000 | \$1.00/1M | \$0.00100 |
| Output | 500 | \$50.00/1M | \$0.02500 |
| **Total** | | | **\$0.02600** |
Each cache hit saves **\$0.01150** compared to a fresh input read.
***
## Live Pricing
View rates anytime in the [Interactive Models Catalog](https://routeway.ai/models), or fetch them programmatically:
```bash theme={null}
curl https://api.routeway.ai/v1/models
```
Each model returns a `pricing` object. The `caching` field is only present when the model supports it, and its shape reflects exactly what that model offers:
```json theme={null}
{
"id": "deepseek-v3",
"pricing": {
"input": { "unit": "1M tokens", "price_per_million_t": 0.23 },
"output": { "unit": "1M tokens", "price_per_million_t": 0.64 }
}
}
```
```json theme={null}
{
"id": "kimi-k2.7-code",
"pricing": {
"input": { "unit": "1M tokens", "price_per_million_t": 0.95 },
"output": { "unit": "1M tokens", "price_per_million_t": 4.0 },
"caching": {
"read": { "unit": "1M tokens", "price_per_million_t": 0.2 }
}
}
}
```
```json theme={null}
{
"id": "claude-fable-5",
"pricing": {
"input": { "unit": "1M tokens", "price_per_million_t": 10.0 },
"output": { "unit": "1M tokens", "price_per_million_t": 50.0 },
"caching": {
"read": { "unit": "1M tokens", "price_per_million_t": 1.0 },
"write": {
"5m": { "unit": "1M tokens", "price_per_million_t": 12.5 },
"1h": { "unit": "1M tokens", "price_per_million_t": 20.0 }
}
}
}
}
```
***
## Payment Methods
Payments are processed securely through **[Polar.sh](https://polar.sh)**:
* Credit / Debit cards (Visa, Mastercard, Amex, and more)
* Automatic invoicing
**Your card details never touch Routeway's servers.** All transactions are handled exclusively by Polar.sh, which is fully PCI-compliant.
**PayPal is supported**, though it isn't available by default. For other payment methods, availability may vary — reach out to us and we'll let you know what's possible:
* **Discord**: Join [discord.gg/RjX2CpdPpd](https://discord.gg/RjX2CpdPpd) and open a ticket — this is the fastest way to reach us.
* **Email**: [help@team.routeway.ai](mailto:help@team.routeway.ai)
# Error Handling
Source: https://docs.routeway.ai/getting-started/errors
Learn how to handle and troubleshoot errors returned by the Routeway API.
Routeway uses standard HTTP response codes to indicate the success or failure of an API request. Error responses contain a detailed JSON body that mirrors the OpenAI error schema, making it easy to parse errors programmatically.
## Error Response Schema
All API errors return a standard JSON object containing an `error` payload:
```json theme={null}
{
"error": {
"message": "Invalid or missing API key",
"type": "invalid_request_error",
"code": 401,
"tip": "Ensure your API key is correctly passed in the Authorization header: Bearer ",
"trace_id": "req_01j0x7p6..."
}
}
```
### Fields
| Field | Type | Description |
| :--------- | :--------------- | :-------------------------------------------------------------------------------------- |
| `message` | `string` | A human-readable description of the error. |
| `type` | `string` | The category of error (e.g., `invalid_request_error`, `rate_limit_error`, `api_error`). |
| `code` | `integer/string` | The HTTP status code or a custom error code. |
| `tip` | `string` | An optional helpful suggestion to quickly resolve the error. |
| `trace_id` | `string` | A unique identifier for the request, useful when contacting support. |
***
## HTTP Status Codes
| Status Code | Error Type | Description |
| :---------- | :---------------------- | :------------------------------------------------------------------------------------------------------- |
| **400** | `invalid_request_error` | **Bad Request** - Invalid JSON body, missing required parameters, or invalid parameters. |
| **401** | `authentication_error` | **Unauthorized** - Missing or incorrect API key, or the key has been revoked. |
| **403** | `permission_error` | **Forbidden** - The key lacks permissions for the requested model or endpoint. |
| **404** | `invalid_request_error` | **Not Found** - The requested endpoint does not exist. |
| **422** | `invalid_request_error` | **Unprocessable Entity** - Schema validation failed, or the input tokens exceeded the model's limit. |
| **429** | `rate_limit_error` | **Too Many Requests** - You have exceeded your tier rate limits or custom key limits. |
| **500** | `api_error` | **Internal Server Error** - An unexpected error occurred on Routeway's servers. |
| **502** | `provider_error` | **Bad Gateway** - The downstream AI provider (e.g., Anthropic, DeepSeek) returned an error or timed out. |
***
## Catching Errors in Code
Since Routeway uses OpenAI-compatible responses, the official OpenAI SDKs will automatically parse and throw structured exceptions.
```javascript theme={null}
const { OpenAI } = require("openai");
const client = new OpenAI({
base_url: "https://api.routeway.ai/v1",
api_key: process.env.ROUTEWAY_API_KEY,
});
async function run() {
try {
const response = await client.chat.completions.create({
model: "gpt-4o-mini",
messages: [{ role: "user", content: "Hello!" }],
});
} catch (error) {
if (error instanceof OpenAI.APIError) {
console.error(`Status: ${error.status}`); // e.g., 401
console.error(`Message: ${error.message}`);
console.error(`Type: ${error.type}`);
} else {
console.error("Non-API Error:", error);
}
}
}
run();
```
```python theme={null}
import os
from openai import OpenAI, APIError
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
try:
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Hello!"}]
)
except APIError as e:
print(f"Status: {e.status_code}") # e.g., 401
print(f"Message: {e.message}")
print(f"Type: {e.code}")
except Exception as e:
print(f"Non-API Error: {e}")
```
# Frequently Asked Questions
Source: https://docs.routeway.ai/getting-started/faq
Answers to common questions about Routeway — billing, models, APIs, and more.
## General
Routeway is a unified API gateway that gives you access to 200+ AI models — from OpenAI, Anthropic, Google, Meta, DeepSeek, MoonshotAI, and more — through a single endpoint and a single API key.
Instead of managing separate accounts, SDKs, and billing across every provider, you point your existing OpenAI-compatible client at `https://api.routeway.ai/v1` and swap models by changing a single string.
When you call providers directly, you get one model per account, separate billing, separate keys, and separate SDKs to manage. With Routeway:
* One API key and one base URL for everything
* Switch between 200+ models without changing your client code
* Single dashboard for usage, spend, and key management
* Access models from providers you may not have a direct account with
Yes. Routeway is fully OpenAI-compatible. You can use the official OpenAI Python or Node.js SDK — just change `base_url` to `https://api.routeway.ai/v1` and swap your key. No other code changes required.
Anthropic-style requests (`/v1/messages`) are also supported for models that expose that interface.
Browse the full list at [routeway.ai/models](https://routeway.ai/models) or fetch it live from the API:
```bash theme={null}
curl https://api.routeway.ai/v1/models \
-H "Authorization: Bearer $ROUTEWAY_API_KEY"
```
Models from OpenAI, Anthropic, Google, Meta (Llama), DeepSeek, MoonshotAI, and others are available. The catalog is updated regularly as new models are released.
***
## Account & API Keys
1. Sign up or log in at [routeway.ai/dashboard](https://routeway.ai/dashboard)
2. Open the **API Keys** tab
3. Click **Create API Key**, give it a name, and confirm
4. Copy and store the key securely — it won't be shown again
See [Authentication](/getting-started/authentication) for full setup instructions.
Routeway API keys start with `sk-` and are 32+ characters long. If your key doesn't match this format, you may be using a key from a different provider.
Yes. You can create as many keys as you need from the dashboard — for example, separate keys per project, environment, or team member. Each key can be named, monitored, and revoked independently.
Revoke it immediately from the **API Keys** section of the [dashboard](https://routeway.ai/dashboard) and create a new one. Check your recent usage in the **Activity** tab to see if there were any unexpected requests.
Yes. Key-level controls are available in the dashboard — you can set spending limits and configure fallback behaviour per key. See [Budget Controls](/getting-started/billing#budget-controls).
***
## Models
Free models have `:free` appended to their ID (e.g. `deepseek-r1:free`). They cost \$0.00 but are rate-limited to **20 requests/minute** and **200 requests/day**.
Paid (PAYG) models have no suffix and are billed per token with no rate limits at the API level. They require a positive account balance.
See [Available Models](/getting-started/models) for more detail.
No — free models are intended for **development, testing, and experimentation only**. Before building anything serious on them, keep in mind:
* **They can be removed at any time.** Free model availability depends on provider capacity and our ability to continue offering them. A model that exists today may be gone tomorrow without notice.
* **They can be unstable.** Free tiers are lower priority in our routing stack, so you may see higher error rates, slower response times, and more frequent `502` upstream errors compared to PAYG models.
* **Rate limits are strict and shared.** 20 RPM and 200 RPD are hard caps with no way to increase them. Under load, you'll hit these quickly.
* **No SLA.** There are no uptime or latency guarantees on free models.
If your project is moving beyond prototyping, switch to the PAYG equivalent (drop the `:free` suffix). The per-token cost on most models is low enough that real workloads are still very affordable.
The daily limit (200 RPD) resets at **00:00 UTC** each day. The per-minute limit (20 RPM) is a rolling window.
Each model in the `/v1/models` response includes a `capabilities` object:
```json theme={null}
"capabilities": {
"vision": true,
"function_call": true,
"reasoning": true
}
```
You can also filter by capability on the [Models Catalog](https://routeway.ai/models).
Some models support optional processing tiers that change both cost and performance. These are **provider features** from OpenAI and Gemini — Routeway forwards your tier choice upstream and applies the price multiplier on your bill. Routeway does not run its own queue or priority system.
* **Flex** — 50% cheaper with the same output quality. Time-to-first-token may occasionally be higher, but generation speed is unchanged. Great for cost-sensitive or high-volume usage.
* **Priority** — 2× the standard price, with lower and more consistent TTFT. Good for user-facing apps where users are waiting on the first token.
Set a tier with `service_tier: "flex"` in your request, or append `:flex` / `:priority` to the model ID (e.g. `gpt-5.6-luna:flex`). Currently available on select OpenAI and Gemini models only — check `capabilities.service_tiers` in the model metadata.
See [Service Tiers](/guides/chat-completions/service-tiers) for details.
The `endpoints` array in the model object lists all supported API paths, e.g.:
```json theme={null}
"endpoints": ["/v1/chat/completions", "/v1/responses", "/v1/messages"]
```
Not all models support all paths. Image models are only available on `/v1/images/generations` or `/v1/images/edits` and won't appear on chat completion routes.
It means the model is temporarily unavailable for routing, usually due to an issue with the upstream provider. Check back shortly or switch to an alternative model. If a model you rely on has been unavailable for an extended period, reach out on [Discord](https://discord.gg/RjX2CpdPpd).
Yes, for models that list `/v1/messages` in their `endpoints`. Point the Anthropic SDK at `https://api.routeway.ai/v1/messages` and use your Routeway API key.
We add new models continuously as they are released by providers or requested by our community.
To get notified immediately when a new model is added, you can join our [Discord Server](https://discord.gg/Mynq6vGVPE) and check the [#models](https://discord.com/channels/1231339329435996280/1411766223011582062) channel.
***
## Billing & Pricing
Routeway uses **Pay-As-You-Go** pricing — you're charged only for the tokens you process, with no monthly commitment. Costs are split between input tokens (prompt + context) and output tokens (generated response), billed per 1 million tokens at rates that vary by model.
See [How Billing Works](/getting-started/billing) for full details and cost examples.
Tokens are the chunks of text AI models process. In English, 1 token is roughly 4 characters or 0.75 words. A typical paragraph is around 100 tokens.
As a rough guide:
* 1,000 tokens ≈ 750 words
* 1M tokens ≈ a full-length novel
Exact tokenisation varies by model. Most providers publish tokeniser tools you can use to measure a specific prompt.
No. Routeway is strictly Pay-As-You-Go — there are no pre-paid credit bundles to buy. Your account is charged based on actual usage, and you only need a valid payment method on file.
Payments are processed by **Polar.sh** (fully PCI-compliant). Accepted methods:
* Credit and debit cards (Visa, Mastercard, Amex, and more)
* Automatic invoicing
Routeway never stores or touches your card details directly.
**PayPal is supported**, though it isn't available by default. For other payment methods, availability may vary — reach out to us and we'll let you know what's possible:
* **Discord**: Join [discord.gg/RjX2CpdPpd](https://discord.gg/RjX2CpdPpd) and open a support ticket — this is the fastest way to reach us.
* **Email**: [help@team.routeway.ai](mailto:help@team.routeway.ai)
Subscriptions give you a reserved monthly or biweekly DeepSeek token budget plus free daily model requests across all plans — useful if you want a predictable DeepSeek allowance.
| Plan | Biweekly | Monthly |
| :----------- | :------------------- | :------------------- |
| **Core** | \$6.99 · 15M tokens | \$15.99 · 35M tokens |
| **Core Pro** | \$12.99 · 28M tokens | \$34.99 · 80M tokens |
Core Pro also includes a 5% discount on all PAYG text usage. See [Billing](/getting-started/billing#deepseek-subscriptions).
Usage automatically continues on PAYG unless you've disabled key fallback in the dashboard. You can turn this off per-key to hard-cap DeepSeek spend.
It depends on the model. Some models offer cache reads at a discounted rate with no write charge (e.g. Kimi K2.7 Code). Others bill both reads and writes, with write rates varying by TTL (e.g. Anthropic models: 5-minute and 1-hour tiers).
Some models have no caching at all. Check `pricing.caching` in the model's API response, or see the [Billing — Prompt Caching](/getting-started/billing#prompt-caching) section for examples.
Set a **hard spending limit** from the [dashboard](https://routeway.ai/dashboard). Once your monthly threshold is hit, the API returns `429` errors for that key until the limit resets or is raised. See [Budget Controls](/getting-started/billing#budget-controls).
The [dashboard](https://routeway.ai/dashboard) shows real-time usage, token breakdowns by model, and historical spend. You can also query it programmatically via the Account API endpoints (`/v1/account/balance`, `/v1/account/activity`).
Because Routeway is Pay-As-You-Go and you only pay for usage already consumed, refunds are not available for processed tokens. If you believe you were charged in error, contact support at [help@team.routeway.ai](mailto:help@team.routeway.ai) or on [Discord](https://discord.gg/RjX2CpdPpd).
***
## API & Integration
| Endpoint | Purpose |
| :---------------------------- | :------------------------------------- |
| `POST /v1/chat/completions` | Text generation (OpenAI-compatible) |
| `POST /v1/responses` | OpenAI Responses API |
| `POST /v1/messages` | Anthropic-style messages |
| `POST /v1/images/generations` | Image generation from a text prompt |
| `POST /v1/images/edits` | Image editing with a mask and prompt |
| `GET /v1/models` | List all available models with pricing |
Not every model is available on every endpoint — check the model's `endpoints` array.
A `401` means your API key is missing, invalid, or revoked. Check that:
* You're passing the key as `Authorization: Bearer `
* The key starts with `clsk-` and was copied in full
* The key hasn't been deleted in the dashboard
See [Authentication](/getting-started/authentication) and [Error Handling](/getting-started/errors).
A `429` means you've hit a rate limit or spending cap:
* **Free models**: 20 RPM / 200 RPD limit reached — wait for the reset or switch to a PAYG model
* **Hard spending limit**: your monthly cap was hit — raise or remove it in the dashboard
* **Key-level limit**: a per-key restriction was triggered — check your key settings
See [Rate Limits](/getting-started/rate-limits).
A `502` means the upstream AI provider returned an error or timed out. This is usually temporary. Retry with exponential backoff, or switch to an equivalent model from a different provider. If the issue persists, check [Discord](https://discord.gg/RjX2CpdPpd) for status updates.
Yes. Pass `"stream": true` in your request body. The response will be delivered as server-sent events (SSE), identical to OpenAI's streaming format. To receive usage data in a streaming response, add `"stream_options": {"include_usage": true}`.
Yes, for models where `capabilities.function_call` is `true`. The request and response format is identical to OpenAI's tool-calling spec.
Yes. Any framework that supports a custom OpenAI base URL will work. Set `base_url` (or equivalent) to `https://api.routeway.ai/v1` and use your Routeway API key. No other changes needed.
Yes. See the [Integrations Overview](/integrations/overview) page for step-by-step setup guides for Lobe Chat, JanitorAI, ChubAI, Cline, Roo Code, and more.
***
## Still stuck?
Can't find what you're looking for? Reach out:
* **Discord**: [discord.gg/RjX2CpdPpd](https://discord.gg/RjX2CpdPpd) — fastest response
* **Email**: [help@team.routeway.ai](mailto:help@team.routeway.ai)
When reporting an issue, include the `trace_id` from the error response — it lets us find your request instantly.
# Introduction
Source: https://docs.routeway.ai/getting-started/introduction
Routeway is a unified, OpenAI-compatible API gateway providing access to 200+ LLMs and image models.
Routeway is an API gateway that enables you to integrate and switch between 200+ AI models (including GPT-5.5, Claude Opus 4.8, Kimi K2.7, DeepSeek V4, Grok 4, and Gemini 3.1 Pro) using a single, unified, OpenAI-compatible API.
Because it is fully compatible with OpenAI, migrating your existing applications takes only a few minutes. See the [Migration Guide](/getting-started/migration) for step-by-step instructions.
Instead of managing multiple SDKs, credentials, and billing accounts, you only need one client configuration and one API key.
## Base URL
All API requests are routed through:
```http theme={null}
https://api.routeway.ai/v1
```
## Quick Example (cURL)
Test your connection with a standard Chat Completions request:
```bash theme={null}
curl https://api.routeway.ai/v1/chat/completions \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "kimi-k2.7-code",
"messages": [
{"role": "user", "content": "Say hello!"}
]
}'
```
## Explore the Docs
Make your first API request with Python, Node.js, or cURL in under 2 minutes.
Migrate your existing OpenAI or Anthropic integration in a few minutes.
Learn how to obtain and securely configure your Routeway API keys.
Browse the list of supported models, pricing, and routing paths.
Explore the full interactive endpoints for chat, images, and accounts.
## Guides
Bundle a model, system prompt, and parameters into a reusable ID.
Receive tokens as they are generated for low-latency, real-time applications.
Let models invoke your functions to fetch live data and take actions.
Generate images and search the web in chat — just append `@flux-2-turbo` or `@online` to any model.
Force models to return valid JSON matching a schema you define.
Use extended thinking models for hard problems that require deeper deliberation.
Send images, PDFs, and documents alongside text in a single request.
Reduce costs and latency by reusing previously processed prompt tokens.
# Migration Guide
Source: https://docs.routeway.ai/getting-started/migration
Learn how to migrate your existing AI applications to Routeway in a few minutes.
Routeway is designed to be a drop-in replacement for the OpenAI API. Since the API layout, request bodies, and response structures are fully compatible, migrating your existing application requires minimal changes.
## 1. Update the Base URL & Authentication
To point your requests to Routeway, you only need to change two configurations: the endpoint URL and your API key.
| Provider | API Endpoint | Authentication Header |
| :----------- | :--------------------------- | :--------------------------------------- |
| **OpenAI** | `https://api.openai.com/v1` | `Authorization: Bearer OPENAI_API_KEY` |
| **Routeway** | `https://api.routeway.ai/v1` | `Authorization: Bearer ROUTEWAY_API_KEY` |
In the authentication header, `ROUTEWAY_API_KEY` represents your actual Routeway API key (e.g., `sk-...`).
***
## 2. SDK Integration Examples
If you are using official OpenAI client libraries, you don't need to install any new dependencies. Simply update the configuration during client initialization.
Change your import/initialization to override the `base_url` and use your `ROUTEWAY_API_KEY`:
```python Before (OpenAI) theme={null}
import os
from openai import OpenAI
client = OpenAI(
api_key=os.getenv("OPENAI_API_KEY")
)
# Calls OpenAI gpt-4o-mini
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Hello!"}]
)
```
```python After (Routeway) theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
# Calls Routeway gpt-4o-mini (or any other of the 200+ supported models)
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Hello!"}]
)
```
Update the client initialization options to point to `https://api.routeway.ai/v1`:
```javascript Before (OpenAI) theme={null}
const { OpenAI } = require("openai");
const client = new OpenAI({
apiKey: process.env.OPENAI_API_KEY,
});
async function main() {
const response = await client.chat.completions.create({
model: "gpt-4o-mini",
messages: [{ role: "user", content: "Hello!" }]
});
}
```
```javascript After (Routeway) theme={null}
const { OpenAI } = require("openai");
const client = new OpenAI({
baseURL: "https://api.routeway.ai/v1",
apiKey: process.env.ROUTEWAY_API_KEY,
});
async function main() {
const response = await client.chat.completions.create({
model: "gpt-4o-mini",
messages: [{ role: "user", content: "Hello!" }]
});
}
```
***
## 3. Migrating from Anthropic (Claude)
If you are already using the Anthropic SDK, you don't need to switch to the OpenAI SDK. Simply update the `base_url` to point to Routeway:
```python Before (Anthropic) theme={null}
import os
from anthropic import Anthropic
client = Anthropic(
api_key=os.getenv("ANTHROPIC_API_KEY"),
)
message = client.messages.create(
model="claude-opus-4.8",
max_tokens=1024,
messages=[{"role": "user", "content": "Hello, Claude!"}],
)
print(message.content)
```
```python After (Routeway) theme={null}
import os
from anthropic import Anthropic
client = Anthropic(
api_key=os.getenv("ROUTEWAY_API_KEY"),
base_url="https://api.routeway.ai",
)
message = client.messages.create(
model="claude-opus-4.8",
max_tokens=1024,
messages=[{"role": "user", "content": "Hello, Claude!"}],
)
print(message.content)
```
***
## 4. Other OpenAI-Compatible Gateways (e.g., DeepInfra, OpenRouter, Together AI)
If you are currently using the OpenAI SDK configured for another proxy, gateway, or compatibility layer (like DeepInfra, OpenRouter, Together AI, Groq, or Open WebUI), migrating is extremely simple.
You only need to update the client configuration references:
* **Base URL**: Change the base URL of your client to `https://api.routeway.ai/v1`.
* **API Key**: Replace your provider-specific API key with your `ROUTEWAY_API_KEY`.
* **Model Identifier**: Update the `model` parameter to one of Routeway's supported models (e.g. `gpt-4o`, `claude-3-5-sonnet`, `deepseek-v3`).
***
## 5. General HTTP / REST Migration
If you are not using an SDK and instead make direct HTTP requests (e.g., using `fetch`, `axios`, or Python `requests`), follow these general guidelines to migrate from any other provider:
1. **Endpoint**: Point your POST requests to:
```http theme={null}
https://api.routeway.ai/v1/chat/completions
```
2. **Headers**: Provide your API key in the standard bearer token format:
```http theme={null}
Authorization: Bearer YOUR_ROUTEWAY_API_KEY
Content-Type: application/json
```
3. **Payload Structure**: Ensure your JSON body uses the standard Chat Completions format:
```json theme={null}
{
"model": "model-id-here",
"messages": [
{
"role": "user",
"content": "Your prompt goes here"
}
]
}
```
See our [Models](/getting-started/models) page for the full list of model identifiers available on Routeway.
# Available Models
Source: https://docs.routeway.ai/getting-started/models
Browse, filter, and query the full catalog of models available through the Routeway API.
Routeway provides access to 200+ models from leading providers — OpenAI, Anthropic, Google, Meta, DeepSeek, MoonshotAI, and more — through a single unified endpoint.
Browse all models with live pricing, context lengths, and capability filters.
Fetch the full model list programmatically with pricing and metadata.
***
## Model Tiers
Free models are identified by a `:free` suffix in their ID (e.g. `deepseek-r1:free`).
| | |
| :------------- | :------------------------------------------- |
| **Cost** | \$0.00 — all tokens are free |
| **Rate limit** | 20 requests / minute · 200 requests / day |
| **Best for** | Development, testing, low-volume prototyping |
Exceeding the rate limit returns a `429`. See [Rate Limits](/getting-started/rate-limits).
All model IDs without a `:free` suffix are PAYG models.
| | |
| :------------- | :------------------------------------------------ |
| **Cost** | Per-token, billed per 1M tokens (varies by model) |
| **Rate limit** | No API-level limit; edge security limits apply |
| **Best for** | Production workloads at any scale |
See [How Billing Works](/getting-started/billing) for pricing details and cost examples.
***
## Listing Models Programmatically
Fetch the live catalog — including pricing, capabilities, and availability — with a single request:
```bash cURL theme={null}
curl https://api.routeway.ai/v1/models
```
```python Python theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
models = client.models.list()
for model in models.data:
print(model.id)
```
```javascript Node.js theme={null}
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.routeway.ai/v1",
apiKey: process.env.ROUTEWAY_API_KEY,
});
const models = await client.models.list();
models.data.forEach((model) => console.log(model.id));
```
### Response shape
```json theme={null}
{
"object": "list",
"data": [
{
"id": "kimi-k2.7-code",
"name": "MoonshotAI: Kimi K2.7 Code",
"short_name": "Kimi K2.7 Code",
"description": "Kimi K2.7 Code is Moonshot AI's coding-focused agentic model...",
"context_length": 262144,
"created": 1781222400,
"owned_by": "moonshot-ai",
"available": true,
"endpoints": ["/v1/chat/completions", "/v1/responses", "/v1/messages"],
"pricing": {
"input": { "unit": "1M tokens", "price_per_million_t": 0.95 },
"output": { "unit": "1M tokens", "price_per_million_t": 4.0 },
"caching": {
"read": { "unit": "1M tokens", "price_per_million_t": 0.2 }
}
},
"capabilities": {
"vision": true,
"function_call": true,
"reasoning": true
},
"supported_parameters": ["temperature", "top_p", "tools", "tool_choice", "..."],
"benchmarks": {
"gpqa": 0.866,
"lcr": 0.733,
"tau2": 0.942
}
}
]
}
```
***
## Model Schema
### Core fields
| Field | Type | Description |
| :--------------- | :--------- | :-------------------------------------------------------------------------------------- |
| `id` | `string` | The model ID to use in API requests (e.g. `"kimi-k2.7-code"`). |
| `name` | `string` | Full display name including provider (e.g. `"MoonshotAI: Kimi K2.7 Code"`). |
| `short_name` | `string` | Abbreviated name without the provider prefix. |
| `description` | `string` | Description of the model's capabilities and intended use. |
| `context_length` | `integer` | Maximum context window in tokens. |
| `created` | `integer` | Unix timestamp of when the model became available on Routeway. |
| `owned_by` | `string` | Originating provider (e.g. `"openai"`, `"anthropic"`, `"moonshot-ai"`). |
| `available` | `boolean` | `true` if the model is currently routable. `false` means it is temporarily unavailable. |
| `endpoints` | `string[]` | API paths this model can be called on (see [Endpoints](#endpoints) below). |
### Pricing fields
| Field | Type | Description |
| :------------------------- | :------- | :------------------------------------------------------------------- |
| `pricing.input` | `object` | Rate for standard input tokens (prompt + context). |
| `pricing.output` | `object` | Rate for generated output tokens. |
| `pricing.caching` | `object` | Present only on models that support prompt caching. |
| `pricing.caching.read` | `object` | Rate for tokens served from cache (always cheaper than fresh input). |
| `pricing.caching.write.5m` | `object` | Rate to write a prompt into cache with a 5-minute TTL. |
| `pricing.caching.write.1h` | `object` | Rate to write a prompt into cache with a 1-hour TTL. |
All rates use `price_per_million_t` (USD per 1M tokens). See [How Billing Works](/getting-started/billing#prompt-caching) for caching cost examples.
### Capabilities
| Field | Type | Description |
| :--------------------------- | :--------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `capabilities.vision` | `boolean` | Accepts image inputs. |
| `capabilities.function_call` | `boolean` | Supports tool / function calling. |
| `capabilities.reasoning` | `boolean` | Exposes a chain-of-thought reasoning mode (e.g. via `reasoning_effort`). |
| `capabilities.service_tiers` | `string[]` | Processing tiers the model supports: `default`, `flex`, and/or `priority`. Currently available on select OpenAI and Gemini models. See [Service Tiers](/guides/chat-completions/service-tiers). |
### Service tier pricing
Present only on models that support service tiers. Multipliers are applied to the standard `pricing.input` and `pricing.output` rates.
| Field | Type | Description |
| :---------------------------------------- | :------- | :------------------------------------------------------------------------ |
| `service_tiers.flex.price_multiplier` | `number` | Cost multiplier for flex processing (e.g. `0.5` = 50% discount). |
| `service_tiers.priority.price_multiplier` | `number` | Cost multiplier for priority processing (e.g. `2.0` = 2× standard price). |
***
### Benchmarks
Common benchmark scores included where available. Fields are `null` when a score has not been published.
| Field | What it measures |
| :--------------------------------------- | :------------------------------------------------- |
| `gpqa` | Graduate-level science reasoning (GPQA Diamond) |
| `hle` | Humanity's Last Exam |
| `lcr` | LiveCodeBench — competitive coding |
| `tau2` | Tool-use & agent task completion (TAU2) |
| `ifbench` | Instruction following |
| `terminalbench_hard` | Terminal / shell task completion |
| `scicode` | Scientific coding |
| `artificial_analysis_intelligence_index` | Composite intelligence score (Artificial Analysis) |
| `artificial_analysis_coding_index` | Coding-specific composite (Artificial Analysis) |
***
## Endpoints
Each model advertises which API paths it supports in the `endpoints` array. Not all models are available on every path.
| Endpoint | Compatible with | Used for |
| :----------------------- | :--------------------------------------- | :---------------------------------------- |
| `/v1/chat/completions` | OpenAI SDK, any OpenAI-compatible client | Text generation (broadest model coverage) |
| `/v1/responses` | OpenAI Responses API | Text generation |
| `/v1/messages` | Anthropic SDK | Text generation |
| `/v1/images/generations` | OpenAI SDK (`images.generate`) | Image generation from a text prompt |
| `/v1/images/edits` | OpenAI SDK (`images.edit`) | Image editing with a mask and prompt |
Use `/v1/chat/completions` as your default for text models. It has the broadest model coverage and is compatible with any OpenAI-compatible library. Image models are only available on their respective `/v1/images/*` paths.
***
## Filtering by Capability
You can filter the response client-side to find models that match your requirements:
```python theme={null}
import requests
data = requests.get("https://api.routeway.ai/v1/models").json()
vision_models = [
m
for m in data["data"]
if m.get("available") and m.get("capabilities", {}).get("vision")
]
for m in vision_models:
print(m["id"], "— input $", m["pricing"]["input"]["price_per_million_t"], "/ 1M")
```
# Quickstart
Source: https://docs.routeway.ai/getting-started/quickstart
Get started with Routeway. Learn how to make your first API call in a few minutes.
Go to the [Routeway Dashboard](https://routeway.ai/dashboard) and log in.
1. Open the **API Keys** section.
2. Click **Create API Key**.
3. Copy your key and keep it secure.
New to APIs? You don't need any code to get started — just grab a key and paste it into your chat app's settings. See our [Integrations Guide](/integrations/overview#chat-uis) or [Roleplay Platforms Guide](/integrations/overview#roleplay-platforms) for setup steps.
Never share your API key or commit it to version control. Anyone with your key can access your billing and use your quota.
Store your API key as an environment variable to authenticate SDKs and client libraries automatically.
```bash theme={null}
export ROUTEWAY_API_KEY="your_key_here"
```
```cmd theme={null}
set ROUTEWAY_API_KEY="your_key_here"
```
```powershell theme={null}
$env:ROUTEWAY_API_KEY="your_key_here"
```
Since Routeway is fully compatible with the OpenAI API, you can use the official OpenAI SDKs or raw HTTP requests.
Install the official OpenAI Python library:
```bash theme={null}
pip install openai
```
Create a new file and run the following script:
```python theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "user", "content": "Write a short story about a robot and a cat."}
]
)
print(response.choices[0].message.content)
```
Install the official OpenAI Node.js library:
```bash theme={null}
npm install openai
```
Create a new file and run the following script:
```javascript theme={null}
const { OpenAI } = require("openai");
const client = new OpenAI({
base_url: "https://api.routeway.ai/v1",
api_key: process.env.ROUTEWAY_API_KEY,
});
async function main() {
const response = await client.chat.completions.create({
model: "gpt-4o-mini",
messages: [
{ role: "user", content: "Write a short story about a robot and a cat." }
]
});
console.log(response.choices[0].message.content);
}
main().catch(console.error);
```
Send a raw HTTP request directly using cURL:
```bash theme={null}
curl https://api.routeway.ai/v1/chat/completions \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o-mini",
"messages": [
{"role": "user", "content": "Write a short story about a robot and a cat."}
]
}'
```
# Rate Limits
Source: https://docs.routeway.ai/getting-started/rate-limits
Understand Routeway rate limits and learn how to handle them in production.
Routeway applies rate limits to ensure fair resource allocation and service stability. Rate limit policies depend on whether you are using free models or Pay-As-You-Go models.
## Rate Limit Tiers
### 1. Free Models (Model IDs ending in `:free`)
Free models (e.g., `deepseek-r1:free`) are subject to the following rate limits:
* **5 Requests Per Minute (RPM)**
* **200 Requests Per Day (RPD)**
Once exceeded, requests to free models will return a `429` status code.
### 2. Pay-As-You-Go Models
For Pay-As-You-Go models (all model IDs without the `:free` suffix), Routeway does not enforce API-level rate limits.
To protect service availability and prevent abuse, standard rate limits and DDoS protection are active at the network edge.
***
## Rate Limit Response Schema
When you exceed a rate limit, the API returns an `HTTP 429 Too Many Requests` status code with the following JSON payload:
```json theme={null}
{
"error": {
"message": "API key per-minute rate limit exceeded (60 RPM).",
"type": "rate_limit_error",
"param": null,
"code": "rate_limit_exceeded",
"status_code": 429,
"id": "b3fca8a1",
"docs": "https://docs.routeway.ai",
"support": "https://discord.gg/RjX2CpdPpd"
}
}
```
| Field | Description |
| ------------- | ---------------------------------------------------- |
| `message` | Human-readable explanation of which limit was hit. |
| `type` | Always `rate_limit_error`. |
| `code` | Machine-readable error code (`rate_limit_exceeded`). |
| `status_code` | HTTP status code (`429`). |
| `id` | Unique trace/request identifier for debugging. |
| `docs` | Link to this documentation. |
| `support` | Link to our Discord support server. |
***
## Rate Limit Headers
Rate-limited responses include headers to help you manage retry timing.
### All Rate-Limited Responses
| Header | Description |
| ------------- | ----------------------------------------------------------- |
| `Retry-After` | Seconds the client must wait before making another request. |
### Minute-based Limits
| Header | Description |
| ------------------------------ | ------------------------------------------------ |
| `X-RateLimit-Limit-Minute` | Total requests allowed per minute window. |
| `X-RateLimit-Remaining-Minute` | Remaining requests in the current minute window. |
| `X-RateLimit-Reset-Minute` | Seconds until the minute window resets. |
### Day-based Limits
| Header | Description |
| --------------------------- | ----------------------------------------------------- |
| `X-RateLimit-Limit-Day` | Total requests allowed per day. |
| `X-RateLimit-Remaining-Day` | Remaining requests in the current day. |
| `X-RateLimit-Reset-Day` | Seconds until the daily window resets (UTC midnight). |
***
## Best Practices: Handling 429 Errors
We recommend implementing client-side retry logic using **exponential backoff with jitter** to handle rate limits gracefully.
```javascript theme={null}
async function fetchWithRetry(fn, retries = 5, delay = 1000) {
try {
return await fn();
} catch (error) {
if (error.status === 429 && retries > 0) {
// Use Retry-After header if available, otherwise exponential backoff
const retryAfter = error.headers?.get('Retry-After');
const waitMs = retryAfter
? parseInt(retryAfter, 10) * 1000
: delay * 2 + Math.random() * 200;
console.warn(`Rate limited. Retrying in ${Math.round(waitMs)}ms...`);
await new Promise(resolve => setTimeout(resolve, waitMs));
return fetchWithRetry(fn, retries - 1, delay * 2);
}
throw error;
}
}
```
```python theme={null}
import time
import random
def call_with_retry(api_func, retries=5, delay=1.0):
for attempt in range(retries):
try:
return api_func()
except Exception as e:
if hasattr(e, 'status_code') and e.status_code == 429 and attempt < retries - 1:
# Use Retry-After header if available, otherwise exponential backoff
retry_after = getattr(e, 'headers', {}).get('Retry-After')
if retry_after:
sleep_time = int(retry_after)
else:
jitter = random.uniform(0, 0.2)
sleep_time = (delay * (2 ** attempt)) + jitter
print(f"Rate limited. Retrying in {sleep_time:.2f}s...")
time.sleep(sleep_time)
else:
raise e
```
# Caching
Source: https://docs.routeway.ai/guides/chat-completions/caching
Reduce costs and latency with prompt caching and gateway-level response caching.
Routeway supports two independent caching systems. Understanding which one to reach for will save you time and money:
Provider-side KV cache. Repeated prompt prefixes are processed cheaper and faster — you still pay for inference, just at a reduced token rate.
Gateway-level full-response cache. Identical requests are returned instantly from Routeway with **no upstream call at all** — zero inference cost.
***
## Prompt Caching
Prompt caching lets you reuse the KV (key-value) cache from a previous request when the beginning of your prompt is identical. If the model has already processed a long system prompt or document, cached tokens cost significantly less and return faster.
Routeway automatically passes through cache control headers and prompt caching parameters to underlying providers that support them, including OpenAI and Anthropic.
***
## How Prompt Caching Works
When you send a request, the model processes each token sequentially and stores intermediate computations in a KV cache. On a subsequent request that begins with the same prefix, the model can skip recomputing those tokens and read from the cache instead.
```
Request 1: [System prompt: 2,000 tokens] + [User message: 50 tokens]
└── All tokens computed, cache populated
Request 2: [System prompt: 2,000 tokens] + [User message: 60 tokens]
└── System prompt served from cache (cheap!)
└── Only 60 new tokens computed
```
Cached tokens are typically **75–90% cheaper** than regular input tokens depending on the provider.
Cache hits skip prompt processing, reducing **time-to-first-token** on long-context requests.
***
## Automatic Caching (OpenAI Models)
OpenAI models (`gpt-4o`, `gpt-4o-mini`, `o3`, `o4-mini`, etc.) cache automatically. No changes to your requests are needed — Routeway forwards the same cache infrastructure OpenAI uses.
Cached tokens appear in the usage object:
```json theme={null}
{
"usage": {
"prompt_tokens": 2050,
"completion_tokens": 120,
"total_tokens": 2170,
"prompt_tokens_details": {
"cached_tokens": 2000,
"audio_tokens": 0
}
}
}
```
The cache is keyed on the exact byte sequence of your prompt prefix. Even a single character difference at the beginning of a prompt will result in a cache miss.
### Maximizing Cache Hits for OpenAI Models
```python theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
# Put stable, long content at the START of the messages array
SYSTEM_PROMPT = """
You are an expert legal document reviewer with 20 years of experience...
[2,000+ tokens of stable instructions and reference material]
"""
def review_clause(clause_text: str) -> str:
response = client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "system", "content": SYSTEM_PROMPT}, # cached after first call
{"role": "user", "content": f"Review this clause:\n\n{clause_text}"} # changes per request
],
)
usage = response.usage
cached = usage.prompt_tokens_details.cached_tokens if usage.prompt_tokens_details else 0
print(f"Cached tokens: {cached} / {usage.prompt_tokens}")
return response.choices[0].message.content
```
***
## Caching for Claude Models
Routeway **automatically enables prompt caching** for all Claude models when the request meets Anthropic's minimum cacheable threshold (\~2,048 tokens). You no longer need to manually add `cache_control` blocks — Routeway injects them on eligible content (system prompts, long user messages, and tool definitions) before forwarding the request to Anthropic.
We made this the default because most Claude requests that exceed the minimum token threshold benefit from caching, yet many developers were missing out on significant cost savings simply because they weren't aware of the `cache_control` parameter. By enabling it automatically, every eligible request gets up to 90% cheaper input token pricing with zero code changes on your side.
Automatic caching applies `cache_control` breakpoints to the largest stable content blocks in your request. If you already include explicit `cache_control` blocks, your configuration takes precedence — Routeway will not override or duplicate them.
You can still use explicit `cache_control` blocks for fine-grained control over exactly which content is cached:
```python theme={null}
import os
import anthropic
client = anthropic.Anthropic(
base_url="https://api.routeway.ai",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
# Load a large document once
with open("legal_document.txt") as f:
document = f.read()
response = client.messages.create(
model="claude-opus-4-5",
max_tokens=1024,
system=[
{
"type": "text",
"text": "You are a legal document analyst. Answer questions about the provided document.",
},
{
"type": "text",
"text": document,
"cache_control": {"type": "ephemeral"} # mark for caching
}
],
messages=[
{"role": "user", "content": "What are the key termination clauses?"}
]
)
# Check cache usage
print(response.usage)
# cache_creation_input_tokens: 8500 (first request — writing to cache)
# cache_read_input_tokens: 0
```
On subsequent requests with the same document and `cache_control`:
```python theme={null}
response2 = client.messages.create(
model="claude-opus-4-5",
max_tokens=1024,
system=[
{
"type": "text",
"text": "You are a legal document analyst. Answer questions about the provided document.",
},
{
"type": "text",
"text": document,
"cache_control": {"type": "ephemeral"} # same content = cache hit
}
],
messages=[
{"role": "user", "content": "What are the payment terms?"} # different question
]
)
print(response2.usage)
# cache_creation_input_tokens: 0
# cache_read_input_tokens: 8500 (cache hit — 90% cheaper!)
```
```python theme={null}
import os
from openai import OpenAI
# Use the OpenAI SDK with Anthropic models via Routeway
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
with open("legal_document.txt") as f:
document = f.read()
response = client.chat.completions.create(
model="claude-opus-4-5",
messages=[
{
"role": "system",
"content": [
{"type": "text", "text": "You are a legal document analyst."},
{
"type": "text",
"text": document,
"cache_control": {"type": "ephemeral"}
}
]
},
{"role": "user", "content": "What are the key termination clauses?"}
]
)
```
***
## What to Cache
Not everything is worth caching. Focus on content that is:
System prompts, reference documents, codebases, and knowledge bases that don't change between requests are ideal cache candidates.
**Minimum size**: Cache only kicks in above \~1,024 tokens (OpenAI) or \~2,048 tokens (Anthropic).
The cache matches on prefixes. Stable content must come **before** the dynamic parts (user messages, session data, etc.).
```
✓ [Large system prompt] → [User message]
✗ [User message] → [Large system prompt]
```
Caching is most valuable when many requests share the same prefix. Single-use prompts don't benefit from caching.
***
## Cache Lifetime
| Provider | Cache TTL | Notes |
| --------- | -------------- | ------------------------------- |
| OpenAI | \~5–10 minutes | Auto-managed, no manual control |
| Anthropic | \~5 minutes | Refreshed on each cache hit |
Caches expire during periods of low traffic. For long-running workflows, periodically "warm" the cache by making a request that exercises the cached prefix.
***
## Checking Cache Usage
Always log cache usage during development to verify your caching strategy is working:
```python theme={null}
def log_cache_stats(usage):
prompt_tokens = usage.prompt_tokens
cached_tokens = 0
if hasattr(usage, "prompt_tokens_details") and usage.prompt_tokens_details:
cached_tokens = usage.prompt_tokens_details.cached_tokens or 0
cache_ratio = cached_tokens / prompt_tokens if prompt_tokens > 0 else 0
print(f"Tokens: {prompt_tokens} total, {cached_tokens} cached ({cache_ratio:.0%})")
response = client.chat.completions.create(...)
log_cache_stats(response.usage)
```
***
## Response Caching
Response caching operates at the Routeway gateway level — before a request ever reaches an upstream provider. When a cache entry exists for an identical request, Routeway returns the stored response immediately. **No inference runs, no provider is charged.** Repeated identical calls cost \$0.
Response caching is separate from prompt caching. Prompt caching reduces the cost of *processing* a request; response caching eliminates the upstream call entirely.
Cache hits bypass the provider completely. Repeated identical requests are **free** — no tokens consumed, no provider charge.
Cached responses return in milliseconds, with no model latency at all.
Classification, batch jobs, FAQ lookups, and retries all benefit most — inputs are predictable and repeat frequently.
Caching is enabled and configured per API key, so different use cases can have different TTL policies.
***
### Enabling Response Caching
Response caching is configured per API key. When creating or editing a key in the [Dashboard](https://routeway.ai/dashboard/keys), scroll to the **Cache Responses** section and toggle it on.
Go to [Dashboard → API Keys](https://routeway.ai/dashboard/keys) and create a new key or click **Edit** on an existing one.
Scroll to the **Cache Responses** section and enable it.
Enter how long cached responses should be kept, in seconds.
| | |
| ------- | ------------------ |
| Minimum | `300` (5 minutes) |
| Maximum | `86400` (24 hours) |
| Default | `3600` (1 hour) |
***
### Cache Key
Cache entries are scoped to:
```
API key + model + exact request body
```
Any change to the request — messages, `temperature`, `max_tokens`, or any other field — produces a cache miss and triggers a fresh upstream call.
***
### Per-Request Headers
You can override the key-level cache settings on individual requests using request headers:
| Header | Values | Description |
| --------------- | ---------------- | ---------------------------------------------------------------------------------- |
| `X-Cache` | `true` / `false` | Enable or disable caching for this request, overriding the key default |
| `X-Cache-TTL` | `` | Override the TTL for this request (clamped to 300–86400) |
| `X-Cache-Clear` | `true` | Invalidate the cached entry for this exact request and force a fresh upstream call |
```python theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
# Cache this request for 10 minutes instead of the key default
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Classify this as spam or not: 'Congratulations, you won!'"}],
extra_headers={
"X-Cache": "true",
"X-Cache-TTL": "600",
},
)
print(response.choices[0].message.content)
```
```javascript theme={null}
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.routeway.ai/v1",
apiKey: process.env.ROUTEWAY_API_KEY,
});
// Cache this request for 10 minutes instead of the key default
const response = await client.chat.completions.create(
{
model: "gpt-4o-mini",
messages: [{ role: "user", content: "Classify this as spam or not: 'Congratulations, you won!'" }],
},
{
headers: {
"X-Cache": "true",
"X-Cache-TTL": "600",
},
}
);
console.log(response.choices[0].message.content);
```
```bash theme={null}
curl https://api.routeway.ai/v1/chat/completions \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-H "X-Cache: true" \
-H "X-Cache-TTL: 600" \
-d '{
"model": "gpt-4o-mini",
"messages": [
{"role": "user", "content": "Classify this as spam or not: 'Congratulations, you won!'"}
]
}'
```
#### Forcing a cache refresh
Send `X-Cache-Clear: true` to bust the cached entry and force a live upstream call. The fresh response is then stored under the same cache key.
```bash theme={null}
curl https://api.routeway.ai/v1/chat/completions \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-H "X-Cache-Clear: true" \
-d '{
"model": "gpt-4o-mini",
"messages": [{"role": "user", "content": "What is the capital of France?"}]
}'
```
***
### Response Headers
Every response includes headers that tell you what the cache did:
| Header | Values | Description |
| ---------------- | -------------- | ---------------------------------------------- |
| `X-Cache-Status` | `HIT` / `MISS` | Whether the response came from cache |
| `X-Cache-Age` | `` | How old the cached response is (only on `HIT`) |
| `X-Cache-TTL` | `` | Remaining time before the cache entry expires |
```python theme={null}
import httpx
import os
# Use httpx directly to inspect response headers
with httpx.Client() as http:
r = http.post(
"https://api.routeway.ai/v1/chat/completions",
headers={
"Authorization": f"Bearer {os.getenv('ROUTEWAY_API_KEY')}",
"Content-Type": "application/json",
"X-Cache": "true",
},
json={
"model": "gpt-4o-mini",
"messages": [{"role": "user", "content": "What is the capital of France?"}],
},
)
print(r.headers.get("X-Cache-Status")) # HIT or MISS
print(r.headers.get("X-Cache-Age")) # e.g. 42 (seconds since cached)
print(r.headers.get("X-Cache-TTL")) # e.g. 3558 (seconds remaining)
```
***
### When to Use Response Caching
Spam detection, sentiment analysis, intent routing, and content moderation often receive duplicate inputs. Cache the result the first time and serve it instantly on repeats.
If a batch pipeline re-processes records or retries failed items, identical inputs will hit the cache instead of re-running inference and being billed again.
A fixed set of questions (product FAQs, support macros, help-centre entries) will cache on the first request and return for free indefinitely within the TTL.
Avoid burning credits on repeated test calls. Enable caching during dev and set a long TTL so identical prompts are free after the first run.
Response caching is **not** suited for free-form conversational chat. Multi-turn conversations almost never produce identical request bodies, so cache hit rates will be near zero and the overhead adds no value.
# Multimodal Inputs
Source: https://docs.routeway.ai/guides/chat-completions/multimodal-inputs
Send images, PDFs, and audio alongside text in a single request.
Multimodal inputs let you send more than plain text to a model. Routeway supports images, PDFs/documents, and audio through the same OpenAI-compatible endpoint — the model can read, describe, and reason over all of them in a single request.
Vision and multimodal support depends on the model, not just the endpoint. Models like `gpt-4o`, `gpt-4o-mini`, `claude-opus-4-5`, and `gemini-2.5-pro` support image and document inputs. Check the [Models](/getting-started/models) page for per-model capability details.
***
## Images
Pass an image by URL or as a base64-encoded data URI inside a `content` array. The model can describe, analyze, compare, or extract information from images.
```python theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
response = client.chat.completions.create(
model="gpt-4o",
messages=[
{
"role": "user",
"content": [
{
"type": "text",
"text": "What's in this image? Describe it in detail."
},
{
"type": "image_url",
"image_url": {
"url": "https://upload.wikimedia.org/wikipedia/commons/thumb/4/47/PNG_transparency_demonstration_1.png/280px-PNG_transparency_demonstration_1.png"
}
}
]
}
],
)
print(response.choices[0].message.content)
```
```javascript theme={null}
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.routeway.ai/v1",
apiKey: process.env.ROUTEWAY_API_KEY,
});
async function main() {
const response = await client.chat.completions.create({
model: "gpt-4o",
messages: [
{
role: "user",
content: [
{
type: "text",
text: "What's in this image? Describe it in detail.",
},
{
type: "image_url",
image_url: {
url: "https://upload.wikimedia.org/wikipedia/commons/thumb/4/47/PNG_transparency_demonstration_1.png/280px-PNG_transparency_demonstration_1.png",
},
},
],
},
],
});
console.log(response.choices[0].message.content);
}
main().catch(console.error);
```
```bash theme={null}
curl https://api.routeway.ai/v1/chat/completions \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o",
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "What is in this image?"
},
{
"type": "image_url",
"image_url": {
"url": "https://example.com/image.png"
}
}
]
}
]
}'
```
### Base64 Images
For local files or private images that aren't publicly accessible via URL, encode them as base64 and embed them directly.
```python theme={null}
import os
import base64
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
with open("screenshot.png", "rb") as f:
image_data = base64.standard_b64encode(f.read()).decode("utf-8")
response = client.chat.completions.create(
model="gpt-4o",
messages=[
{
"role": "user",
"content": [
{"type": "text", "text": "Describe any UI issues you see in this screenshot."},
{
"type": "image_url",
"image_url": {
"url": f"data:image/png;base64,{image_data}"
}
}
]
}
],
)
print(response.choices[0].message.content)
```
### Image Detail Level
Control how much detail the model uses when processing an image. Higher detail increases token usage and cost.
```python theme={null}
{
"type": "image_url",
"image_url": {
"url": "https://example.com/chart.png",
"detail": "high" # "low", "high", or "auto" (default)
}
}
```
| Value | Behavior | Best for |
| -------- | --------------------------------- | ----------------------------- |
| `"auto"` | Model chooses based on image size | General use |
| `"low"` | Fast, cheap — 85 tokens fixed | Thumbnails, simple images |
| `"high"` | Tiles the image for fine detail | Charts, documents, dense text |
***
## Multiple Images
Pass several images in the same message to compare, diff, or analyze them together.
```python theme={null}
response = client.chat.completions.create(
model="gpt-4o",
messages=[
{
"role": "user",
"content": [
{"type": "text", "text": "What are the differences between these two designs?"},
{"type": "image_url", "image_url": {"url": "https://example.com/design-v1.png"}},
{"type": "image_url", "image_url": {"url": "https://example.com/design-v2.png"}},
]
}
],
)
```
***
## PDFs and Documents
Send PDF files as base64-encoded data for the model to read and reason over. Useful for contract analysis, document Q\&A, and data extraction.
```python theme={null}
import os
import base64
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
with open("contract.pdf", "rb") as f:
pdf_data = base64.standard_b64encode(f.read()).decode("utf-8")
response = client.chat.completions.create(
model="gpt-4o",
messages=[
{
"role": "user",
"content": [
{
"type": "text",
"text": "Summarize the key obligations and termination clauses in this contract."
},
{
"type": "image_url",
"image_url": {
"url": f"data:application/pdf;base64,{pdf_data}"
}
}
]
}
],
)
print(response.choices[0].message.content)
```
```javascript theme={null}
import OpenAI from "openai";
import fs from "fs";
const client = new OpenAI({
baseURL: "https://api.routeway.ai/v1",
apiKey: process.env.ROUTEWAY_API_KEY,
});
async function main() {
const pdfBuffer = fs.readFileSync("contract.pdf");
const pdfBase64 = pdfBuffer.toString("base64");
const response = await client.chat.completions.create({
model: "gpt-4o",
messages: [
{
role: "user",
content: [
{
type: "text",
text: "Summarize the key obligations and termination clauses in this contract.",
},
{
type: "image_url",
image_url: {
url: `data:application/pdf;base64,${pdfBase64}`,
},
},
],
},
],
});
console.log(response.choices[0].message.content);
}
main().catch(console.error);
```
PDF support depends on the model. `gpt-4o` and `claude-opus-4-5` handle multi-page PDFs well. For very large documents, consider extracting the relevant pages first to reduce token cost.
***
## Multi-turn Vision Conversations
Images persist in the conversation history just like text messages. The model can refer back to a previously sent image in later turns.
```python theme={null}
messages = [
{
"role": "user",
"content": [
{"type": "text", "text": "Here is a chart of our Q3 sales data."},
{"type": "image_url", "image_url": {"url": "https://example.com/q3-chart.png"}}
]
},
{
"role": "assistant",
"content": "The chart shows a strong uptick in September with a peak of $2.4M..."
},
{
"role": "user",
"content": "Which month had the lowest performance and what might explain it?"
}
]
response = client.chat.completions.create(
model="gpt-4o",
messages=messages,
)
```
***
## Supported Formats
| Type | Formats | Notes |
| ---------------------- | -------------------- | ---------------------------------- |
| Images | PNG, JPEG, WebP, GIF | GIF uses only the first frame |
| Documents | PDF | Via base64 data URI |
| Max image size | 20 MB per image | Resize large images before sending |
| Max images per request | 10–20 | Model-dependent |
***
## Best Practices
Large images increase token usage significantly with little quality gain. Resize to 1024×1024 or smaller before encoding. The model doesn't need 4K resolution to understand content.
When you only need a rough description or classification, use `"detail": "low"`. It uses a fixed 85 tokens regardless of image size and returns much faster.
Tell the model exactly what to look for. Vague prompts like "describe this" produce generic results. "List every line item and its price from this receipt" produces structured, useful output.
Pair vision with a JSON schema to extract structured data from images. Ideal for receipts, invoices, forms, and screenshots.
```python theme={null}
from pydantic import BaseModel
class Invoice(BaseModel):
vendor: str
total: float
line_items: list[str]
due_date: str | None = None
response = client.beta.chat.completions.parse(
model="gpt-4o",
messages=[
{
"role": "user",
"content": [
{"type": "text", "text": "Extract the invoice details."},
{"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}
]
}
],
response_format=Invoice,
)
invoice = response.choices[0].message.parsed
```
# Chat Completions
Source: https://docs.routeway.ai/guides/chat-completions/overview
Generate responses using the OpenAI-compatible chat completions endpoint.
The Chat Completions API is the primary way to interact with language models through Routeway. Send a list of messages and receive a model-generated response. The endpoint is fully compatible with OpenAI's `/v1/chat/completions` format, so any existing client or SDK works without changes.
Routeway supports the same request and response schema as the OpenAI Chat Completions API. Point your base URL to `https://api.routeway.ai/v1` and you're set.
**New:** Append `@flux-2-turbo` or `@online` to any model ID to generate images or search the web directly in chat — no client-side tool setup required. See [Server Tools](/guides/chat-completions/server-tools).
***
## Quick Example
```python theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is the capital of France?"}
]
)
print(response.choices[0].message.content)
```
```javascript theme={null}
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.routeway.ai/v1",
apiKey: process.env.ROUTEWAY_API_KEY,
});
async function main() {
const response = await client.chat.completions.create({
model: "gpt-4o-mini",
messages: [
{ role: "system", content: "You are a helpful assistant." },
{ role: "user", content: "What is the capital of France?" },
],
});
console.log(response.choices[0].message.content);
}
main().catch(console.error);
```
```bash theme={null}
curl https://api.routeway.ai/v1/chat/completions \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o-mini",
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is the capital of France?"}
]
}'
```
***
## Response Shape
A successful response looks like this:
```json theme={null}
{
"id": "chatcmpl-abc123",
"object": "chat.completion",
"created": 1718500000,
"model": "gpt-4o-mini",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "The capital of France is Paris."
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 24,
"completion_tokens": 8,
"total_tokens": 32
}
}
```
| Field | Description |
| ---------------------------- | --------------------------------------------------------- |
| `choices[0].message.content` | The model's text response. |
| `choices[0].finish_reason` | Why the model stopped: `stop`, `length`, or `tool_calls`. |
| `usage` | Token counts for billing and monitoring. |
***
## Request Model
These are the most common fields you can include in a Chat Completions request:
| Field | Type | Description |
| ------------------ | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `model` | `string` | The model to use (e.g. `gpt-4o-mini`, `claude-sonnet-4-20250514`). |
| `messages` | `array` | The conversation history. Each message has a `role` and `content`. |
| `tools` | `array` | Function definitions the model can call. See [Tool Calling](/guides/chat-completions/tool-calling). |
| `response_format` | `object` | Constrain output to JSON or a JSON Schema. See [Structured Outputs](/guides/chat-completions/structured-outputs). |
| `stream` | `boolean` | Return tokens incrementally via SSE. See [Streaming](/guides/chat-completions/streaming). |
| `stream_options` | `object` | Options for streaming (e.g. `include_usage`). |
| `reasoning_effort` | `string` | Control thinking depth for reasoning models. See [Reasoning](/guides/chat-completions/reasoning). |
| `service_tier` | `string` | Choose processing tier (`default`, `flex`, or `priority`) on supported OpenAI and Gemini models. See [Service Tiers](/guides/chat-completions/service-tiers). |
***
## Learn More
Receive tokens as they are generated for real-time output.
Let the model invoke functions and external tools.
Use image generation and online search via model tag syntax.
Constrain responses to JSON or a specific schema.
Use extended thinking for complex multi-step problems.
Send images and other media alongside text messages.
Reduce costs and latency with prompt caching.
Trade cost for speed with flex and priority processing.
Apply reusable configurations to your requests.
# Presets
Source: https://docs.routeway.ai/guides/chat-completions/presets
Save a model, system prompt, and parameters as a reusable preset — then call it like a model ID.
Presets let you bundle a model, system prompt, and generation parameters into a single reusable configuration. Once created, you reference a preset exactly like a model ID — no extra code, no repeated config.
Presets can be **private** (only your API key can use them) or **public** (shareable with anyone).
***
## Creating a Preset
Go to [Dashboard → Presets → Create Preset](https://routeway.ai/dashboard/presets) and fill in the fields:
Give it a short, descriptive name. This is for your reference only.
Select any supported model, or choose **Any Model** to leave the model open at request time. With Any Model, callers must specify the model using the `@` syntax — see [Overriding the model](#overriding-the-model) below.
Enter the system prompt that should apply to every request using this preset. This is the main reason to use presets — ship a locked, versioned persona or instruction set without exposing the prompt in client code.
Configure defaults like `temperature`, `max_tokens`, `top_p`, and other generation parameters. These apply automatically to every request unless overridden.
Choose **Private** (default) or **Public**. Public presets can be used by anyone with the preset ID.
***
## Using a Preset
After creating a preset, copy its ID from the dashboard. It looks like:
```
presets/0ede00c3-b202-4102-bda7-ec9a4b398767
```
Paste it into the `model` field of any Chat Completions request — everything else stays the same.
```python theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
response = client.chat.completions.create(
model="presets/0ede00c3-b202-4102-bda7-ec9a4b398767",
messages=[
{"role": "user", "content": "hi"}
],
)
print(response.choices[0].message.content)
```
```javascript theme={null}
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.routeway.ai/v1",
apiKey: process.env.ROUTEWAY_API_KEY,
});
async function main() {
const response = await client.chat.completions.create({
model: "presets/0ede00c3-b202-4102-bda7-ec9a4b398767",
messages: [
{ role: "user", content: "hi" }
],
});
console.log(response.choices[0].message.content);
}
main().catch(console.error);
```
```bash theme={null}
curl https://api.routeway.ai/v1/chat/completions \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "presets/0ede00c3-b202-4102-bda7-ec9a4b398767",
"messages": [
{"role": "user", "content": "hi"}
]
}'
```
The preset's system prompt and parameters are applied server-side — your request payload only needs `model` and `messages`.
***
## Overriding the Model
If a preset was created with **Any Model**, the model must be specified at request time using `@` after the preset ID:
```
presets/0ede00c3-b202-4102-bda7-ec9a4b398767@gpt-4o
```
```python theme={null}
response = client.chat.completions.create(
model="presets/0ede00c3-b202-4102-bda7-ec9a4b398767@gpt-4o",
messages=[
{"role": "user", "content": "hi"}
],
)
```
This is useful when you want a shared system prompt and parameter set but need the caller to decide which underlying model to use — for example, letting users pick between `gpt-4o` and `gpt-4o-mini` while keeping the same persona.
***
## Streaming with Presets
Presets work with all standard Chat Completions parameters, including `stream`:
```python theme={null}
stream = client.chat.completions.create(
model="presets/0ede00c3-b202-4102-bda7-ec9a4b398767",
messages=[
{"role": "user", "content": "Tell me a short story."}
],
stream=True,
)
for chunk in stream:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)
```
***
## Public vs Private Presets
| | Private | Public |
| --------------------- | ------------------------------- | ----------------------------------------- |
| Who can use it | Only your API key | Anyone with the preset ID |
| Visible in dashboard | Yes | Yes |
| System prompt exposed | No | Yes |
| Use case | Internal apps, personal tooling | Shared assistants, published integrations |
Public presets expose the system prompt to anyone who has the preset ID. Don't include secrets, internal instructions, or sensitive information in the system prompt of a public preset.
***
## When to Use Presets
Define a customer support agent, coding assistant, or brand voice once and reference it by ID across all your apps.
Create a new preset when you iterate on a system prompt instead of updating in-place, so you can roll back.
Public presets let anyone on your team (or your users) call a consistent configuration without needing to manage the prompt themselves.
Offload model selection and prompt engineering to the dashboard. Client code only needs the preset ID.
# Reasoning
Source: https://docs.routeway.ai/guides/chat-completions/reasoning
Use extended thinking models for hard problems that require deeper deliberation.
Reasoning models spend additional compute "thinking through" a problem before producing a response. This extended deliberation makes them significantly better at complex tasks like multi-step math, code debugging, strategic planning, and hard classification problems where a direct answer is often wrong.
Reasoning models supported through Routeway include **GPT 5.4**, **Claude Opus 4.8**, **DeepSeek V4** and much more. Check the [Models](/getting-started/models) page for the current list.
***
## Enabling Reasoning
By default, Routeway **disables reasoning** for models. There are two ways to enable it:
### Option 1: Model Suffix
Append `:thinking` to the end of the model ID to enable reasoning with the default effort level.
```python theme={null}
response = client.chat.completions.create(
model="claude-opus-4-5:thinking",
messages=[{"role": "user", "content": "Solve this step by step..."}]
)
```
You can also specify the effort level directly in the suffix using `:high`, `:medium`, or `:low`:
```python theme={null}
# High effort reasoning
response = client.chat.completions.create(
model="o4-mini:thinking:high",
messages=[{"role": "user", "content": "Prove that there are infinitely many primes."}]
)
# Low effort reasoning
response = client.chat.completions.create(
model="o4-mini:thinking:low",
messages=[{"role": "user", "content": "What is 2 + 2?"}]
)
```
### Option 2: `reasoning_effort` Parameter
Pass the `reasoning_effort` parameter in the request body to enable reasoning and control the effort level:
```python theme={null}
response = client.chat.completions.create(
model="o4-mini",
messages=[{"role": "user", "content": "Explain the Riemann hypothesis."}],
reasoning_effort="high"
)
```
Both approaches achieve the same result. The model suffix is convenient for quick testing or when there is no way to change reasoning level via ui, while `reasoning_effort` gives you programmatic control without changing the model ID.
***
## When to Use Reasoning Models
* Multi-step math or logic puzzles
* Debugging complex code
* Long-horizon planning and strategy
* Hard classification with many edge cases
* Tasks where chain-of-thought improves accuracy
* Simple Q\&A or retrieval
* Short creative tasks
* High-throughput, latency-sensitive pipelines
* Tasks where standard models already perform well
***
## Controlling Reasoning Effort
You can control reasoning effort using the `reasoning_effort` parameter.
| Value | Behavior | Best for |
| ---------- | -------------------------- | ------------------------------------------- |
| `"low"` | Minimal reasoning, fastest | Simple tasks, cost-sensitive |
| `"medium"` | Balanced (default) | Most production use cases |
| `"high"` | Deep reasoning, slowest | Hard math, complex code, critical decisions |
| `"none"` | No reasoning, fastest | Simple Q\&A or retrieval |
```python theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
response = client.chat.completions.create(
model="o4-mini",
messages=[
{
"role": "user",
"content": "A farmer has 17 sheep. All but 9 run away. How many sheep does the farmer have left? Show your reasoning."
}
],
reasoning_effort="medium",
)
print(response.choices[0].message.content)
```
```javascript theme={null}
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.routeway.ai/v1",
apiKey: process.env.ROUTEWAY_API_KEY,
});
async function main() {
const response = await client.chat.completions.create({
model: "o4-mini",
messages: [
{
role: "user",
content:
"A farmer has 17 sheep. All but 9 run away. How many sheep does the farmer have left? Show your reasoning.",
},
],
reasoning_effort: "medium",
});
console.log(response.choices[0].message.content);
}
main().catch(console.error);
```
```bash theme={null}
curl https://api.routeway.ai/v1/chat/completions \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "o4-mini",
"messages": [
{
"role": "user",
"content": "A farmer has 17 sheep. All but 9 run away. How many does he have left?"
}
],
"reasoning_effort": "medium"
}'
```
# Server Tools
Source: https://docs.routeway.ai/guides/chat-completions/server-tools
Extend model capabilities directly in chat using @-tag syntax for Image Generation and Online Search.
Routeway Server Tools allow language models to execute server-side actions — such as generating images or searching the internet in real time — without requiring any client-side tool definitions or complex API schemas.
Server tools work directly in chat requests with **zero extra setup**: simply append the tool tag to your model ID using `@` syntax and start chatting.
Unlike custom function calling (where your application code receives tool calls and executes them locally), Server Tools are handled entirely on Routeway's infrastructure. Results are seamlessly integrated into the model's response.
***
## Available Server Tools
| Tool | Tag Syntax | Example Model String | Description |
| :------------------- | :--------------- | :--------------------------------- | :----------------------------------------------------- |
| **Image Generation** | `@` | `qwen3.8-max-preview@flux-2-turbo` | Generates images in chat via an image generation model |
| **Online Search** | `@online` | `qwen3.8-max-preview@online` | Searches the web for up-to-date knowledge |
***
## Image Generation
The Image Generation tool lets language models create visual content right from your chat prompt.
To enable Image Generation, combine a base text model with an image model using `@`:
```
@
```
For example, using `qwen3.8-max-preview@flux-2-turbo` routes your prompt to `qwen3.8-max-preview`, which can trigger `flux-2-turbo` to render the requested image.
### How It Works
1. Send a request with an `@` tag in the model field.
2. Ask the model to generate an image (e.g., *"Draw a majestic lion sitting on a cliff at sunset"*).
3. The server tool generates the image in a few seconds and returns it in the response as an inline HTML `![]()
` element.
### Response Format
The server tool delivers generated images as an inline HTML `![]()
` tag directly inside the message content:
```html theme={null}
```
#### Example Payloads
```json theme={null}
{
"id": "chatcmpl-1784633736182-7a42142963b942aa",
"object": "chat.completion",
"created": 1784633752,
"model": "qwen3.8-max-preview@flux-2-turbo",
"system_fingerprint": null,
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Sure! Let me generate a cat for you.\n\n![\"A](\"https://api.routeway.ai/images/f3f2954a-1c05-432f-813a-bbfed41d1c67\")
"
},
"logprobs": null,
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 357,
"completion_tokens": 115,
"total_tokens": 472
}
}
```
```json theme={null}
data: {"id":"chatcmpl-1784615844951-9a94287c163e489b","object":"chat.completion.chunk","created":1784615863,"model":"qwen3.8-max-preview@flux-2-turbo","choices":[{"index":0,"delta":{"content":"\n\n![\"A](\"https://api.routeway.ai/images/923fd842-b3d6-4b89-9ab8-079b7959eed8\")
"},"finish_reason":null,"logprobs":null}],"system_fingerprint":null,"usage":null}
```
### Client Rendering & Compatibility
Rendering behavior depends on your chat client or frontend interface:
Most chat clients (like **SillyTavern**, LibreChat, or custom web UI frontends) automatically render HTML `![]()
` tags inline within the chat stream.
Some chat clients (like **JanitorAI**) do not automatically render HTML image tags inline. In these clients, copy the image URL from the `src` attribute (e.g. `https://api.routeway.ai/images/...`) and open it in your browser to view the image.
#### Example: SillyTavern
In [SillyTavern](/integrations/roleplay/sillytavern), generated images appear inline in the chat when you use a model tagged with an image server tool (for example `qwen3.8-max-preview@flux-2-turbo`):
![SillyTavern chat showing an image generated inline via Routeway Server Tools]()
To display these images in SillyTavern, open **User Settings** (gear icon) on the top menu bar and **disable** **Forbid External Media**.
***
## Online Search
The Online Search tool connects language models to live internet data.
To enable Online Search, append `@online` to your model ID:
```
@online
```
For example, `qwen3.8-max-preview@online` or `gpt-4o@online`.
### How It Works
1. Send a request with `@online` appended to the model name.
2. Ask a question about current events, recent releases, live stock prices, or up-to-date information.
3. When the model needs additional real-time context, it automatically searches the web, processes the results, and incorporates fresh data into its answer.
Use `@online` when asking models about events after their knowledge cutoff date or for real-time information such as live sports scores, current news, or market data.
### Pricing
| Allowance | Condition |
| :--------------------------- | :----------------------------------------------------------------------- |
| **Up to 5 searches / day** | Default daily free allowance |
| **Up to 100 searches / day** | Account balance is greater than `$0`, or you have an active subscription |
| **\~\$0.007 per search** | After you exceed 100 searches in a day |
Want to use your own search provider key instead? See [Web Search BYOK](/guides/chat-completions/web-search-byok). Successful BYOK searches are not billed by Routeway for search usage.
***
## Code Examples
You can use Server Tools with standard OpenAI SDKs or HTTP requests by passing the tagged model name in the `model` parameter.
```python theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
# Example 1: Image Generation in Chat
response = client.chat.completions.create(
model="qwen3.8-max-preview@flux-2-turbo",
messages=[
{"role": "user", "content": "Generate an image of a futuristic neon city at night."}
]
)
print(response.choices[0].message.content)
# Example 2: Real-time Online Search
search_response = client.chat.completions.create(
model="qwen3.8-max-preview@online",
messages=[
{"role": "user", "content": "What were the major technology news stories today?"}
]
)
print(search_response.choices[0].message.content)
```
```javascript theme={null}
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.routeway.ai/v1",
apiKey: process.env.ROUTEWAY_API_KEY,
});
// Example 1: Image Generation
const imageRes = await client.chat.completions.create({
model: "qwen3.8-max-preview@flux-2-turbo",
messages: [
{ role: "user", content: "Generate an image of a cute kitten playing with yarn." }
],
});
console.log(imageRes.choices[0].message.content);
// Example 2: Online Search
const searchRes = await client.chat.completions.create({
model: "qwen3.8-max-preview@online",
messages: [
{ role: "user", content: "What is the latest score in the championship game?" }
],
});
console.log(searchRes.choices[0].message.content);
```
```bash theme={null}
# Image Generation
curl https://api.routeway.ai/v1/chat/completions \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen3.8-max-preview@flux-2-turbo",
"messages": [
{"role": "user", "content": "Generate an image of a cozy cabin in the woods."}
]
}'
# Online Search
curl https://api.routeway.ai/v1/chat/completions \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen3.8-max-preview@online",
"messages": [
{"role": "user", "content": "What is the latest status of space launch missions this week?"}
]
}'
```
***
## Best Practices
Server tools require no code updates or schema changes. Simply set the model ID string with the desired `@` tag.
Combine text models with any available image generation model supported by Routeway (such as `flux-2-turbo`).
If your chat UI doesn't display images directly inline, extract the image URL from the returned `src` attribute.
Add `@online` whenever your application requires access to live web data, current events, or real-time search results.
# Service Tiers
Source: https://docs.routeway.ai/guides/chat-completions/service-tiers
Choose between default, flex, and priority processing to balance cost and latency.
Some models support **service tiers** — processing modes that let you pay less or get faster responses depending on your needs. Flex is a great default when you want to cut costs without sacrificing output quality.
Service tiers are a **provider feature** from OpenAI and Gemini — not something Routeway implements on its own. When you set a tier, Routeway forwards it to the upstream provider and applies the matching price multiplier on your bill. Routeway does **not** run a separate queue or priority system; any latency or capacity differences come from how the provider handles that tier.
Service tiers are currently available on **select OpenAI and Gemini models only** — not every model from those providers, and not models from other providers. Check the [Models Catalog](https://routeway.ai/models) or the `capabilities.service_tiers` field in the [GET /v1/models](/getting-started/models) response to confirm a specific model supports them.
***
## Available Tiers
| Tier | Cost | Latency | Best for |
| :----------- | :--------------------------- | :--------------------------------------------------------------------------- | :------------------------------------------------------------- |
| **Default** | Standard model pricing | Balanced time-to-first-token (TTFT) and throughput | General production workloads |
| **Flex** | **50% off** standard pricing | Same generation speed once streaming starts; TTFT may occasionally be higher | Cost-sensitive workloads, high-volume usage, evals, batch jobs |
| **Priority** | **2×** standard pricing | Lower, more consistent TTFT on the provider side — even during peak demand | User-facing chat, real-time assistants, latency-sensitive apps |
Flex does not reduce output quality or token generation speed. The main difference is that time-to-first-token can be a bit higher on some requests. If the upstream provider's flex capacity is temporarily limited, the request may return `429` from the provider — you are not charged in that case.
***
## Supported Models
Service tiers are currently supported on a **subset of OpenAI and Gemini models**. Availability varies by model — Routeway does not enable tiers on every model from those providers, and models from Anthropic, DeepSeek, and other providers do not support them yet.
### Models Catalog
Open any model page on the [Models Catalog](https://routeway.ai/models) and scroll to the bottom. If the model supports service tiers, you'll see a **Service Tiers** section and tier-specific rates under **Token Pricing**.
For example, [GPT 5.6 Luna](https://routeway.ai/models/gpt-5.6-luna) shows all three tiers with their multipliers:
![GPT 5.6 Luna model page showing Service Tiers and Token Pricing for default, flex, and priority]()
If a model does not support tiers, those sections are omitted from its page.
### API
Query the models endpoint and look for `service_tier` in `supported_parameters` and the tier list in `capabilities.service_tiers`:
```bash theme={null}
curl https://api.routeway.ai/v1/models \
-H "Authorization: Bearer $ROUTEWAY_API_KEY"
```
```json theme={null}
{
"id": "gpt-5.6-luna",
"owned_by": "openai",
"supported_parameters": ["temperature", "tools", "service_tier", "..."],
"capabilities": {
"service_tiers": ["default", "flex", "priority"]
},
"service_tiers": {
"flex": {
"price_multiplier": 0.5,
"note": "50% discount"
},
"priority": {
"price_multiplier": 2.0,
"note": "2x price"
}
}
}
```
Filter client-side to list all eligible models:
```python theme={null}
data = client.models.list()
tiered_models = [
m for m in data.data
if m.capabilities and getattr(m.capabilities, "service_tiers", None)
]
for m in tiered_models:
print(m.id, "—", m.owned_by, "—", m.capabilities.service_tiers)
```
Service tiers apply to the listed per-token rates in the model's `pricing` object. See [How Billing Works](/getting-started/billing) for how multipliers affect your bill.
***
## Setting a Service Tier
There are two ways to select a tier. Both produce the same result — use whichever fits your setup.
### Option 1: `service_tier` Parameter
Pass `service_tier` in the request body. This is the recommended approach when you control the API payload.
```python theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
# Flex — 50% cheaper
response = client.chat.completions.create(
model="gpt-5.6-luna",
messages=[{"role": "user", "content": "Summarize this dataset..."}],
service_tier="flex",
)
# Priority — 2× cost, lower TTFT
response = client.chat.completions.create(
model="gpt-5.6-luna",
messages=[{"role": "user", "content": "Help me debug this error."}],
service_tier="priority",
)
```
```javascript theme={null}
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.routeway.ai/v1",
apiKey: process.env.ROUTEWAY_API_KEY,
});
// Flex — 50% cheaper
const flexResponse = await client.chat.completions.create({
model: "gpt-5.6-luna",
messages: [{ role: "user", content: "Summarize this dataset..." }],
service_tier: "flex",
});
// Priority — 2× cost, lower TTFT
const priorityResponse = await client.chat.completions.create({
model: "gpt-5.6-luna",
messages: [{ role: "user", content: "Help me debug this error." }],
service_tier: "priority",
});
```
```bash theme={null}
# Flex
curl https://api.routeway.ai/v1/chat/completions \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-5.6-luna",
"messages": [{"role": "user", "content": "Summarize this dataset..."}],
"service_tier": "flex"
}'
# Priority
curl https://api.routeway.ai/v1/chat/completions \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-5.6-luna",
"messages": [{"role": "user", "content": "Help me debug this error."}],
"service_tier": "priority"
}'
```
Accepted values: `"default"`, `"flex"`, and `"priority"`. Omit the parameter (or set `"default"`) to use standard pricing and performance.
### Option 2: Model Suffix
Append `:flex` or `:priority` to the model ID. This is useful in chat UIs and other tools where you cannot pass extra request parameters.
```python theme={null}
# Flex via model suffix
response = client.chat.completions.create(
model="gpt-5.6-luna:flex",
messages=[{"role": "user", "content": "Run this evaluation batch."}],
)
# Priority via model suffix
response = client.chat.completions.create(
model="gpt-5.6-luna:priority",
messages=[{"role": "user", "content": "What is the weather in Tokyo right now?"}],
)
```
Both approaches achieve the same result. Use `service_tier` when you have programmatic control over the request body, and the `:flex` / `:priority` suffix when you can only change the model name — for example in Open WebUI, SillyTavern, or other chat frontends.
***
## Response Field
The response includes a `service_tier` field indicating which tier actually processed the request:
```json theme={null}
{
"id": "chatcmpl-abc123",
"model": "gpt-5.6-luna",
"service_tier": "flex",
"choices": [
{
"message": {
"role": "assistant",
"content": "Here is the summary..."
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 120,
"completion_tokens": 85,
"total_tokens": 205
}
}
```
***
## When to Use Each Tier
Your everyday choice. Standard pricing with balanced performance for most production traffic.
Save 50% with the same output quality. Ideal when you want lower costs and can tolerate the occasional slower start — TTFT may be higher, but generation speed is unchanged.
Pay a premium for lower, more predictable TTFT when users are waiting on the first token in real time.
# Streaming
Source: https://docs.routeway.ai/guides/chat-completions/streaming
Receive tokens as they are generated instead of waiting for the full response.
Streaming lets your application display or process tokens as the model generates them, dramatically reducing perceived latency for users. Instead of waiting for the entire response to finish, you receive a continuous stream of small chunks over a persistent HTTP connection.
Routeway supports Server-Sent Events (SSE) streaming using the same format as the OpenAI API. Any client that works with OpenAI streaming will work with Routeway out of the box.
## How It Works
Set `"stream": true` in your request. The response will be delivered as a series of `data:` lines, each containing a JSON chunk. The stream ends with `data: [DONE]`.
```
data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"delta":{"content":"Hello"},"finish_reason":null}]}
data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"delta":{"content":" world"},"finish_reason":null}]}
data: [DONE]
```
Each chunk carries a `delta` object instead of a full `message`. Text arrives through `delta.content`. The final chunk sets `finish_reason` to indicate why the model stopped.
***
## Basic Example
```python theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
stream = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Explain quantum entanglement simply."}],
stream=True,
)
for chunk in stream:
delta = chunk.choices[0].delta
if delta.content:
print(delta.content, end="", flush=True)
```
```javascript theme={null}
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.routeway.ai/v1",
apiKey: process.env.ROUTEWAY_API_KEY,
});
async function main() {
const stream = await client.chat.completions.create({
model: "gpt-4o-mini",
messages: [{ role: "user", content: "Explain quantum entanglement simply." }],
stream: true,
});
for await (const chunk of stream) {
const content = chunk.choices[0]?.delta?.content ?? "";
process.stdout.write(content);
}
}
main().catch(console.error);
```
```bash theme={null}
curl -N https://api.routeway.ai/v1/chat/completions \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o-mini",
"messages": [
{"role": "user", "content": "Explain quantum entanglement simply."}
],
"stream": true
}'
```
The `-N` flag disables buffering so chunks print immediately.
***
## Streaming with Tool Calls
When a model decides to call a tool, arguments arrive incrementally through `delta.tool_calls`. Accumulate the chunks before executing the function.
```python theme={null}
import json
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather for a city.",
"parameters": {
"type": "object",
"properties": {
"city": {"type": "string"}
},
"required": ["city"],
},
},
}
]
stream = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "What's the weather in Tokyo?"}],
tools=tools,
stream=True,
)
tool_call_chunks = {}
for chunk in stream:
delta = chunk.choices[0].delta
# Accumulate text content
if delta.content:
print(delta.content, end="", flush=True)
# Accumulate tool call arguments
if delta.tool_calls:
for tc in delta.tool_calls:
idx = tc.index
if idx not in tool_call_chunks:
tool_call_chunks[idx] = {"id": tc.id, "name": tc.function.name, "args": ""}
if tc.function.arguments:
tool_call_chunks[idx]["args"] += tc.function.arguments
# After stream ends, execute the tool
for tc in tool_call_chunks.values():
args = json.loads(tc["args"])
print(f"\nCalling {tc['name']} with {args}")
```
***
## Include Usage Stats
To receive token usage at the end of a stream, pass `stream_options`:
```json theme={null}
{
"model": "gpt-4o-mini",
"messages": [...],
"stream": true,
"stream_options": {
"include_usage": true
}
}
```
A final chunk will include a `usage` object after `[DONE]`:
```json theme={null}
{
"usage": {
"prompt_tokens": 42,
"completion_tokens": 128,
"total_tokens": 170
}
}
```
***
## Handling Errors in Streams
Errors can appear mid-stream as a regular SSE data frame containing an `error` key. Always check for this before assuming a chunk contains `choices`.
```python theme={null}
for chunk in stream:
# Check for an error frame
if hasattr(chunk, "error"):
print(f"Stream error: {chunk.error}")
break
delta = chunk.choices[0].delta
if delta.content:
print(delta.content, end="", flush=True)
```
Do not treat the stream as plain text. Parse each chunk as JSON and handle `finish_reason` and error frames explicitly.
***
## Best Practices
Use `flush=True` (Python) or `process.stdout.write` (Node.js) so tokens render in real time.
Network interruptions can cut streams short. Track the last received content and retry if needed.
Tool call arguments arrive in fragments. Always concatenate before calling `JSON.parse`.
`length` means the model hit `max_tokens`. `stop` means a natural end. `tool_calls` means a function was requested.
# Structured Outputs
Source: https://docs.routeway.ai/guides/chat-completions/structured-outputs
Force models to return valid JSON matching an exact schema.
Structured Outputs let you enforce that a model's response conforms to a JSON schema you define. Instead of asking the model to *try* to return JSON, the API guarantees the output matches your schema — no extra parsing, no malformed responses.
Structured Outputs are available on models that support `response_format` with `json_schema` (e.g. `gpt-4o`, `gpt-4o-mini`). For other models, use `json_object` mode with a system prompt.
***
## JSON Mode vs. Structured Outputs
| Feature | `json_object` mode | `json_schema` mode |
| ------------------------- | --------------------- | -------------------- |
| Guarantees valid JSON | ✓ | ✓ |
| Enforces specific keys | ✗ | ✓ |
| Supports required fields | ✗ | ✓ |
| Validates enums and types | ✗ | ✓ |
| Best for | Simple JSON responses | Production pipelines |
Use `json_schema` when downstream code depends on specific fields always being present.
***
## Basic Example
```python theme={null}
import os
import json
from openai import OpenAI
from pydantic import BaseModel
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
# Define your schema with Pydantic
class CalendarEvent(BaseModel):
name: str
date: str
participants: list[str]
location: str | None = None
response = client.beta.chat.completions.parse(
model="gpt-4o",
messages=[
{
"role": "user",
"content": "Alice and Bob are attending a machine learning workshop in Berlin on July 15th."
}
],
response_format=CalendarEvent,
)
event = response.choices[0].message.parsed
print(event.name) # "Machine Learning Workshop"
print(event.participants) # ["Alice", "Bob"]
print(event.date) # "July 15th"
print(event.location) # "Berlin"
```
```javascript theme={null}
import OpenAI from "openai";
import { zodResponseFormat } from "openai/helpers/zod";
import { z } from "zod";
const client = new OpenAI({
baseURL: "https://api.routeway.ai/v1",
apiKey: process.env.ROUTEWAY_API_KEY,
});
const CalendarEvent = z.object({
name: z.string(),
date: z.string(),
participants: z.array(z.string()),
location: z.string().optional(),
});
async function main() {
const response = await client.beta.chat.completions.parse({
model: "gpt-4o",
messages: [
{
role: "user",
content: "Alice and Bob are attending a machine learning workshop in Berlin on July 15th.",
},
],
response_format: zodResponseFormat(CalendarEvent, "calendar_event"),
});
const event = response.choices[0].message.parsed;
console.log(event.name); // "Machine Learning Workshop"
console.log(event.participants); // ["Alice", "Bob"]
}
main().catch(console.error);
```
```bash theme={null}
curl https://api.routeway.ai/v1/chat/completions \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o",
"messages": [
{
"role": "user",
"content": "Alice and Bob are attending a workshop in Berlin on July 15th."
}
],
"response_format": {
"type": "json_schema",
"json_schema": {
"name": "calendar_event",
"strict": true,
"schema": {
"type": "object",
"properties": {
"name": {"type": "string"},
"date": {"type": "string"},
"participants": {
"type": "array",
"items": {"type": "string"}
},
"location": {"type": ["string", "null"]}
},
"required": ["name", "date", "participants", "location"],
"additionalProperties": false
}
}
}
}'
```
***
## Common Schema Patterns
### Extraction
Pull structured records out of unstructured text.
```python theme={null}
class ProductReview(BaseModel):
sentiment: Literal["positive", "neutral", "negative"]
score: int # 1-5
key_issues: list[str]
recommended: bool
response = client.beta.chat.completions.parse(
model="gpt-4o",
messages=[
{"role": "system", "content": "Extract a structured review from the customer feedback."},
{"role": "user", "content": "I love the design but the battery only lasts 4 hours. Not worth the price."}
],
response_format=ProductReview,
)
```
### Classification
Route or label inputs reliably.
```python theme={null}
class SupportTicket(BaseModel):
category: Literal["billing", "technical", "account", "general"]
priority: Literal["low", "medium", "high", "urgent"]
summary: str
response = client.beta.chat.completions.parse(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": "Classify the support ticket."},
{"role": "user", "content": "My card was charged twice for the same subscription last week."}
],
response_format=SupportTicket,
)
```
### Nested Objects
Schemas can be deeply nested.
```python theme={null}
class Address(BaseModel):
street: str
city: str
country: str
class Contact(BaseModel):
name: str
email: str
address: Address
tags: list[str]
```
***
## JSON Object Mode (Fallback)
For models that don't support `json_schema`, use `json_object` mode with a system prompt instructing the model to return JSON.
```python theme={null}
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{
"role": "system",
"content": 'Return a JSON object with keys: "name" (string), "score" (number 1-10), "tags" (array of strings).'
},
{"role": "user", "content": "Analyze this product description: ..."}
],
response_format={"type": "json_object"},
)
data = json.loads(response.choices[0].message.content)
```
`json_object` mode guarantees valid JSON but not schema conformance. Always validate the output against your expected shape before using it in production.
***
## Schema Design Tips
Prevents the model from including unexpected fields that could break downstream parsers.
```json theme={null}
{
"type": "object",
"properties": {...},
"required": ["name", "value"],
"additionalProperties": false
}
```
With `strict: true`, every property must appear in `required`. Optional fields should use a union type like `["string", "null"]`.
```json theme={null}
"nickname": {"type": ["string", "null"]}
```
Enums are strictly enforced and reduce hallucination on fields with a known set of values.
```json theme={null}
"status": {"type": "string", "enum": ["active", "inactive", "pending"]}
```
Deeply nested schemas increase the chance of refusal on complex inputs. Flatten where the structure allows.
# Tool Calling
Source: https://docs.routeway.ai/guides/chat-completions/tool-calling
Let models invoke your functions to fetch live data and take actions.
Tool calling (also called function calling) lets an AI model request that your application execute a specific function and return the result. The model decides when to use a tool, what arguments to pass, and how to incorporate the result into its final response.
Routeway supports the OpenAI tool calling format. Any code that works with OpenAI function calling works with Routeway unchanged — just swap the `base_url`.
## How It Works
```mermaid theme={null}
sequenceDiagram
participant App
participant Routeway
participant Model
App->>Routeway: Request + tools[]
Routeway->>Model: Forward request
Model-->>Routeway: tool_calls (name + arguments)
Routeway-->>App: Response with tool_calls
App->>App: Execute function locally
App->>Routeway: New request with tool result
Routeway->>Model: Continue generation
Model-->>Routeway: Final answer
Routeway-->>App: Final response
```
1. Define your tools in the request as JSON schemas.
2. The model returns a `tool_calls` array when it wants to call a function.
3. Your app executes the function and sends back a `role: "tool"` message.
4. The model generates its final answer using the result.
***
## Basic Example
```python theme={null}
import os
import json
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY")
)
# Step 1: Define tools
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather for a given city.",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "The city name, e.g. 'London'"
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "Temperature unit"
}
},
"required": ["city"],
"additionalProperties": False
},
"strict": True
}
}
]
messages = [{"role": "user", "content": "What's the weather like in Paris right now?"}]
# Step 2: Send request with tools
response = client.chat.completions.create(
model="gpt-4o",
messages=messages,
tools=tools,
tool_choice="auto",
)
message = response.choices[0].message
# Step 3: Check if model wants to call a tool
if message.tool_calls:
for tool_call in message.tool_calls:
args = json.loads(tool_call.function.arguments)
print(f"Model wants to call: {tool_call.function.name}({args})")
# Step 4: Execute the function (your logic here)
result = {"temperature": 18, "conditions": "Partly cloudy", "city": args["city"]}
# Step 5: Send result back
messages.append(message)
messages.append({
"role": "tool",
"tool_call_id": tool_call.id,
"content": json.dumps(result)
})
# Step 6: Get final answer
final = client.chat.completions.create(
model="gpt-4o",
messages=messages,
tools=tools,
)
print(final.choices[0].message.content)
```
```javascript theme={null}
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.routeway.ai/v1",
apiKey: process.env.ROUTEWAY_API_KEY,
});
const tools = [
{
type: "function",
function: {
name: "get_weather",
description: "Get the current weather for a given city.",
parameters: {
type: "object",
properties: {
city: { type: "string", description: "The city name" },
unit: { type: "string", enum: ["celsius", "fahrenheit"] },
},
required: ["city"],
additionalProperties: false,
},
strict: true,
},
},
];
async function main() {
const messages = [
{ role: "user", content: "What's the weather like in Paris right now?" },
];
const response = await client.chat.completions.create({
model: "gpt-4o",
messages,
tools,
tool_choice: "auto",
});
const message = response.choices[0].message;
if (message.tool_calls) {
for (const toolCall of message.tool_calls) {
const args = JSON.parse(toolCall.function.arguments);
console.log(`Calling: ${toolCall.function.name}`, args);
// Your function logic here
const result = { temperature: 18, conditions: "Partly cloudy", city: args.city };
messages.push(message);
messages.push({
role: "tool",
tool_call_id: toolCall.id,
content: JSON.stringify(result),
});
}
const final = await client.chat.completions.create({
model: "gpt-4o",
messages,
tools,
});
console.log(final.choices[0].message.content);
}
}
main().catch(console.error);
```
```bash theme={null}
# Step 1 — send request with tools
curl https://api.routeway.ai/v1/chat/completions \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o",
"messages": [
{"role": "user", "content": "What is the weather in Paris?"}
],
"tools": [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather for a city.",
"parameters": {
"type": "object",
"properties": {
"city": {"type": "string"}
},
"required": ["city"]
}
}
}
],
"tool_choice": "auto"
}'
```
***
## Controlling Tool Use
Use `tool_choice` to control how the model selects tools.
| Value | Behavior |
| --------------------------------------------------- | ---------------------------------------------- |
| `"auto"` | Model decides whether to call a tool (default) |
| `"none"` | Model will not call any tools |
| `"required"` | Model must call at least one tool |
| `{"type": "function", "function": {"name": "..."}}` | Force a specific tool |
```json theme={null}
{
"tool_choice": {
"type": "function",
"function": {"name": "get_weather"}
}
}
```
***
## Parallel Tool Calls
Models may request multiple tools simultaneously. Iterate over `message.tool_calls`, execute each in parallel, and send all results back before requesting the final answer.
```python theme={null}
import asyncio
import json
# After receiving a response with multiple tool_calls:
async def execute_all_tools(tool_calls):
results = await asyncio.gather(*[run_tool(tc) for tc in tool_calls])
return results
async def run_tool(tool_call):
args = json.loads(tool_call.function.arguments)
# dispatch to the right function...
return {"tool_call_id": tool_call.id, "result": "..."}
```
Send all `role: "tool"` messages in the same follow-up request. The model needs all results together to compose its final answer.
***
## Designing Good Tool Schemas
Tool names should clearly describe the action. The model uses the name and description to decide when to call the tool.
```json theme={null}
// Good
"name": "search_knowledge_base"
// Too vague
"name": "search"
```
Descriptions directly influence the model's decision to call a tool. Be specific about what the tool does and when to use it.
```json theme={null}
"description": "Search the internal knowledge base for articles about a topic. Use this when the user asks about product features, pricing, or support issues."
```
Avoid free-form string fields when enums or typed fields work. Tight schemas reduce parsing errors and improve reliability.
```json theme={null}
// Good
"status": {"type": "string", "enum": ["open", "closed", "pending"]}
// Fragile
"status": {"type": "string", "description": "open, closed, or pending"}
```
Return JSON objects from your functions instead of plain strings. The model can reason over structured data more reliably.
```json theme={null}
// Good
{"temperature": 18, "unit": "celsius", "conditions": "Partly cloudy"}
// Weaker
"It's 18°C and partly cloudy"
```
***
## Common Mistakes
**Don't expose tools with dangerous side effects without application-level guards.** The model can call tools in unexpected sequences. Always validate arguments before executing writes, deletions, or external API calls.
* **Forgetting to replay the assistant message** — the `messages` array must include the assistant turn containing `tool_calls` before any `role: "tool"` messages.
* **Parsing arguments before the stream ends** — when streaming, accumulate `delta.tool_calls` fragments before calling `JSON.parse`.
* **Assuming a final text response always follows** — on the first turn, the model may return *only* tool calls with no `content`. Handle both cases.
# Web Search BYOK
Source: https://docs.routeway.ai/guides/chat-completions/web-search-byok
Use your own Tavily, Exa, or Valyu API key for online search in chat completions.
**Bring Your Own Key (BYOK)** lets you use your own search provider credentials when using [Online Search](/guides/chat-completions/server-tools) (`@online`). Successful BYOK searches are not billed by Routeway for search usage — you pay the provider directly.
BYOK only affects **search** billing. Chat model token costs still apply as usual.
***
## Platform search vs BYOK
By default, Routeway handles search for you when you append `@online` to a model ID. Daily limits and optional overage billing apply — see [Server Tools pricing](/guides/chat-completions/server-tools#pricing).
With BYOK, Routeway calls your chosen provider (Tavily, Exa, or Valyu) using credentials you supply. When BYOK is used successfully, Routeway does not charge for that search.
| Search path | Routeway daily limit | Routeway search charge |
| :------------------------------- | :------------------- | :----------------------------------------------- |
| **BYOK** (Tavily, Exa, or Valyu) | Not counted | **\$0** |
| **Platform search** (default) | Yes | Free tier included; paid overage after daily cap |
If BYOK is attempted but fails (invalid key, provider error, etc.), Routeway falls back to platform search. Standard platform search limits and billing apply for that search.
Up to **3 search rounds** can run per chat request when the model needs multiple lookups.
***
## Supported providers
| Provider | Value | Notes |
| :--------- | :------- | :---------------------------- |
| **Tavily** | `tavily` | Keys often start with `tvly-` |
| **Exa** | `exa` | |
| **Valyu** | `valyu` | |
You can save one key per provider per account. Get API keys from your provider's dashboard:
* [Tavily](https://tavily.com)
* [Exa](https://exa.ai)
* [Valyu](https://valyu.ai)
***
## Quick start
1. Open [Dashboard → Settings](https://routeway.ai/dashboard/settings) and scroll to **Web Search Keys**.
2. Add a key for Tavily, Exa, or Valyu.
3. Enable **Prefer BYOK** so saved keys are used automatically on `@online` requests.
Then use `@online` as usual — no extra headers needed:
```python theme={null}
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY"),
)
response = client.chat.completions.create(
model="gpt-4o@online",
messages=[{"role": "user", "content": "What happened in AI this week?"}],
)
```
***
## How search is chosen
```mermaid theme={null}
flowchart TD
start["Chat request with @online"]
headerKey{"X-Web-Search-API-Key set?"}
useRequest["Use request key + provider"]
prefer{"Prefer BYOK enabled?"}
stored{"Saved key for provider?"}
byok["Call your provider"]
platform["Platform search"]
bill["Routeway limits + overage"]
free["No Routeway search billing"]
start --> headerKey
headerKey -->|yes| useRequest --> byok
headerKey -->|no| prefer
prefer -->|yes| stored
prefer -->|no| platform --> bill
stored -->|yes| byok
stored -->|no| platform --> bill
byok -->|success| free
byok -->|fail| platform --> bill
```
**Resolution order (highest priority first):**
1. **Per-request key** — `X-Web-Search-API-Key` (+ optional `X-Web-Search-Provider`)
2. **Saved key** — only when **Prefer BYOK** is enabled in [Settings](https://routeway.ai/dashboard/settings), or via the `X-Prefer-Web-Search-BYOK` header
3. **Platform search** — default fallback
***
## Per-request headers
For integrations that need to pass a key on each request (for example, multi-tenant apps), add these headers to `POST /v1/chat/completions` alongside your normal Routeway auth.
| Header | Required | Values | Purpose |
| :------------------------- | :--------------------------- | :----------------------- | :--------------------------------------------------------------- |
| `X-Web-Search-API-Key` | For one-off BYOK | Provider API key | Use this key for search on **this request only**. Not stored. |
| `X-Web-Search-Provider` | Recommended with request key | `tavily`, `exa`, `valyu` | Which provider to call. |
| `X-Prefer-Web-Search-BYOK` | Optional | `true` / `false` | Override the **Prefer BYOK** dashboard setting for this request. |
**Behavior details:**
* **`X-Web-Search-API-Key` present** → BYOK is used for that request. **Prefer BYOK** does not need to be enabled.
* **`X-Web-Search-Provider` omitted** with a request key → Tavily is assumed; keys starting with `tvly-` are inferred as Tavily.
* **`X-Web-Search-Provider` only** (no API key) → selects which **saved** key to use, but only if **Prefer BYOK** is enabled (in Settings or via header).
* Headers are **case-insensitive** (`x-web-search-api-key` works).
If multiple providers are saved and no `X-Web-Search-Provider` is set, priority is: **tavily → exa → valyu**.
### Example: one-off Tavily key
```bash theme={null}
curl https://api.routeway.ai/v1/chat/completions \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "X-Web-Search-API-Key: tvly-your-tavily-key" \
-H "X-Web-Search-Provider: tavily" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o@online",
"messages": [
{"role": "user", "content": "What happened in AI this week?"}
]
}'
```
```python theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY"),
default_headers={
"X-Web-Search-API-Key": os.getenv("TAVILY_API_KEY"),
"X-Web-Search-Provider": "tavily",
},
)
response = client.chat.completions.create(
model="gpt-4o@online",
messages=[{"role": "user", "content": "What happened in AI this week?"}],
)
```
Per-request keys are ideal for multi-tenant apps where each end user has their own search provider key. Keys sent in headers are never stored on Routeway's servers.
***
## Typical setups
1. Add your provider key under [Settings → Web Search Keys](https://routeway.ai/dashboard/settings)
2. Enable **Prefer BYOK**
3. Call chat with `model: "...@online"` — no extra headers needed
Send `X-Web-Search-API-Key` (+ `X-Web-Search-Provider`) on each `@online` chat request. Good for multi-tenant apps where each end user has their own search key.
Use `@online` with no BYOK headers and **Prefer BYOK** turned off. Routeway handles search; normal daily limits and overage apply. See [Server Tools](/guides/chat-completions/server-tools#online-search).
***
## Security
Routeway treats search provider keys as sensitive credentials. Here's how they're handled:
Keys saved in the dashboard are encrypted before storage and are never stored in plain text. They are decrypted only in memory when required to perform a search request and are not logged in plaintext.
After you add a key, only the last four characters are shown in the dashboard. The full value cannot be retrieved later.
Search provider keys are stored independently from your Routeway `sk-` API keys.
Keys sent via `X-Web-Search-API-Key` are used only for that request and are not saved to your account.
Routeway does not log or cache decrypted search provider keys. Keys are decrypted only when needed to make a search request to your chosen provider.
### Best practices
* **Use the dashboard or headers** — save keys in [Settings → Web Search Keys](https://routeway.ai/dashboard/settings), or pass them via `X-Web-Search-API-Key`. Never include provider keys in chat messages, prompts, or model strings.
* **Use environment variables** — load keys from a secrets manager or environment variables in your application, not hardcoded in source code.
* **Rotate if exposed** — if a key may have been leaked, update it immediately in the dashboard. The old key is replaced as soon as you save a new one.
* **Scope keys at the provider** — create separate search API keys per environment (production, staging) in your provider's dashboard when possible.
***
## FAQ
Yes. BYOK only affects search billing. Model input and output tokens are billed at the usual rates for your chosen model.
Routeway falls back to platform search for that request. Standard platform search limits and billing apply for the fallback search.
Yes — one key per provider in [Settings → Web Search Keys](https://routeway.ai/dashboard/settings). If multiple are saved and no `X-Web-Search-Provider` is set, Routeway tries them in order: tavily → exa → valyu.
Save keys in the dashboard under **Web Search Keys**, or pass them per request via the `X-Web-Search-API-Key` header. **Do not** put provider keys in the chat message body or model string.
Open [Dashboard → Settings](https://routeway.ai/dashboard/settings), scroll to **Web Search Keys**, and update the key for your provider. The old key is replaced immediately.
Yes. Saved keys are encrypted at rest and only decrypted when needed to perform a search. The full key is never shown again after you save it — only a short hint (last four characters) is displayed in the dashboard. Routeway does not log decrypted keys.
# Editing Images
Source: https://docs.routeway.ai/guides/image-generation/editing-images
Modify existing images using text prompts and optional masks.
The image editing endpoint lets you modify existing images by providing a text prompt describing the desired changes. You can optionally include a mask to constrain edits to specific regions.
```
POST https://api.routeway.ai/v1/images/edits
```
## Basic Image Editing
Provide an image and a prompt describing how to modify it.
```python theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY"),
)
response = client.images.edit(
model="gpt-image-1",
image=open("input.png", "rb"),
prompt="Add a rainbow in the sky",
)
print(response.data[0].url)
```
```javascript theme={null}
import fs from "fs";
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.routeway.ai/v1",
apiKey: process.env.ROUTEWAY_API_KEY,
});
const response = await client.images.edit({
model: "gpt-image-1",
image: fs.createReadStream("input.png"),
prompt: "Add a rainbow in the sky",
});
console.log(response.data[0].url);
```
```bash theme={null}
curl -X POST https://api.routeway.ai/v1/images/edits \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-F model="gpt-image-1" \
-F image="@input.png" \
-F prompt="Add a rainbow in the sky"
```
## Inpainting with Masks
Use a mask to specify which region of the image should be edited. The mask is a same-size image where **transparent areas** indicate where edits should be applied.
```python theme={null}
response = client.images.edit(
model="gpt-image-1",
image=open("room.png", "rb"),
mask=open("mask.png", "rb"),
prompt="A modern red sofa",
size="1024x1024",
)
print(response.data[0].url)
```
```javascript theme={null}
const response = await client.images.edit({
model: "gpt-image-1",
image: fs.createReadStream("room.png"),
mask: fs.createReadStream("mask.png"),
prompt: "A modern red sofa",
size: "1024x1024",
});
console.log(response.data[0].url);
```
```bash theme={null}
curl -X POST https://api.routeway.ai/v1/images/edits \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-F model="gpt-image-1" \
-F image="@room.png" \
-F mask="@mask.png" \
-F prompt="A modern red sofa" \
-F size="1024x1024"
```
The mask must be a PNG with an alpha channel. Transparent pixels mark the area to edit; opaque pixels are preserved.
## Supported Formats
| Format | Supported |
| ----------------- | --------- |
| PNG | ✅ |
| JPEG | ✅ |
| WebP | ✅ |
| GIF (first frame) | ✅ |
Images must be square and no larger than 4MB. Non-square images may be automatically cropped or rejected depending on the model.
## Size Constraints
The `size` parameter controls the output dimensions of the edited image.
| Size | Notes |
| ----------- | ---------------------------------- |
| `1024x1024` | Default. Works with all models. |
| `512x512` | Smaller output, faster processing. |
| `256x256` | Minimum supported size. |
For best results, provide an input image that matches the requested output size. Mismatched sizes may result in distortion or quality loss.
## Editing with Multiple Outputs
Generate several variations of an edit by setting `n`.
```python theme={null}
response = client.images.edit(
model="gpt-image-1",
image=open("photo.png", "rb"),
prompt="Replace the background with a snowy mountain landscape",
n=3,
)
for i, image in enumerate(response.data):
print(f"Variation {i + 1}: {image.url}")
```
```javascript theme={null}
const response = await client.images.edit({
model: "gpt-image-1",
image: fs.createReadStream("photo.png"),
prompt: "Replace the background with a snowy mountain landscape",
n: 3,
});
response.data.forEach((image, i) => {
console.log(`Variation ${i + 1}: ${image.url}`);
});
```
```bash theme={null}
curl -X POST https://api.routeway.ai/v1/images/edits \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-F model="gpt-image-1" \
-F image="@photo.png" \
-F prompt="Replace the background with a snowy mountain landscape" \
-F n=3
```
## Saving Edited Images
```python theme={null}
import base64
response = client.images.edit(
model="gpt-image-1",
image=open("input.png", "rb"),
prompt="Make it look like a pencil sketch",
response_format="b64_json",
)
image_data = base64.b64decode(response.data[0].b64_json)
with open("edited.png", "wb") as f:
f.write(image_data)
print("Edited image saved to edited.png")
```
```javascript theme={null}
const response = await client.images.edit({
model: "gpt-image-1",
image: fs.createReadStream("input.png"),
prompt: "Make it look like a pencil sketch",
response_format: "b64_json",
});
const imageBuffer = Buffer.from(response.data[0].b64_json, "base64");
fs.writeFileSync("edited.png", imageBuffer);
console.log("Edited image saved to edited.png");
```
```bash theme={null}
curl -X POST https://api.routeway.ai/v1/images/edits \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-F model="gpt-image-1" \
-F image="@input.png" \
-F prompt="Make it look like a pencil sketch" \
-F response_format="b64_json" \
| jq -r '.data[0].b64_json' | base64 -d > edited.png
```
## Best Practices
Without a mask, the model decides what to change. For predictable results, provide a mask that clearly marks the target region.
Describe only what should change, not the entire image. "A blue vase with flowers" works better than re-describing the whole scene.
Provide images at the same resolution you request in `size` to avoid quality degradation from rescaling.
# Generating Images
Source: https://docs.routeway.ai/guides/image-generation/generating-images
Turn text prompts into images with full control over size, quality, and format.
## Basic Generation
Generate an image by providing a model and a text prompt.
```python theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY"),
)
response = client.images.generate(
model="gpt-image-1",
prompt="A watercolor painting of a mountain lake at dawn",
)
print(response.data[0].url)
```
```javascript theme={null}
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.routeway.ai/v1",
apiKey: process.env.ROUTEWAY_API_KEY,
});
const response = await client.images.generate({
model: "gpt-image-1",
prompt: "A watercolor painting of a mountain lake at dawn",
});
console.log(response.data[0].url);
```
```bash theme={null}
curl -X POST https://api.routeway.ai/v1/images/generations \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-image-1",
"prompt": "A watercolor painting of a mountain lake at dawn"
}'
```
## Image Sizes
Control the output dimensions with the `size` parameter.
| Size | Aspect Ratio | Best For |
| ----------- | ------------ | ------------------------------------ |
| `1024x1024` | 1:1 | Square images, avatars, icons |
| `1792x1024` | 16:9 | Landscape, banners, headers |
| `1024x1792` | 9:16 | Portrait, mobile wallpapers, stories |
```python theme={null}
response = client.images.generate(
model="gpt-image-1",
prompt="A panoramic view of a tropical beach",
size="1792x1024",
)
```
```javascript theme={null}
const response = await client.images.generate({
model: "gpt-image-1",
prompt: "A panoramic view of a tropical beach",
size: "1792x1024",
});
```
```bash theme={null}
curl -X POST https://api.routeway.ai/v1/images/generations \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-image-1",
"prompt": "A panoramic view of a tropical beach",
"size": "1792x1024"
}'
```
## Quality Settings
The `quality` parameter controls the detail level and generation time.
| Quality | Description |
| -------- | ------------------------------------------------------ |
| `auto` | Automatically selects the best quality for the prompt. |
| `low` | Fastest generation, lower detail. |
| `medium` | Balanced speed and detail. |
| `high` | Maximum detail, slower generation. |
Use `auto` for most use cases. It provides the best balance without manual tuning.
## Generating Multiple Images
Set `n` to generate multiple images in a single request.
```python theme={null}
response = client.images.generate(
model="gpt-image-1",
prompt="A minimalist logo for a coffee shop",
n=4,
size="1024x1024",
)
for i, image in enumerate(response.data):
print(f"Image {i + 1}: {image.url}")
```
```javascript theme={null}
const response = await client.images.generate({
model: "gpt-image-1",
prompt: "A minimalist logo for a coffee shop",
n: 4,
size: "1024x1024",
});
response.data.forEach((image, i) => {
console.log(`Image ${i + 1}: ${image.url}`);
});
```
```bash theme={null}
curl -X POST https://api.routeway.ai/v1/images/generations \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-image-1",
"prompt": "A minimalist logo for a coffee shop",
"n": 4,
"size": "1024x1024"
}'
```
Generating multiple images multiplies token/credit usage accordingly.
## Response Format
Choose between a temporary URL or base64-encoded image data.
| Format | Use Case |
| ---------- | ---------------------------------------------------------------------------- |
| `url` | Default. Returns a short-lived URL to download the image. |
| `b64_json` | Returns the image as a base64-encoded string for direct embedding or saving. |
URLs are temporary and expire after a short period. If you need to persist the image, download it immediately or use `b64_json`.
## Saving Images Locally
```python theme={null}
import base64
response = client.images.generate(
model="gpt-image-1",
prompt="A detailed sketch of a cat wearing a top hat",
response_format="b64_json",
)
image_data = base64.b64decode(response.data[0].b64_json)
with open("output.png", "wb") as f:
f.write(image_data)
print("Image saved to output.png")
```
```javascript theme={null}
import fs from "fs";
const response = await client.images.generate({
model: "gpt-image-1",
prompt: "A detailed sketch of a cat wearing a top hat",
response_format: "b64_json",
});
const imageBuffer = Buffer.from(response.data[0].b64_json, "base64");
fs.writeFileSync("output.png", imageBuffer);
console.log("Image saved to output.png");
```
```bash theme={null}
curl -X POST https://api.routeway.ai/v1/images/generations \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-image-1",
"prompt": "A detailed sketch of a cat wearing a top hat",
"response_format": "b64_json"
}' | jq -r '.data[0].b64_json' | base64 -d > output.png
```
## Tips for Better Prompts
Instead of "a dog", try "a golden retriever sitting in a sunlit meadow, photorealistic, soft bokeh background". More detail gives the model stronger guidance.
Include style cues like "watercolor", "3D render", "pixel art", "oil painting", or "studio photography" to steer the aesthetic.
Mention framing details such as "close-up", "bird's-eye view", "centered", or "rule of thirds" for more intentional compositions.
# Image Generation
Source: https://docs.routeway.ai/guides/image-generation/overview
Generate and edit images from text prompts using DALL-E and other models.
Routeway provides an OpenAI-compatible image generation endpoint that lets you create and edit images from text prompts.
```
POST https://api.routeway.ai/v1/images/generations
```
You can also generate images **inside a chat conversation** by appending an image model tag to any text model (e.g. `qwen3.8-max-preview@flux-2-turbo`). See [Server Tools](/guides/chat-completions/server-tools).
## Quick Example
```python theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.routeway.ai/v1",
api_key=os.getenv("ROUTEWAY_API_KEY"),
)
response = client.images.generate(
model="gpt-image-1",
prompt="A futuristic cityscape at sunset with flying cars",
size="1024x1024",
quality="auto",
n=1,
)
print(response.data[0].url)
```
```javascript theme={null}
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.routeway.ai/v1",
apiKey: process.env.ROUTEWAY_API_KEY,
});
const response = await client.images.generate({
model: "gpt-image-1",
prompt: "A futuristic cityscape at sunset with flying cars",
size: "1024x1024",
quality: "auto",
n: 1,
});
console.log(response.data[0].url);
```
```bash theme={null}
curl -X POST https://api.routeway.ai/v1/images/generations \
-H "Authorization: Bearer $ROUTEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-image-1",
"prompt": "A futuristic cityscape at sunset with flying cars",
"size": "1024x1024",
"quality": "auto",
"n": 1
}'
```
## Response
The API returns an array of image objects. Depending on your `response_format`, each object contains either a URL or base64-encoded image data.
```json theme={null}
{
"created": 1700000000,
"data": [
{
"url": "https://...",
"b64_json": null
}
]
}
```
## Common Parameters
| Parameter | Type | Description |
| ----------------- | ------- | -------------------------------------------------------------- |
| `model` | string | The model to use (e.g. `gpt-image-1`). |
| `prompt` | string | A text description of the desired image. |
| `n` | integer | Number of images to generate (default: 1). |
| `size` | string | Image dimensions (e.g. `1024x1024`, `1792x1024`, `1024x1792`). |
| `quality` | string | Quality level — `auto`, `low`, `medium`, or `high`. |
| `response_format` | string | `url` or `b64_json`. |
## Learn More
Create images from text prompts with full control over size, quality, and format.
Modify existing images using text prompts and optional masks.
# Claude Code
Source: https://docs.routeway.ai/integrations/agents/claude-code
Use Claude Code with Routeway to access multiple models through a single Anthropic-compatible endpoint.
Claude Code connects to Routeway using the **Anthropic Messages API**. This lets you run Claude Code with your **Routeway API key** and choose any model that supports function calling (tools).
Requires a valid Routeway API key. Get yours from the [dashboard](https://routeway.ai/dashboard/api-keys).
***
## Quick Start
Follow the [official installation guide](https://docs.anthropic.com/en/docs/claude-code/overview) for your platform:
```bash macOS / Linux / WSL theme={null}
curl -fsSL https://claude.ai/install.sh | bash
```
```powershell Windows PowerShell theme={null}
irm https://claude.ai/install.ps1 | iex
```
```bash npm theme={null}
npm install -g @anthropic-ai/claude-code
```
Claude Code reads Anthropic-compatible environment variables. Set the base URL to Routeway and provide your API key.
Claude Code does not read `.env` files by default. Set these variables in your shell profile or use the project settings file method below.
```bash Shell (macOS / Linux / WSL) theme={null}
export ANTHROPIC_BASE_URL="https://api.routeway.ai"
export ANTHROPIC_AUTH_TOKEN="YOUR_ROUTEWAY_API_KEY"
export ANTHROPIC_API_KEY="" # Must be explicitly empty to prevent conflicts
# Optional: map Claude Code tiers to specific model IDs
export ANTHROPIC_DEFAULT_HAIKU_MODEL="claude-haiku-4.5"
export ANTHROPIC_DEFAULT_SONNET_MODEL="claude-sonnet-4.5"
export ANTHROPIC_DEFAULT_OPUS_MODEL="claude-opus-4.5"
```
```powershell Windows PowerShell theme={null}
$env:ANTHROPIC_BASE_URL="https://api.routeway.ai"
$env:ANTHROPIC_AUTH_TOKEN="YOUR_ROUTEWAY_API_KEY"
$env:ANTHROPIC_API_KEY="" # Must be explicitly empty to prevent conflicts
# Optional: map Claude Code tiers to specific model IDs
$env:ANTHROPIC_DEFAULT_HAIKU_MODEL="claude-haiku-4.5"
$env:ANTHROPIC_DEFAULT_SONNET_MODEL="claude-sonnet-4.5"
$env:ANTHROPIC_DEFAULT_OPUS_MODEL="claude-opus-4.5"
```
```json Project Settings (.claude/settings.local.json) theme={null}
{
"env": {
"ANTHROPIC_BASE_URL": "https://api.routeway.ai",
"ANTHROPIC_AUTH_TOKEN": "YOUR_ROUTEWAY_API_KEY",
"ANTHROPIC_API_KEY": "",
"ANTHROPIC_DEFAULT_HAIKU_MODEL": "claude-haiku-4.5",
"ANTHROPIC_DEFAULT_SONNET_MODEL": "claude-sonnet-4.5",
"ANTHROPIC_DEFAULT_OPUS_MODEL": "claude-opus-4.5"
}
}
```
Keep the project settings file out of version control if it contains a real API key.
You can replace the example model IDs with any model that supports function calling. See the [models list](https://routeway.ai/models).
In your project directory, run:
```bash theme={null}
claude
```
Your requests will be routed through Routeway's Anthropic-compatible endpoint (`/v1/messages`).
Inside Claude Code, run:
```
/status
```
You should see output like:
```
Auth token: ANTHROPIC_AUTH_TOKEN
Anthropic base URL: https://api.routeway.ai
```
***
## Example Prompts
Try these once you're connected:
* *"Summarize this project: key folders, build system, and entry points."*
* *"Find where X is implemented and list the relevant files."*
* *"Run tests and fix the first failing test."*
* *"Refactor: extract helper Y and update all call sites."*
# Codex
Source: https://docs.routeway.ai/integrations/agents/codex
Use OpenAI's Codex with Routeway by adding a custom provider that points to Routeway's OpenAI-compatible API.
Codex is OpenAI's open-source local coding agent and desktop application. You can connect it to Routeway by defining a custom provider in your config file and using your **Routeway API key**.
Requires a valid Routeway API key. Get yours from the [dashboard](https://routeway.ai/dashboard/api-keys).
***
## Setup
Follow the [official installation instructions](https://github.com/openai/codex) for your platform. Both the CLI and desktop app use the same settings file.
Open (or create) `~/.codex/config.toml` and add a custom provider pointing to Routeway:
```toml theme={null}
model_provider = "routeway"
model_reasoning_effort = "high"
model = "o4-mini"
[model_providers.routeway]
name = "routeway"
base_url = "https://api.routeway.ai/v1"
env_key = "ROUTEWAY_API_KEY"
```
`o4-mini` is one example starter model. You can use any model that supports tools — browse available options at [routeway.ai/models](https://routeway.ai/models).
Codex reads the API key from the environment variable named in `env_key`.
```bash Shell (macOS / Linux / WSL) theme={null}
export ROUTEWAY_API_KEY="YOUR_ROUTEWAY_API_KEY"
```
```powershell Windows PowerShell theme={null}
$env:ROUTEWAY_API_KEY="YOUR_ROUTEWAY_API_KEY"
```
Keep real API keys out of version control and shell history.
In your project directory, run:
```bash theme={null}
codex
```
Your requests will be sent through Routeway using the provider defined in `config.toml`.
***
## Reference
| Setting | Value |
| --------------- | ---------------------------- |
| Config file | `~/.codex/config.toml` |
| Base URL | `https://api.routeway.ai/v1` |
| API key env var | `ROUTEWAY_API_KEY` |
| Example model | `o4-mini` |
***
## Example Prompts
Try these once you're connected:
* *"Summarize this repository: structure, build system, and entry points."*
* *"Find where feature X is implemented and list the relevant files."*
* *"Run tests and fix the first failing test."*
* *"Refactor module Y and update all call sites."*
# Cursor
Source: https://docs.routeway.ai/integrations/agents/cursor
Use Cursor AI code editor with Routeway by overriding the OpenAI base URL.
Cursor is an AI-powered code editor. You can connect it to Routeway by overriding the OpenAI base URL in settings, giving you access to all Routeway models directly inside the editor.
Requires a valid Routeway API key. Get yours from the [dashboard](https://routeway.ai/dashboard/api-keys).
***
## Setup
Go to **Settings** → **Models** in Cursor.
Set the **Override OpenAI Base URL** field to:
```
https://api.routeway.ai/v1/
```
Enter your Routeway API key in the **API Key** field.
Click **+ Add Model** and enter the model ID you want to use. You can add multiple models. Browse available options at [routeway.ai/models](https://routeway.ai/models).
Examples:
* `gpt-4.1`
* `claude-sonnet-4.5`
* `gemini-2.5-flash`
Select your added model from the model picker in Cursor and start using it for code generation, chat, and edits.
***
## Reference
| Setting | Value |
| ------------------------ | ------------------------------- |
| Override OpenAI Base URL | `https://api.routeway.ai/v1/` |
| API Key | Your Routeway API key |
| Model format | Exact model ID (e.g. `gpt-4.1`) |
# Droid
Source: https://docs.routeway.ai/integrations/agents/droid
Use the Droid CLI agent with Routeway by adding a custom model to your config.
Droid is a CLI coding agent by Factory. You can connect it to Routeway by adding a custom model entry in `~/.factory/config.json` that points to Routeway's OpenAI-compatible API.
Requires a valid Routeway API key. Get yours from the [dashboard](https://routeway.ai/dashboard/api-keys).
***
## Setup
```bash theme={null}
curl -fsSL https://app.factory.ai/cli | sh
```
Add (or merge) the following block into `~/.factory/config.json`:
```json theme={null}
{
"custom_models": [
{
"model_display_name": "GPT 4.1 (Routeway)",
"model": "gpt-4.1",
"base_url": "https://api.routeway.ai/v1/",
"api_key": "YOUR_ROUTEWAY_API_KEY",
"provider": "generic-chat-completion-api"
}
]
}
```
Keep the trailing slash on `base_url` — Droid expects it when talking to OpenAI-compatible APIs.
If you already have other `custom_models`, just append this object to the existing array. You can add multiple Routeway models with different IDs.
Replace the `model` value with any model from [routeway.ai/models](https://routeway.ai/models).
Open a new terminal and run:
```bash theme={null}
droid
```
When prompted for a model, pick the display name you configured (e.g. **GPT 4.1 (Routeway)**) to route requests through Routeway.
***
## Reference
| Setting | Value |
| ----------- | ------------------------------------------------------- |
| Config file | `~/.factory/config.json` |
| Base URL | `https://api.routeway.ai/v1/` (trailing slash required) |
| Provider | `generic-chat-completion-api` |
***
## Example Prompts
* *"Summarize this project and list the key entry points."*
* *"Find where feature X is implemented and explain the flow."*
* *"Run tests and fix the first failure."*
# Gemini CLI
Source: https://docs.routeway.ai/integrations/agents/gemini-cli
Use Gemini CLI with Routeway through a community fork that supports OpenAI-compatible providers.
Gemini CLI can work with Routeway through a community fork that supports OpenRouter-style configuration. This lets you use Gemini CLI with any Routeway model.
This integration depends on a community fork, not the official Gemini CLI release. Review the repository before using it in production.
Requires a valid Routeway API key. Get yours from the [dashboard](https://routeway.ai/dashboard/api-keys).
***
## Setup
Use the community fork with OpenRouter support:
```bash theme={null}
git clone https://github.com/heartyguy/gemini-cli
cd gemini-cli
git checkout feature/openrouter-support
```
Point the fork at Routeway's OpenAI-compatible endpoint:
```bash Shell (macOS / Linux / WSL) theme={null}
export OPENROUTER_BASE_URL="https://api.routeway.ai/v1"
export OPENROUTER_API_KEY="YOUR_ROUTEWAY_API_KEY"
```
```powershell Windows PowerShell theme={null}
$env:OPENROUTER_BASE_URL="https://api.routeway.ai/v1"
$env:OPENROUTER_API_KEY="YOUR_ROUTEWAY_API_KEY"
```
If you want repo search, file edits, or command execution, choose a model that supports function calling. Browse available options at [routeway.ai/models](https://routeway.ai/models).
Requires Node.js 18+:
```bash theme={null}
npm install
npm start
```
Ask the CLI to inspect the current project or explain a file. If it answers normally and can use tools when needed, the Routeway connection is working.
***
## Notes
* **Base URL**: `https://api.routeway.ai/v1`
* **Auth**: Standard Routeway API key
* This setup may drift as the community fork evolves
***
## Example Prompts
Try these once you're connected:
* *"Summarize the project structure and key modules."*
* *"Find the code responsible for X and explain how it works."*
* *"Propose a refactor plan for module Y and list the files to change."*
# Grok CLI
Source: https://docs.routeway.ai/integrations/agents/grok-cli
Use Grok CLI with Routeway to access Grok models and 50+ other models through a single API key.
Grok CLI is a third-party terminal client that works with OpenAI-compatible endpoints. Routeway provides a compatible endpoint, so you can use Grok CLI with your Routeway API key to access Grok models and many others.
Requires a valid Routeway API key. Get yours from the [dashboard](https://routeway.ai/dashboard/api-keys).
***
## Setup
Requires Node.js 18 or newer:
```bash theme={null}
npm install -g @vibe-kit/grok-cli
```
Verify the installation:
```bash theme={null}
grok --version
```
If you encounter permission issues, try `sudo` (macOS/Linux) or run the command prompt as administrator (Windows).
Grok CLI reads `GROK_BASE_URL` and `GROK_API_KEY` from your environment:
```bash Shell (macOS / Linux / WSL) theme={null}
export GROK_BASE_URL="https://api.routeway.ai/v1"
export GROK_API_KEY="YOUR_ROUTEWAY_API_KEY"
```
```powershell Windows PowerShell theme={null}
$env:GROK_BASE_URL="https://api.routeway.ai/v1"
$env:GROK_API_KEY="YOUR_ROUTEWAY_API_KEY"
```
To make these persistent, add them to your shell profile (e.g. `~/.zshrc` or `~/.bashrc`).
Launch with a specific model:
```bash theme={null}
grok --model grok-3
```
Since you're connected to Routeway, you can use any available model:
```bash theme={null}
# Grok models
grok --model grok-3
grok --model grok-3-fast
grok --model grok-3-mini
# Other providers through Routeway
grok --model gpt-4.1
grok --model claude-sonnet-4.5
grok --model gemini-2.5-flash
```
Browse all available models at [routeway.ai/models](https://routeway.ai/models).
***
## Available Grok Models
| Model | Description |
| ------------- | ----------------------------------- |
| `grok-3` | xAI flagship model |
| `grok-3-fast` | Faster, more cost-effective version |
| `grok-3-mini` | Lightweight version |
***
## Known Limitations
Grok CLI has limited compatibility with thinking/reasoning models. When you use a thinking model, the full reasoning content may be shown in the output.
If you want first-party CLI support for Routeway, consider [Claude Code](/integrations/agents/claude-code) or [Codex](/integrations/agents/codex).
***
## Troubleshooting
| Issue | Solution |
| -------------------------------- | ------------------------------------------------------------------------- |
| `401 Unauthorized` | Check your API key is correct and has credits |
| `GROK_BASE_URL not set` | Ensure the environment variable is exported |
| `Model not found` | Verify the model name at [routeway.ai/models](https://routeway.ai/models) |
| `Command not found: grok` | Run `npm install -g @vibe-kit/grok-cli` again |
| Thinking content flooding output | Use a non-thinking model variant |
# Kilo Code
Source: https://docs.routeway.ai/integrations/agents/kilo-code
Use the Kilo Code VS Code extension with Routeway via the OpenAI-compatible API.
Kilo Code can connect to Routeway through the OpenAI-compatible API. Use this setup to run Kilo Code against Routeway models instead of a default provider.
Requires a valid Routeway API key. Get yours from the [dashboard](https://routeway.ai/dashboard/api-keys).
***
## Setup
* Open Extensions in VS Code (`Ctrl+Shift+X`)
* Search for **Kilo Code**
* Click **Install**
In Kilo Code settings, enter:
* **Provider**: OpenAI Compatible
* **Base URL**: `https://api.routeway.ai/v1`
* **API key**: Your Routeway API key
For coding workflows, choose a model that supports function calling. Browse available options at [routeway.ai/models](https://routeway.ai/models).
Open a repository in VS Code and ask Kilo Code to inspect the project or explain a file. If it can answer normally and use tools when needed, the setup is working.
***
## Example Prompts
Try these once you're connected:
* *"Scan this repo and explain the architecture in 10 bullets."*
* *"Find the entry point for feature X and show the call chain."*
* *"Run tests and fix the first failing test."*
* *"Refactor module X: rename Y → Z and update all imports/usages."*
# OpenClaude
Source: https://docs.routeway.ai/integrations/agents/openclaude
Use OpenClaude terminal chat client with Routeway by adding a custom provider.
OpenClaude is an open-source terminal chat client. You can connect it to Routeway by adding a custom provider through the built-in `/providers` command.
Requires a valid Routeway API key. Get yours from the [dashboard](https://routeway.ai/dashboard/api-keys).
***
## Setup
Follow the installation instructions at [github.com/Gitlawb/openclaude](https://github.com/Gitlawb/openclaude).
```bash theme={null}
openclaude
```
Type `/providers` and press Enter.
From the list, select **Add provider**. Then scroll through the provider list using arrow keys until you see **Custom** and press Enter.
Fill in the prompts as follows:
| Prompt | Value |
| --------------------- | -------------------------------------------------------------------------------- |
| **Provider name** | Any label, e.g. `Routeway` |
| **Base URL** | `https://api.routeway.ai/v1` |
| **Default model** | A model ID from [routeway.ai/models](https://routeway.ai/models), e.g. `gpt-4.1` |
| **API mode** | `Chat Completions` |
| **Auth header** | Skip (press Enter) |
| **Auth header value** | Skip (press Enter) |
| **API key** | Your Routeway API key |
| **Custom headers** | Skip (press Enter) |
Select your new Routeway provider and begin your conversation.
***
## Reference
| Setting | Value |
| -------- | --------------------------------------- |
| Base URL | `https://api.routeway.ai/v1` |
| API mode | Chat Completions |
| Auth | API key field only (skip header fields) |
***
## Example Prompts
* *"Explain this code snippet and suggest improvements."*
* *"Write a Python function that does X."*
* *"Summarize the key differences between A and B."*
# OpenClaw
Source: https://docs.routeway.ai/integrations/agents/openclaw
Connect OpenClaw to Routeway through the OpenAI-compatible API.
OpenClaw is an open-source personal AI assistant that can run on your own devices and connect to messaging channels you already use. It connects to Routeway through the OpenAI-compatible Chat Completions endpoint.
Requires a valid Routeway API key. Get yours from the [dashboard](https://routeway.ai/dashboard/api-keys).
***
## Setup
Ensure OpenClaw is installed and running on your device. Follow the [official OpenClaw documentation](https://github.com/openclaw) for installation instructions.
Open `~/.openclaw/openclaw.json` and add a provider named `routeway`. Replace `YOUR_MODEL_ID` with a model from [routeway.ai/models](https://routeway.ai/models):
```json theme={null}
{
"models": {
"mode": "merge",
"providers": {
"routeway": {
"baseUrl": "https://api.routeway.ai/v1",
"apiKey": "YOUR_ROUTEWAY_API_KEY",
"api": "openai-completions",
"models": [
{
"id": "YOUR_MODEL_ID",
"name": "Routeway model",
"api": "openai-completions"
}
]
}
}
},
"agents": {
"defaults": {
"model": {
"primary": "routeway/YOUR_MODEL_ID"
}
}
}
}
```
If your OpenClaw setup expects more model metadata, copy an existing provider entry and only change the base URL, API key, and model ID.
Apply the updated file and restart OpenClaw if needed:
```bash theme={null}
openclaw gateway config.apply --file ~/.openclaw/openclaw.json
```
Start OpenClaw and run a simple prompt. If the assistant responds and your selected model is shown correctly, the integration is working.
***
## Notes
* **Base URL** must include `/v1`: `https://api.routeway.ai/v1`
* OpenClaw model names use the provider prefix, e.g. `routeway/YOUR_MODEL_ID`
* Keep real API keys out of version control
# OpenCode
Source: https://docs.routeway.ai/integrations/agents/opencode
Use OpenCode with Routeway by adding a custom provider to the OpenCode config file.
OpenCode is an AI coding agent that runs in your terminal. You can connect it to Routeway by defining a custom provider in `~/.config/opencode/opencode.json` and authenticating with your **Routeway API key** inside OpenCode.
Requires a valid Routeway API key. Get yours from the [dashboard](https://routeway.ai/dashboard/api-keys).
***
## Setup
Follow the official installation docs at [opencode.ai](https://opencode.ai/docs).
```bash Install script theme={null}
curl -fsSL https://opencode.ai/install | bash
```
```bash npm theme={null}
npm install -g opencode-ai
```
Open `~/.config/opencode/opencode.json` and configure a custom provider named `routeway`:
```json theme={null}
{
"$schema": "https://opencode.ai/config.json",
"disabled_providers": [],
"provider": {
"routeway": {
"name": "Routeway",
"npm": "@ai-sdk/openai",
"models": {
"gpt-4.1": {
"name": "GPT 4.1",
"limit": {
"context": 1048576,
"output": 32768
},
"modalities": {
"input": ["text", "image"],
"output": ["text"]
}
}
},
"options": {
"baseURL": "https://api.routeway.ai/v1"
}
}
}
}
```
This example registers `gpt-4.1`, but you can add any Routeway model. For agentic coding, prefer models with tools support — browse at [routeway.ai/models](https://routeway.ai/models).
Start OpenCode in your project directory:
```bash theme={null}
opencode
```
Then run:
```
/connect
```
In the provider list:
* Select **Routeway**
* Paste your **Routeway API key**
* Save the credential
After connecting, select the configured Routeway model in OpenCode and start your session.
***
## Reference
| Setting | Value |
| ------------- | ---------------------------------- |
| Config file | `~/.config/opencode/opencode.json` |
| Base URL | `https://api.routeway.ai/v1` |
| Provider name | `routeway` |
| Example model | `gpt-4.1` |
***
## Example Prompts
Try these once you're connected:
* *"Summarize this repository: structure, build system, and entry points."*
* *"Find where feature X is implemented and list the relevant files."*
* *"Run tests and fix the first failing test."*
* *"Refactor module Y and update all call sites."*
# OpenHands
Source: https://docs.routeway.ai/integrations/agents/openhands
Use OpenHands autonomous agent with Routeway's OpenAI-compatible API.
OpenHands is an autonomous AI coding agent. You can connect it to Routeway by adding a custom LLM endpoint that points to Routeway's OpenAI-compatible API.
Requires a valid Routeway API key. Get yours from the [dashboard](https://routeway.ai/dashboard/api-keys).
***
## Setup
In OpenHands, add a new **LLM Endpoint** and toggle **Advanced** to reveal all fields.
Fill in the following:
| Field | Value |
| ---------------- | ----------------------------------------- |
| **Custom Model** | `openai/MODEL_ID` (e.g. `openai/gpt-4.1`) |
| **Base URL** | `https://api.routeway.ai/v1` |
| **API Key** | Your Routeway API key |
| **Agent** | `CodeActAgent` (recommended) |
The Base URL must **not** end with a trailing `/`. Use `https://api.routeway.ai/v1` exactly.
Prefix the model with `openai/` and use the exact model ID from [routeway.ai/models](https://routeway.ai/models):
* `openai/gpt-4.1`
* `openai/claude-sonnet-4.5`
* `openai/gemini-2.5-flash`
Save the configuration and start a new session. OpenHands will route requests through Routeway.
***
## Reference
| Setting | Value |
| ----------------- | ------------------------------------------------ |
| Base URL | `https://api.routeway.ai/v1` (no trailing slash) |
| Model prefix | `openai/` |
| Recommended agent | `CodeActAgent` |
***
## Example Prompts
* *"Set up a basic Express.js server with health check endpoint."*
* *"Find and fix the bug causing test failures in the auth module."*
* *"Refactor this class to use dependency injection."*
# Roo Code
Source: https://docs.routeway.ai/integrations/agents/roo-code
Use the Roo Code VS Code extension with Routeway via the OpenAI-compatible API.
Roo Code can connect to Routeway through the OpenAI-compatible API. Use this setup to run Roo Code against Routeway models inside VS Code.
Requires a valid Routeway API key. Get yours from the [dashboard](https://routeway.ai/dashboard/api-keys).
***
## Setup
* Open Extensions in VS Code (`Ctrl+Shift+X`)
* Search for **Roo Code**
* Click **Install**
In Roo Code settings, enter:
* **Provider**: OpenAI Compatible
* **Base URL**: `https://api.routeway.ai/v1`
* **API key**: Your Routeway API key
For coding workflows, choose a model that supports function calling. Browse available options at [routeway.ai/models](https://routeway.ai/models).
Open a project in VS Code and ask Roo Code to inspect the repository or explain an implementation path. If it responds normally and can use tools when needed, the setup is working.
***
## Example Prompts
Try these once you're connected:
* *"Map the main modules in this repo and what each one does."*
* *"Find where request X is handled and explain the flow."*
* *"Refactor: extract this logic into a helper and update call sites."*
* *"Run lint/tests and fix issues until it passes."*
# LibreChat
Source: https://docs.routeway.ai/integrations/chat-uis/librechat
Connect LibreChat to Routeway for a self-hosted ChatGPT-like interface.
LibreChat is an open-source, self-hosted chat interface with multi-model support. You can connect it to Routeway by adding a provider configuration to your `librechat.yml` file.
Requires a valid Routeway API key. Get yours from the [dashboard](https://routeway.ai/dashboard/api-keys).
***
## Setup
Find your LibreChat installation's `librechat.example.yml` file and create a copy named `librechat.yml` (if you haven't already).
Add the following to your `librechat.yml` under the endpoints section:
```yaml theme={null}
- name: 'Routeway'
apiKey: '${ROUTEWAY_API_KEY}'
baseURL: 'https://api.routeway.ai/v1/'
models:
default: [
"gpt-4.1",
"claude-sonnet-4.5",
"gemini-2.5-flash",
]
fetch: true
titleConvo: true
titleModel: 'gpt-4.1'
modelDisplayLabel: 'Routeway'
```
Setting `fetch: true` will automatically load all available models from Routeway. The `default` list provides quick-access models shown at the top.
Add your Routeway API key as an environment variable:
```bash Shell (macOS / Linux / WSL) theme={null}
export ROUTEWAY_API_KEY="YOUR_ROUTEWAY_API_KEY"
```
```powershell Windows PowerShell theme={null}
$env:ROUTEWAY_API_KEY="YOUR_ROUTEWAY_API_KEY"
```
Restart LibreChat to load the new configuration. Routeway will appear as a provider in the UI with all available models.
***
## Reference
| Setting | Value |
| ----------------- | ----------------------------- |
| Base URL | `https://api.routeway.ai/v1/` |
| Config file | `librechat.yml` |
| Env variable | `ROUTEWAY_API_KEY` |
| Auto-fetch models | `fetch: true` |
# Open WebUI
Source: https://docs.routeway.ai/integrations/chat-uis/openwebui
Connect Open WebUI to Routeway for a self-hosted ChatGPT-like interface with 200+ models.
Open WebUI is an open-source, self-hosted chat interface that looks and feels like ChatGPT. You can connect it to Routeway to access all available models through a familiar UI.
Requires a valid Routeway API key. Get yours from the [dashboard](https://routeway.ai/dashboard/api-keys).
***
## Setup
In Open WebUI, go to **Settings** and find the **Add Connection** option.
Enter the following details:
| Field | Value |
| ------------- | ------------------------------------------- |
| **URL** | `https://api.routeway.ai/v1` |
| **API Key** | Your Routeway API key |
| **Prefix ID** | Leave empty (optional) |
| **Model IDs** | Leave empty to include all available models |
Save the connection. You'll now have access to all Routeway models through Open WebUI's model selector.
***
## Tips
* Leave **Model IDs** empty to automatically fetch all available models from Routeway.
* Browse the full model list at [routeway.ai/models](https://routeway.ai/models) if you want to know what's available.
* Streaming works out of the box for real-time token output.
# TypingMind
Source: https://docs.routeway.ai/integrations/chat-uis/typingmind
Connect TypingMind to Routeway for an enhanced chat experience with any model.
TypingMind is a polished chat UI that supports custom OpenAI-compatible endpoints. You can add Routeway as a custom model to use any of its 200+ models.
Requires a valid Routeway API key. Get yours from the [dashboard](https://routeway.ai/dashboard/api-keys).
***
## Setup
In TypingMind, go to **Add Custom Model** and select **Create Manually**.
Fill in the following fields:
| Field | Value |
| ----------------- | ----------------------------------------------------------------------------- |
| **Name** | Any label, e.g. `Routeway` |
| **API Type** | OpenAI Compatible API |
| **Endpoint** | `https://api.routeway.ai/v1/chat/completions` |
| **Model ID** | A model from [routeway.ai/models](https://routeway.ai/models), e.g. `gpt-4.1` |
| **Authorization** | `Bearer YOUR_ROUTEWAY_API_KEY` |
Click **Test** to verify the connection, then click **Add Model**.
Select your new model from the model picker and begin your conversation.
***
## Tips
* You can add multiple Routeway models by repeating the steps with different Model IDs.
* Browse all available models at [routeway.ai/models](https://routeway.ai/models).
* The **Icon URL** field is optional — you can leave it blank or use your own icon.
# Other Integrations
Source: https://docs.routeway.ai/integrations/other
Connect any OpenAI-compatible tool or framework to Routeway.
Routeway exposes a standard **OpenAI-compatible API**, which means any application, SDK, or framework that lets you configure a custom base URL and API key can connect to Routeway with no code changes.
All integrations require a valid Routeway API key. Get yours from the [dashboard](https://routeway.ai/dashboard/api-keys).
***
## Generic Connection Details
| Setting | Value |
| ------------------------------- | --------------------------------------- |
| Base URL (OpenAI-compatible) | `https://api.routeway.ai/v1` |
| Base URL (Anthropic-compatible) | `https://api.routeway.ai` |
| Auth header | `Authorization: Bearer YOUR_API_KEY` |
| Models endpoint | `GET https://api.routeway.ai/v1/models` |
***
## How to Connect Any Tool
If your tool asks for an "OpenAI API key" and "base URL", you just need two values:
Replace the default OpenAI URL with:
```
https://api.routeway.ai/v1
```
Use your Routeway API key wherever the tool asks for an "OpenAI API key" or "API token".
Enter a model ID from [routeway.ai/models](https://routeway.ai/models). If the tool has a "fetch models" button, click it to load the full list automatically.
***
## SDK Examples
Drop-in replacement for the OpenAI SDK in any language:
```python Python theme={null}
from openai import OpenAI
client = OpenAI(
api_key="YOUR_ROUTEWAY_API_KEY",
base_url="https://api.routeway.ai/v1"
)
response = client.chat.completions.create(
model="gpt-4.1",
messages=[{"role": "user", "content": "Hello!"}]
)
```
```javascript JavaScript / TypeScript theme={null}
import OpenAI from 'openai';
const client = new OpenAI({
apiKey: 'YOUR_ROUTEWAY_API_KEY',
baseURL: 'https://api.routeway.ai/v1',
});
const response = await client.chat.completions.create({
model: 'gpt-4.1',
messages: [{ role: 'user', content: 'Hello!' }],
});
```
```bash cURL theme={null}
curl -X POST "https://api.routeway.ai/v1/chat/completions" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer YOUR_ROUTEWAY_API_KEY" \
-d '{
"model": "gpt-4.1",
"messages": [{"role": "user", "content": "Hello!"}]
}'
```
***
## Common Tools That Work Out of the Box
Any of these tools can be pointed at Routeway by changing the base URL:
| Tool / Framework | Where to set the base URL |
| ---------------------- | --------------------------------------------------------- |
| **LangChain** | `ChatOpenAI(base_url="https://api.routeway.ai/v1")` |
| **LlamaIndex** | `OpenAI(api_base="https://api.routeway.ai/v1")` |
| **Vercel AI SDK** | `createOpenAI({ baseURL: "https://api.routeway.ai/v1" })` |
| **AutoGen** | Set `base_url` in the OAI config list |
| **CrewAI** | Use the OpenAI-compatible LLM config |
| **Continue (VS Code)** | Set `apiBase` in `config.json` |
| **Cursor** | Settings → Models → OpenAI API Base |
| **LibreChat** | Set the endpoint URL in admin config |
| **TypingMind** | Custom API Endpoint field |
| **BoltAI** | Custom provider → API Base URL |
If a tool supports "OpenAI-compatible" or "custom endpoint" configuration, it will work with Routeway. Just set the base URL and API key.
***
## Anthropic-Compatible Tools
Some tools (like Claude Code) use the Anthropic Messages API instead of OpenAI's format. For those, use:
| Setting | Value |
| ---------- | ------------------------- |
| Base URL | `https://api.routeway.ai` |
| Auth token | Your Routeway API key |
| Endpoint | `/v1/messages` |
See the [Claude Code integration](/integrations/agents/claude-code) for a full example.
***
## Troubleshooting
Double-check that your API key is correct and has available balance (for paid models). Create a new key from the [dashboard](https://routeway.ai/dashboard/api-keys) if needed.
Verify the model ID matches exactly. Use the [models endpoint](https://api.routeway.ai/v1/models) or browse [routeway.ai/models](https://routeway.ai/models) to see available models.
Ensure you're using `https://api.routeway.ai/v1` (with HTTPS and `/v1`). Some tools require the trailing `/v1`, others don't — try both if you get errors.
Check if the tool supports environment variables like `OPENAI_BASE_URL` or `OPENAI_API_BASE`. Many tools read these as fallbacks.
# Integrations Overview
Source: https://docs.routeway.ai/integrations/overview
Connect Routeway to coding agents, chat UIs, and any OpenAI-compatible tool.
Routeway works with any tool that supports an OpenAI-compatible or Anthropic-compatible API. Point the tool at Routeway's base URL, provide your API key, and you're ready to go.
***
## Coding Agents
Terminal-based and IDE-based AI agents that can read, write, and execute code on your behalf.
Anthropic's CLI agent. Connects via the Anthropic Messages API.
OpenAI's open-source coding agent with custom provider support.
AI-powered code editor with OpenAI base URL override.
VS Code extension with OpenAI-compatible provider config.
Terminal coding agent with JSON-based provider configuration.
Open-source personal AI assistant for your own devices.
Autonomous AI coding agent with custom LLM endpoint support.
Factory's CLI agent with custom model configuration.
VS Code extension with OpenAI-compatible provider config.
Community fork with OpenRouter-style configuration.
Third-party Grok terminal client via OpenAI-compatible endpoint.
Open-source terminal chat client with custom provider support.
***
## Chat UIs
Web-based and self-hosted chat interfaces.
Self-hosted ChatGPT-like interface with automatic model fetching.
Polished chat UI with custom OpenAI-compatible endpoint support.
Open-source multi-model chat interface with YAML configuration.
***
## Roleplay Platforms
Character-based chat and roleplay frontends.
Web-based AI chat with Routeway as a built-in provider.
Local frontend for character-based AI chat via Chat Completions.
Web-based character chat with OpenAI-compatible endpoint support.
Web-based character AI with proxy configuration.
Character hub with built-in chat and OpenAI proxy support.
***
## Other Integrations
Routeway is fully OpenAI-compatible, so it works with virtually any tool or framework that lets you configure a custom base URL. See the [Other Integrations](/integrations/other) page for a generic connection guide.
# ChubAI
Source: https://docs.routeway.ai/integrations/roleplay/chubai
Connect ChubAI to Routeway for character-based roleplay with any model.
ChubAI is a character hub with built-in chat support. You can connect it to Routeway by configuring the OpenAI proxy settings and selecting a model from the fetched list.
Requires a valid Routeway API key. Get yours from the [dashboard](https://routeway.ai/dashboard/api-keys).
***
## Setup
Open any chat on ChubAI, then click the **Settings** icon and navigate to **Secrets**.
Under **OpenAI**, fill in:
* **API Key**: Your Routeway API key
* **Base URL**: `https://api.routeway.ai/v1`
Close the Secrets panel after saving.
Open Settings again and go to **Configuration** → **Prompt Structure**. Make sure the **API** is set to **OpenAI/Proxy**.
Click the **Model** dropdown. It will take a moment (\~2 seconds) to fetch all available models from Routeway. Once the list loads, select any model you want to use.
Close settings and begin your conversation. All requests will be routed through Routeway.
***
## Troubleshooting
Make sure the Base URL is exactly `https://api.routeway.ai/v1` and your API key is valid. Try closing and re-opening the model dropdown.
Verify that the API is set to **OpenAI/Proxy** in Configuration → Prompt Structure. If it's set to something else, ChubAI won't route requests to your configured endpoint.
Check that your Routeway account has balance (for paid models) or that you haven't hit rate limits (for free models). See [billing](/getting-started/billing) and [rate limits](/getting-started/rate-limits) for details.
# JanitorAI
Source: https://docs.routeway.ai/integrations/roleplay/janitorai
Connect JanitorAI to Routeway using the proxy configuration.
JanitorAI is a web-based character AI platform. You can route your JanitorAI conversations through Routeway by adding a proxy configuration.
Requires a valid Routeway API key. Get yours from the [dashboard](https://routeway.ai/dashboard/api-keys).
***
## Setup
In JanitorAI, go to **API Settings**.
Choose **Proxy** as the provider from the list.
Click **+ Add** under Proxy Configurations and fill in:
| Field | Value |
| ---------------------- | -------------------------------------------------------------------------------- |
| **Configuration Name** | Any label, e.g. `Routeway` |
| **Model Name** | A model ID from [routeway.ai/models](https://routeway.ai/models), e.g. `gpt-4.1` |
| **Proxy URL** | `https://api.routeway.ai/v1/chat/completions` |
| **API Key** | Your Routeway API key |
Click **Apply**, then make sure the Routeway proxy configuration is marked **Active**.
JanitorAI will now route your conversations through Routeway. Pick a character and begin.
***
## Troubleshooting
The most common causes:
* You selected a paid model but your Routeway account has no balance. Add funds from the [dashboard](https://routeway.ai/dashboard).
* The proxy URL is incorrect. It must be exactly `https://api.routeway.ai/v1/chat/completions`.
* Your API key has extra spaces — re-paste it carefully.
Try these steps:
1. Re-enter your API key
2. Refresh the page
3. Restart your browser
4. Try a different browser if the issue persists
Verify the model ID matches exactly. Browse available models at [routeway.ai/models](https://routeway.ai/models) or use the [models endpoint](https://api.routeway.ai/v1/models).
This error is **not** caused by Routeway. It appears when JanitorAI's own backend fails internally before your message ever reaches your configured proxy. Your request never makes it to Routeway.
![JanitorAI unable to process request error]()
What you can do:
1. Wait a few minutes and try again — it usually resolves on its own
2. Refresh the page or restart the browser
3. Try a different character or start a new chat
4. If it persists, JanitorAI may be experiencing downtime
If you see the error message **"A network error occurred, you may be rate limited or having connection issues: Failed to fetch"**\*, the most common cause is an incorrect proxy URL.
Verify that your proxy URL is set to:
`https://api.routeway.ai/v1/chat/completions`
Make sure the URL does **not** include a trailing slash (`/`).
If responses consistently take more than **90–100 seconds** and eventually fail with an error, the most common cause is that **Text streaming is disabled** in JanitorAI.
To enable it:
1. Open any chat
2. Click **Customize**
3. Find **Text streaming**
4. Enable it
After enabling **Text streaming**, responses should begin appearing as they're generated instead of waiting for the entire completion, preventing most 90–100 second timeouts.
# RisuAI
Source: https://docs.routeway.ai/integrations/roleplay/risuai
Connect RisuAI to Routeway for character-based chat and roleplay.
RisuAI is a web-based character chat platform. You can connect it to Routeway so your conversations run through Routeway's models.
Requires a valid Routeway API key. Get yours from the [dashboard](https://routeway.ai/dashboard/api-keys).
***
## Setup
From the RisuAI home screen, open the left menu (hamburger icon), then go to **Settings**.
If RisuAI shows onboarding prompts, skip them — you'll configure the connection in the main settings.
In **Settings**, open the **Chat Bot** panel.
Look for the API provider or custom endpoint fields and configure:
* **API Type**: OpenAI Compatible (or Custom)
* **API URL / Base URL**: `https://api.routeway.ai/v1`
* **API Key**: Your Routeway API key
Select a model from [routeway.ai/models](https://routeway.ai/models):
* Use **Select from List** if RisuAI can fetch available models.
* Use **Manual Input** if the list doesn't load — just type the model ID directly (e.g. `gpt-4.1`, `claude-sonnet-4.5`).
Toggle **Response Streaming** on for real-time token output during conversations.
***
## Troubleshooting
Switch to **Manual Input** and enter the model ID directly. You can find all available IDs at [routeway.ai/models](https://routeway.ai/models).
Confirm your Routeway API key is valid and pasted without extra spaces. Try generating a new key from the [dashboard](https://routeway.ai/dashboard/api-keys).
Ensure the API URL is set to `https://api.routeway.ai/v1` (with HTTPS and `/v1`).
# Saucepan
Source: https://docs.routeway.ai/integrations/roleplay/saucepan
Connect Saucepan to Routeway as a built-in provider for character-based chat.
Saucepan is a web-based AI chat platform that supports Routeway as a built-in provider. No custom URLs needed — just add your API key and pick a model.
Requires a valid Routeway API key. Get yours from the [dashboard](https://routeway.ai/dashboard/api-keys).
***
## Setup
Go to [saucepan.ai/settings/providers](https://saucepan.ai/settings/providers) (or navigate to **Settings** → **Providers**).
Click **Add Provider** and choose **Routeway** from the list.
Paste your Routeway API key into the token field.
Select a Model ID from the dropdown. You can use any model available at [routeway.ai/models](https://routeway.ai/models).
You're all set. Open or create a chat and your messages will be routed through Routeway.
***
## Troubleshooting
Double-check that your API key is correct and has no extra spaces. Try generating a new key from the [dashboard](https://routeway.ai/dashboard/api-keys).
Make sure you're using a valid model ID. Browse the full list at [routeway.ai/models](https://routeway.ai/models).
# SillyTavern
Source: https://docs.routeway.ai/integrations/roleplay/sillytavern
Connect SillyTavern to Routeway for character-based chat with 200+ models, including inline image generation via Server Tools.
SillyTavern is a popular local frontend for character-based AI chat. It connects to Routeway through the OpenAI-compatible Chat Completions endpoint.
Requires a valid Routeway API key. Get yours from the [dashboard](https://routeway.ai/dashboard/api-keys).
***
## Setup
Launch SillyTavern and open the **API** connection panel (the plug icon in the top bar).
Choose **Chat Completions** from the API type dropdown. Then select **OpenAI** as the source.
If you choose the Chat Completions endpoint, Routeway may automatically appear as an option in some SillyTavern versions.
Fill in the following:
* **API URL**: `https://api.routeway.ai/v1`
* **API Key**: Your Routeway API key
Set the model to any ID from [routeway.ai/models](https://routeway.ai/models), for example:
* `gpt-4.1`
* `claude-sonnet-4.5`
* `gemini-2.5-flash`
You can also click **Fetch models** if your SillyTavern version supports it.
Load a character card and begin your conversation. All requests will be routed through Routeway.
***
## Image Generation with Server Tools
You can generate images **inside SillyTavern chat** using Routeway [Server Tools](/guides/chat-completions/server-tools). Append an image model tag to your text model ID — for example `qwen3.8-max-preview@flux-2-turbo` — then ask the character to draw or show something. The image renders inline in the message:
![SillyTavern chat showing an image generated inline via Routeway Server Tools]()
### Enable external media
By default, SillyTavern may block external images. To allow Server Tool images to display:
1. Click **User Settings** (the gear icon) on the top menu bar.
2. Find **Forbid External Media** and **disable** it.
Once disabled, generated images from Routeway appear directly in the chat stream.
***
## Tips
* **Streaming** works out of the box — enable it in SillyTavern's settings for real-time token output.
* **Context size** — check the model's max context on the [models page](https://routeway.ai/models) and set SillyTavern's context limit accordingly.
* **Free vs. paid models** — free models have rate limits. If you hit them frequently during RP sessions, consider switching to a paid model.
* **Image generation** — use a `@` tagged model and disable **Forbid External Media** so images render inline. See [Server Tools](/guides/chat-completions/server-tools).