> ## Documentation Index
> Fetch the complete documentation index at: https://docs.gnani.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Tool calling

> Let the model call your functions, such as an order lookup or a booking API, and use the results in its answer.

Early access
**Gnani Evon v3.5** is now available as an API to early access registrants.

Tool calling (also called function calling) lets the model use your code. You describe your functions in the request. When the model needs one, it does not answer straight away; it replies with the function name and arguments. **Your code runs the function** and sends the result back, and the model uses it to write the answer.

The model never runs anything itself. It only asks.

## When to use this

* **Look up live data** in your own systems, such as an order's delivery status.
* **Take actions** through your APIs, such as booking an appointment.
* **Combine your functions with built-in [web search](https://docs.gnani.ai/api/LLM/web-search.md)** in one request.

> [!WARNING]
>
> **`tool_choice: "required"` is only allowed when `tools` has exactly one tool.** With more, the request is refused with `400`. To force one of several tools, name it with the object form. See [Choosing when the model calls a tool](#choosing-when-the-model-calls-a-tool).

> [!WARNING]
>
> **Check `message.tool_calls`, not `finish_reason`.** When you force a function with `tool_choice`, the reply can have `finish_reason: "stop"` even though it contains a tool call.

***

## Complete example

This loop handles any number of tool calls and rounds. The cURL tab shows the same flow as two requests.

```bash cURL expandable theme={null}
# Step 1: send the conversation and your tools. The reply has tool_calls.
curl https://api.vachana.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key-ID: $GNANI_API_KEY" \
  -d '{
    "model": "evon-v3.5",
    "reasoning_effort": "none",
    "messages": [
      {"role": "system", "content": "You are a support assistant for an online store."},
      {"role": "user", "content": "Where is my order ORD-48213?"}
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_order_status",
          "description": "Look up the delivery status of a customer'\''s order.",
          "parameters": {
            "type": "object",
            "properties": {
              "order_id": {"type": "string", "description": "The order ID, for example ORD-48213."}
            },
            "required": ["order_id"]
          }
        }
      }
    ]
  }'

# Step 2: run get_order_status in your own code with the arguments from the reply.

# Step 3: send the conversation again with the assistant message and the tool result.
curl https://api.vachana.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key-ID: $GNANI_API_KEY" \
  -d '{
    "model": "evon-v3.5",
    "reasoning_effort": "none",
    "messages": [
      {"role": "system", "content": "You are a support assistant for an online store."},
      {"role": "user", "content": "Where is my order ORD-48213?"},
      {
        "role": "assistant",
        "content": null,
        "tool_calls": [
          {
            "id": "call_9f2c41",
            "type": "function",
            "function": {"name": "get_order_status", "arguments": "{\"order_id\": \"ORD-48213\"}"}
          }
        ]
      },
      {
        "role": "tool",
        "tool_call_id": "call_9f2c41",
        "content": "{\"status\": \"shipped\", \"expected_delivery\": \"2026-11-14\"}"
      }
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_order_status",
          "description": "Look up the delivery status of a customer'\''s order.",
          "parameters": {
            "type": "object",
            "properties": {
              "order_id": {"type": "string", "description": "The order ID, for example ORD-48213."}
            },
            "required": ["order_id"]
          }
        }
      }
    ]
  }'
```

```python Python (OpenAI SDK) expandable highlight={68} theme={null}
import json
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.vachana.ai/v1",
    api_key=os.environ["GNANI_API_KEY"],
)

def get_order_status(order_id: str) -> dict:
    # Replace with a call to your own system.
    return {"order_id": order_id, "status": "shipped", "expected_delivery": "2026-11-14"}

FUNCTIONS = {"get_order_status": get_order_status}

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_order_status",
            "description": "Look up the delivery status of a customer's order.",
            "parameters": {
                "type": "object",
                "properties": {
                    "order_id": {"type": "string", "description": "The order ID, for example ORD-48213."}
                },
                "required": ["order_id"],
            },
        },
    }
]

messages = [
    {"role": "system", "content": "You are a support assistant for an online store."},
    {"role": "user", "content": "Where is my order ORD-48213?"},
]

for _ in range(5):  # a safety limit on rounds
    completion = client.chat.completions.create(
        model="evon-v3.5",
        messages=messages,
        tools=tools,
        reasoning_effort="none",
    )
    message = completion.choices[0].message

    if not message.tool_calls:          # no more calls: this is the answer
        print(message.content)
        break

    # 1. Send back the model's message with its tool calls, unchanged.
    messages.append({
        "role": "assistant",
        "content": message.content,
        "tool_calls": [call.model_dump() for call in message.tool_calls],
    })

    # 2. Run each call and send back its result.
    for call in message.tool_calls:
        function = FUNCTIONS.get(call.function.name)
        try:
            arguments = json.loads(call.function.arguments)
            result = function(**arguments) if function else {"error": f"Unknown function {call.function.name}"}
        except (json.JSONDecodeError, TypeError) as error:
            result = {"error": f"Invalid arguments: {error}"}
        messages.append({
            "role": "tool",
            "tool_call_id": call.id,
            "content": json.dumps(result),
        })
```

```javascript JavaScript (OpenAI SDK) expandable highlight={64} theme={null}
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.vachana.ai/v1",
  apiKey: process.env.GNANI_API_KEY,
});

function getOrderStatus({ order_id }) {
  // Replace with a call to your own system.
  return { order_id, status: "shipped", expected_delivery: "2026-11-14" };
}

const FUNCTIONS = { get_order_status: getOrderStatus };

const tools = [
  {
    type: "function",
    function: {
      name: "get_order_status",
      description: "Look up the delivery status of a customer's order.",
      parameters: {
        type: "object",
        properties: {
          order_id: { type: "string", description: "The order ID, for example ORD-48213." },
        },
        required: ["order_id"],
      },
    },
  },
];

const messages = [
  { role: "system", content: "You are a support assistant for an online store." },
  { role: "user", content: "Where is my order ORD-48213?" },
];

for (let round = 0; round < 5; round++) {  // a safety limit on rounds
  const completion = await client.chat.completions.create({
    model: "evon-v3.5",
    messages,
    tools,
    reasoning_effort: "none",
  });
  const message = completion.choices[0].message;

  if (!message.tool_calls?.length) {        // no more calls: this is the answer
    console.log(message.content);
    break;
  }

  // 1. Send back the model's message with its tool calls, unchanged.
  messages.push({ role: "assistant", content: message.content, tool_calls: message.tool_calls });

  // 2. Run each call and send back its result.
  for (const call of message.tool_calls) {
    const fn = FUNCTIONS[call.function.name];
    let result;
    try {
      const args = JSON.parse(call.function.arguments);
      result = fn ? fn(args) : { error: `Unknown function ${call.function.name}` };
    } catch (error) {
      result = { error: `Invalid arguments: ${error.message}` };
    }
    messages.push({ role: "tool", tool_call_id: call.id, content: JSON.stringify(result) });
  }
}
```

***

## How it works

```mermaid theme={null}
sequenceDiagram
    participant App as Your app
    participant API as LLM API
    App->>API: messages and tools
    API-->>App: tool_calls with name and arguments
    Note over App: Run the function
    App->>API: messages and assistant tool_calls and tool result
    API-->>App: Final answer in content
```

The example below lets the model look up an order.

**1. Describe your functions in `tools`**

Each tool has a `name`, a `description` that tells the model when to use it, and `parameters`, a [JSON Schema](https://json-schema.org/understanding-json-schema) for the arguments.

```json highlight={7} theme={null}
{
  "model": "evon-v3.5",
  "messages": [
    {"role": "system", "content": "You are a support assistant for an online store."},
    {"role": "user", "content": "Where is my order ORD-48213?"}
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_order_status",
        "description": "Look up the delivery status of a customer's order.",
        "parameters": {
          "type": "object",
          "properties": {
            "order_id": {"type": "string", "description": "The order ID, for example ORD-48213."}
          },
          "required": ["order_id"]
        }
      }
    }
  ]
}
```

**2. Read the tool call**

The model replies with `tool_calls` instead of an answer. `arguments` is a **JSON string**, not an object:

```json highlight={14} theme={null}
{
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "tool_calls": [
          {
            "id": "call_9f2c41",
            "type": "function",
            "function": {
              "name": "get_order_status",
              "arguments": "{\"order_id\": \"ORD-48213\"}"
            }
          }
        ]
      },
      "finish_reason": "tool_calls"
    }
  ]
}
```

**3. Run the function and send back the result**

Send the conversation again with two more messages: the model's `assistant` message with its `tool_calls`, unchanged, and one `tool` message per call. `tool_call_id` must match the call's `id`, and `content` holds the result as a string.

```json highlight={19} theme={null}
{
  "model": "evon-v3.5",
  "messages": [
    {"role": "system", "content": "You are a support assistant for an online store."},
    {"role": "user", "content": "Where is my order ORD-48213?"},
    {
      "role": "assistant",
      "content": null,
      "tool_calls": [
        {
          "id": "call_9f2c41",
          "type": "function",
          "function": {"name": "get_order_status", "arguments": "{\"order_id\": \"ORD-48213\"}"}
        }
      ]
    },
    {
      "role": "tool",
      "tool_call_id": "call_9f2c41",
      "content": "{\"status\": \"shipped\", \"expected_delivery\": \"2026-11-14\"}"
    }
  ]
}
```

Send the same `tools` as in step 1 with this request, so the model can call them again if it needs to.

**4. Get the answer**

The model now answers in `content`, for example: *"Your order ORD-48213 has shipped and should arrive on 14 November."* It may also ask for another tool call; repeat step 3 until `tool_calls` is empty.

***

## Options

### Choosing when the model calls a tool

`tool_choice` controls whether the model calls a tool:

| `tool_choice` | What happens |
| - | - |
| `"auto"` (default) | The model decides whether to call a tool or answer. |
| `"none"` | The model answers without calling any tool. |
| `"required"` | The model must call a tool. **Only allowed when `tools` has exactly one tool**; with more, the request is refused with `400`. |
| `{"type": "function", "function": {"name": "get_order_status"}}` | The model must call this function. The name must be one of your `tools`. |

To force one of several tools, name it with the object form.

### Streaming tool calls

With `"stream": true`, a tool call arrives in pieces in `choices[0].delta.tool_calls`. The first piece for each call has its `index`, `id` and `function.name`; the following pieces add to `function.arguments`. Join the pieces by `index` until a chunk has a `finish_reason`, then parse the arguments.

```python Python (OpenAI SDK) expandable highlight={14-15} theme={null}
import json

# client, messages and tools as in the complete example
stream = client.chat.completions.create(
    model="evon-v3.5",
    messages=messages,
    tools=tools,
    stream=True,
    reasoning_effort="none",
)

calls = {}  # index -> {"id", "name", "arguments"}
for chunk in stream:
    for piece in (chunk.choices[0].delta.tool_calls if chunk.choices else None) or []:
        call = calls.setdefault(piece.index, {"id": None, "name": None, "arguments": ""})
        if piece.id:
            call["id"] = piece.id
        if piece.function and piece.function.name:
            call["name"] = piece.function.name
        if piece.function and piece.function.arguments:
            call["arguments"] += piece.function.arguments

for index in sorted(calls):
    call = calls[index]
    print(call["id"], call["name"], json.loads(call["arguments"]))
```

```python Python (Gnani SDK, async) expandable highlight={14-15} theme={null}
import json

# client (AsyncGnaniLLMClient), messages and tools as in the complete example
stream = await client.chat.completions.create(
    model="evon-v3.5",
    messages=messages,
    tools=tools,
    stream=True,
    reasoning_effort="none",
)

calls = {}  # index -> {"id", "name", "arguments"}
async for chunk in stream:
    for piece in (chunk.choices[0].delta.tool_calls if chunk.choices else None) or []:
        call = calls.setdefault(piece.index, {"id": None, "name": None, "arguments": ""})
        if piece.id:
            call["id"] = piece.id
        if piece.function and piece.function.name:
            call["name"] = piece.function.name
        if piece.function and piece.function.arguments:
            call["arguments"] += piece.function.arguments

for index in sorted(calls):
    call = calls[index]
    print(call["id"], call["name"], json.loads(call["arguments"]))
```

```javascript JavaScript (OpenAI SDK) expandable highlight={12-13} theme={null}
// client, messages and tools as in the complete example
const stream = await client.chat.completions.create({
  model: "evon-v3.5",
  messages,
  tools,
  stream: true,
  reasoning_effort: "none",
});

const calls = new Map();  // index -> { id, name, arguments }
for await (const chunk of stream) {
  for (const piece of chunk.choices[0]?.delta?.tool_calls ?? []) {
    if (!calls.has(piece.index)) calls.set(piece.index, { id: null, name: null, arguments: "" });
    const call = calls.get(piece.index);
    if (piece.id) call.id = piece.id;
    if (piece.function?.name) call.name = piece.function.name;
    if (piece.function?.arguments) call.arguments += piece.function.arguments;
  }
}

for (const index of [...calls.keys()].sort((a, b) => a - b)) {
  const call = calls.get(index);
  console.log(call.id, call.name, JSON.parse(call.arguments));
}
```

> [!NOTE]
>
> The Python examples on this page work with the Gnani SDK's `GnaniLLMClient` as they are, because it is called the same way as OpenAI's client. With `AsyncGnaniLLMClient`, add `await` to each call and use `async for` over a stream, as in the async tab above. See [Python SDK: LLM](https://docs.gnani.ai/python-sdk/llm.md).

### Web search

You do not need to write a search function. Add the built-in `{"type": "web_search"}` tool and the service runs the searches for you. You can use it alongside your own functions. See [Web search](https://docs.gnani.ai/api/LLM/web-search.md).

***

## Gotchas and good practice

* **Validate the arguments before you run anything.** `arguments` is written by the model. It is usually valid JSON that matches your schema, but check it, and never pass it straight to a database query or a shell.
* **Return errors as tool results.** If a function fails, send the error message as the `tool` message's `content`. The model can then explain the problem or try again.
* **Write clear descriptions.** The model decides from `description` and the parameter descriptions when to call a tool and what to pass.
* **Every round is a separate request.** Each one counts toward your [rate limits and daily token quota](https://docs.gnani.ai/api/LLM/limits.md), and sends the whole conversation again.
* **Tool results are screened like user messages.** Text in a tool result that tries to give the model new instructions can get the request refused with `content_filter`. See [Safety and privacy](https://docs.gnani.ai/api/LLM/safety.md).

***

## Related

> [card] [Web search](https://docs.gnani.ai/api/LLM/web-search.md)
>
> The built-in search tool that runs for you.

> [card] [Streaming](https://docs.gnani.ai/api/LLM/streaming.md)
>
> Read replies and tool calls as they arrive.

> [card] [JSON output](https://docs.gnani.ai/api/LLM/structured-outputs.md)
>
> Get replies as JSON that matches a schema.

> [card] [Safety and privacy](https://docs.gnani.ai/api/LLM/safety.md)
>
> How tool results are screened.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.