Skip to main content
Early accessGnani Evon v3.5 is now available as an API to early access registrants.
Tool calling (also called function calling) lets the model use your code. You describe your functions in the request. When the model needs one, it does not answer straight away; it replies with the function name and arguments. Your code runs the function and sends the result back, and the model uses it to write the answer. The model never runs anything itself. It only asks.

When to use this

  • Look up live data in your own systems, such as an order’s delivery status.
  • Take actions through your APIs, such as booking an appointment.
  • Combine your functions with built-in web search in one request.
tool_choice: "required" is only allowed when tools has exactly one tool. With more, the request is refused with 400. To force one of several tools, name it with the object form. See Choosing when the model calls a tool.
Check message.tool_calls, not finish_reason. When you force a function with tool_choice, the reply can have finish_reason: "stop" even though it contains a tool call.

Complete example

This loop handles any number of tool calls and rounds. The cURL tab shows the same flow as two requests.

How it works

The example below lets the model look up an order.
1

Describe your functions in tools

Each tool has a name, a description that tells the model when to use it, and parameters, a JSON Schema for the arguments.
2

Read the tool call

The model replies with tool_calls instead of an answer. arguments is a JSON string, not an object:
3

Run the function and send back the result

Send the conversation again with two more messages: the model’s assistant message with its tool_calls, unchanged, and one tool message per call. tool_call_id must match the call’s id, and content holds the result as a string.
Send the same tools as in step 1 with this request, so the model can call them again if it needs to.
4

Get the answer

The model now answers in content, for example: “Your order ORD-48213 has shipped and should arrive on 14 November.” It may also ask for another tool call; repeat step 3 until tool_calls is empty.

Options

Choosing when the model calls a tool

tool_choice controls whether the model calls a tool: To force one of several tools, name it with the object form.

Streaming tool calls

With "stream": true, a tool call arrives in pieces in choices[0].delta.tool_calls. The first piece for each call has its index, id and function.name; the following pieces add to function.arguments. Join the pieces by index until a chunk has a finish_reason, then parse the arguments.
The Python examples on this page work with the Gnani SDK’s GnaniLLMClient as they are, because it is called the same way as OpenAI’s client. With AsyncGnaniLLMClient, add await to each call and use async for over a stream, as in the async tab above. See Python SDK: LLM.
You do not need to write a search function. Add the built-in {"type": "web_search"} tool and the service runs the searches for you. You can use it alongside your own functions. See Web search.

Gotchas and good practice

  • Validate the arguments before you run anything. arguments is written by the model. It is usually valid JSON that matches your schema, but check it, and never pass it straight to a database query or a shell.
  • Return errors as tool results. If a function fails, send the error message as the tool message’s content. The model can then explain the problem or try again.
  • Write clear descriptions. The model decides from description and the parameter descriptions when to call a tool and what to pass.
  • Every round is a separate request. Each one counts toward your rate limits and daily token quota, and sends the whole conversation again.
  • Tool results are screened like user messages. Text in a tool result that tries to give the model new instructions can get the request refused with content_filter. See Safety and privacy.

Web search

The built-in search tool that runs for you.

Streaming

Read replies and tool calls as they arrive.

JSON output

Get replies as JSON that matches a schema.

Safety and privacy

How tool results are screened.