Early accessGnani Evon v3.5 is now available as an API to early access registrants.
Tool calling (also called function calling) lets the model use your code. You describe your functions in the request. When the model needs one, it does not answer straight away; it replies with the function name and arguments. Your code runs the function and sends the result back, and the model uses it to write the answer.
The model never runs anything itself. It only asks.
When to use this
- Look up live data in your own systems, such as an order’s delivery status.
- Take actions through your APIs, such as booking an appointment.
- Combine your functions with built-in web search in one request.
Complete example
This loop handles any number of tool calls and rounds. The cURL tab shows the same flow as two requests.How it works
The example below lets the model look up an order.1
Describe your functions in tools
Each tool has a
name, a description that tells the model when to use it, and parameters, a JSON Schema for the arguments.2
Read the tool call
The model replies with
tool_calls instead of an answer. arguments is a JSON string, not an object:3
Run the function and send back the result
Send the conversation again with two more messages: the model’s Send the same
assistant message with its tool_calls, unchanged, and one tool message per call. tool_call_id must match the call’s id, and content holds the result as a string.tools as in step 1 with this request, so the model can call them again if it needs to.4
Get the answer
The model now answers in
content, for example: “Your order ORD-48213 has shipped and should arrive on 14 November.” It may also ask for another tool call; repeat step 3 until tool_calls is empty.Options
Choosing when the model calls a tool
tool_choice controls whether the model calls a tool:
To force one of several tools, name it with the object form.
Streaming tool calls
With"stream": true, a tool call arrives in pieces in choices[0].delta.tool_calls. The first piece for each call has its index, id and function.name; the following pieces add to function.arguments. Join the pieces by index until a chunk has a finish_reason, then parse the arguments.
The Python examples on this page work with the Gnani SDK’s
GnaniLLMClient as they are, because it is called the same way as OpenAI’s client. With AsyncGnaniLLMClient, add await to each call and use async for over a stream, as in the async tab above. See Python SDK: LLM.Web search
You do not need to write a search function. Add the built-in{"type": "web_search"} tool and the service runs the searches for you. You can use it alongside your own functions. See Web search.
Gotchas and good practice
- Validate the arguments before you run anything.
argumentsis written by the model. It is usually valid JSON that matches your schema, but check it, and never pass it straight to a database query or a shell. - Return errors as tool results. If a function fails, send the error message as the
toolmessage’scontent. The model can then explain the problem or try again. - Write clear descriptions. The model decides from
descriptionand the parameter descriptions when to call a tool and what to pass. - Every round is a separate request. Each one counts toward your rate limits and daily token quota, and sends the whole conversation again.
- Tool results are screened like user messages. Text in a tool result that tries to give the model new instructions can get the request refused with
content_filter. See Safety and privacy.
Related
Web search
The built-in search tool that runs for you.
Streaming
Read replies and tool calls as they arrive.
JSON output
Get replies as JSON that matches a schema.
Safety and privacy
How tool results are screened.