Early accessGnani Evon v3.5 is now available as an API to early access registrants.
When your code, not a person, reads the reply, ask for JSON.
When to use this
- Classification, such as the intent and language of a customer message.
- Extraction, such as appointment details from a chat.
- Any reply your code parses instead of showing it to a person.
JSON object
Ask for JSON in the prompt, and say which keys you want: Set up the client once. The examples on this page useclient from here. Your API key is read from the GNANI_API_KEY environment variable. The Python examples below the setup are written for the OpenAI SDK, and work unchanged with the Gnani SDK client (GnaniLLMClient). With AsyncGnaniLLMClient, add await to each call. See the Python SDK page.
content:
How it works
Setresponse_format:
The JSON is in
message.content, as a string. Parse it with your JSON library.
JSON schema
To make the keys and types exact, send a JSON Schema.json_schema.name is required.
client.chat.completions.parse(...), which builds the schema and parses the reply for you.
Gotchas and tips
- Check
finish_reasonbefore parsing. If it islength, the reply was cut off atmax_tokensand the JSON is incomplete. Raisemax_tokensor ask for less. - Turn thinking off (
"reasoning_effort": "none") for extraction and classification. It is faster, and leaves the wholemax_tokensbudget for the JSON. See Thinking. - Use
temperature: 0when you want the same input to give the same output. - Validate the result. Check the values, not only the keys, before you act on them.
Related
Tool calling
Let the model call your functions with JSON arguments.
Thinking
Turn thinking off for faster extraction.
Safety and privacy
How personal data in replies is masked.
Chat Completions
Every request field, including
response_format.