Skip to main content
Early accessGnani Evon v3.5 is now available as an API to early access registrants.
When your code, not a person, reads the reply, ask for JSON.

When to use this

  • Classification, such as the intent and language of a customer message.
  • Extraction, such as appointment details from a chat.
  • Any reply your code parses instead of showing it to a person.

JSON object

Ask for JSON in the prompt, and say which keys you want: Set up the client once. The examples on this page use client from here. Your API key is read from the GNANI_API_KEY environment variable. The Python examples below the setup are written for the OpenAI SDK, and work unchanged with the Gnani SDK client (GnaniLLMClient). With AsyncGnaniLLMClient, add await to each call. See the Python SDK page.
The reply’s content:

How it works

Set response_format: The JSON is in message.content, as a string. Parse it with your JSON library.

JSON schema

To make the keys and types exact, send a JSON Schema. json_schema.name is required.
With OpenAI’s Python SDK you can also pass a Pydantic model to client.chat.completions.parse(...), which builds the schema and parses the reply for you.

Gotchas and tips

Personal data is masked in replies. A phone number, email or ID that the model writes into the JSON is replaced with a placeholder such as [REDACTED_PHONE]. See Safety and privacy.
  • Check finish_reason before parsing. If it is length, the reply was cut off at max_tokens and the JSON is incomplete. Raise max_tokens or ask for less.
  • Turn thinking off ("reasoning_effort": "none") for extraction and classification. It is faster, and leaves the whole max_tokens budget for the JSON. See Thinking.
  • Use temperature: 0 when you want the same input to give the same output.
  • Validate the result. Check the values, not only the keys, before you act on them.

Tool calling

Let the model call your functions with JSON arguments.

Thinking

Turn thinking off for faster extraction.

Safety and privacy

How personal data in replies is masked.

Chat Completions

Every request field, including response_format.