> ## Documentation Index
> Fetch the complete documentation index at: https://docs.gnani.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Python SDK — LLM

> Call the Gnani Evon chat completions API from Python with a synchronous and an asynchronous client.

Early access
**Gnani Evon v3.5** is now available as an API to early access registrants.

## Overview

The SDK has two clients for the [Chat Completions API](https://docs.gnani.ai/api/LLM/chat-completions.md), one blocking and one for `asyncio`. They are built on the `openai` package, so you call them the way you call OpenAI's Python SDK. They add the Gnani defaults: the base URL, your `GNANI_API_KEY`, and the headers the Gnani API reads.

| Class | Style | Use it for |
| - | - | - |
| `GnaniLLMClient` | sync | Scripts, notebooks, web handlers that block, anything that is not `asyncio`. |
| `AsyncGnaniLLMClient` | `async` | `asyncio` apps, FastAPI, voice agents, many requests at once. |

```python theme={null}
from gnani.llm import GnaniLLMClient, AsyncGnaniLLMClient
```

> [!NOTE]
>
> The PyPI package is `gnani-vachana`, but the Python import package is `gnani`. Install `gnani-vachana`, then `from gnani.llm import ...`.

***

## Installation

The LLM clients need the `openai` package. It is not installed with the base SDK, so ask for the `llm` extra:

```bash pip theme={null}
pip install "gnani-vachana[llm]"
```

```bash uv theme={null}
uv add "gnani-vachana[llm]"
```

**Requirements:** Python 3.9+, and `openai` 1.40 or later (the extra installs it).

Speech-to-Text and Text-to-Speech do not need the extra. If you import `gnani.llm` without it, Python raises:

```text theme={null}
ImportError: gnani.llm needs the 'openai' package. Install it with: pip install 'gnani-vachana[llm]'
```

***

## Authentication

Create a key in the [Gnani API dashboard](https://app.gnani.ai/voice). The key must have access to the LLM API; a key without it gets `403 SCOPE_NOT_ALLOWED`.

Set it as an environment variable. Both clients read it automatically:

```bash theme={null}
export GNANI_API_KEY="<your-api-key>"
```

```python theme={null}
from gnani.llm import GnaniLLMClient

client = GnaniLLMClient()                       # picks up GNANI_API_KEY
client = GnaniLLMClient(api_key="<your-api-key>")  # or pass it explicitly
```

If there is no key, the constructor raises an error before any network call.

> [!NOTE]
>
> The key is read from `GNANI_API_KEY` only, never from `OPENAI_API_KEY`. An OpenAI key in your environment is never sent to Gnani.

***

## Your first request

```python Sync theme={null}
from gnani.llm import GnaniLLMClient

client = GnaniLLMClient()

completion = client.chat.completions.create(
    model="evon-v3.5",
    messages=[{"role": "user", "content": "Say hello in Hindi."}],
    reasoning_effort="none",
)
print(completion.choices[0].message.content)
```

```python Async theme={null}
import asyncio
from gnani.llm import AsyncGnaniLLMClient


async def main():
    client = AsyncGnaniLLMClient()

    completion = await client.chat.completions.create(
        model="evon-v3.5",
        messages=[{"role": "user", "content": "Say hello in Hindi."}],
        reasoning_effort="none",
    )
    print(completion.choices[0].message.content)


asyncio.run(main())
```

```text theme={null}
नमस्ते! आप कैसे हैं?
```

> [!NOTE]
>
> `model` defaults to `evon-v3.5`, the model the API serves, so you can leave it out. The examples on this page set it explicitly. Any other model name fails with `404 model_not_found`.

> [!NOTE]
>
> **The model thinks before it answers, by default.** Thinking counts toward `max_tokens`, so a short reply can come back with empty `content` and `finish_reason: "length"`. The examples send `reasoning_effort="none"` to turn thinking off. See [Thinking](https://docs.gnani.ai/api/LLM/thinking.md).

***

## Streaming

`stream` is `False` unless you set it. With `stream=True` the sync client returns an iterator and the async client returns an async iterator.

```python Sync theme={null}
from gnani.llm import GnaniLLMClient

client = GnaniLLMClient()

stream = client.chat.completions.create(
    model="evon-v3.5",
    messages=[{"role": "user", "content": "Explain UPI in three sentences."}],
    stream=True,
    stream_options={"include_usage": True},
    reasoning_effort="none",
)

for chunk in stream:
    if not chunk.choices:            # the usage chunk
        print("\n", chunk.usage)
        continue
    text = chunk.choices[0].delta.content
    if text:
        print(text, end="", flush=True)
```

```python Async theme={null}
import asyncio
from gnani.llm import AsyncGnaniLLMClient


async def main():
    client = AsyncGnaniLLMClient()

    stream = await client.chat.completions.create(
        model="evon-v3.5",
        messages=[{"role": "user", "content": "Explain UPI in three sentences."}],
        stream=True,
        stream_options={"include_usage": True},
        reasoning_effort="none",
    )

    async for chunk in stream:
        if not chunk.choices:        # the usage chunk
            print("\n", chunk.usage)
            continue
        text = chunk.choices[0].delta.content
        if text:
            print(text, end="", flush=True)


asyncio.run(main())
```

`stream_options` is accepted only with `stream=True`; without it the API returns `400 invalid_request`. See [Streaming](https://docs.gnani.ai/api/LLM/streaming.md) for the event format.

***

## Parameters

Every parameter on the [Chat Completions](https://docs.gnani.ai/api/LLM/chat-completions.md) page works with both clients. What changes is how you pass it.

### Parameters OpenAI also has

Pass these as normal keyword arguments: `messages`, `model`, `stream`, `stream_options`, `max_tokens`, `max_completion_tokens`, `temperature`, `top_p`, `stop`, `seed`, `frequency_penalty`, `presence_penalty`, `reasoning_effort`, `tools`, `tool_choice`, `parallel_tool_calls`, `response_format` and `user`.

```python theme={null}
completion = client.chat.completions.create(
    model="evon-v3.5",
    messages=[{"role": "user", "content": "Write a two-line poem about the monsoon."}],
    temperature=0.8,
    max_tokens=200,
    seed=42,
)
```

### Gnani-only parameters

These are not part of OpenAI's API, so `openai`'s method signature does not have them. Pass them in `extra_body`, as a dictionary. The SDK sends them with the rest of the request.

| Field | Type and range | What it does |
| - | - | - |
| `top_k` | integer, `-1` or `0` for no limit, or positive | Sample from the `top_k` most likely tokens. |
| `min_p` | number, `0` to `1` | Drop tokens less likely than `min_p` times the most likely one. |
| `typical_p` | number, above `0`, up to `1` | Typical sampling. |
| `repetition_penalty` | number, above `0`, up to `2` | Discourages repetition. |
| `no_repeat_ngram_size` | integer, `0` to `20` | Never repeat a run of this many tokens. |
| `enable_thinking` | boolean | Turns thinking on or off. Wins over `reasoning_effort`. |
| `reasoning_budget` | integer, `0` to `65536` | The most tokens the model may spend thinking. |

The built-in web search is a tool, so it needs no `extra_body`: `tools=[{"type": "web_search"}]`. See [Web search](https://docs.gnani.ai/api/LLM/web-search.md).

```python Sync theme={null}
completion = client.chat.completions.create(
    model="evon-v3.5",
    messages=[{"role": "user", "content": "Write a two-line poem about the monsoon."}],
    temperature=0.8,
    extra_body={"top_k": 50, "repetition_penalty": 1.1},
)
```

```python Async theme={null}
completion = await client.chat.completions.create(
    model="evon-v3.5",
    messages=[{"role": "user", "content": "Write a two-line poem about the monsoon."}],
    temperature=0.8,
    extra_body={"top_k": 50, "repetition_penalty": 1.1},
)
```

> [!WARNING]
>
> Passing a Gnani-only field as a keyword argument, for example `top_k=50`, raises `TypeError: unexpected keyword argument`. Put it in `extra_body`.

> [!TIP]
>
> The SDK does not check values. The API does, and answers `400 invalid_request` with a message that names the field. A field name it does not know, such as a typo like `topk`, is refused with `400 unknown_parameter`.

### Not supported

`n` greater than `1` and image content parts are refused with `400`. `logprobs`, `top_logprobs`, `logit_bias`, `modalities`, `prediction`, `store`, `metadata` and `service_tier` are accepted and ignored. See [Migrating from OpenAI](https://docs.gnani.ai/api/LLM/openai-compatibility.md).

***

## Thinking

When thinking is on, the model's reasoning comes back apart from the answer, in `message.reasoning`. It is not in OpenAI's typed response, but the SDK keeps it, so you read it as an attribute:

```python theme={null}
completion = client.chat.completions.create(
    model="evon-v3.5",
    messages=[{"role": "user", "content": "Is 221 a prime number?"}],
    max_tokens=1024,
    extra_body={"reasoning_budget": 256},
)

message = completion.choices[0].message
print(message.reasoning)   # the thinking
print(message.content)     # the answer
```

While streaming, thinking arrives in `chunk.choices[0].delta.reasoning`, before the answer's `content`. Because `delta` is typed by `openai`, read it with `getattr(delta, "reasoning", None)`.

See [Thinking](https://docs.gnani.ai/api/LLM/thinking.md) for how it counts toward `max_tokens` and when to turn it off.

***

## Tool calling

Tool calling is the standard OpenAI shape. Both clients take `tools` and `tool_choice`, and return `message.tool_calls`.

```python theme={null}
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a city.",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"],
            },
        },
    }
]

completion = client.chat.completions.create(
    model="evon-v3.5",
    messages=[{"role": "user", "content": "What is the weather in Pune?"}],
    tools=tools,
    reasoning_effort="none",
)

for call in completion.choices[0].message.tool_calls or []:
    print(call.function.name, call.function.arguments)
```

Run the function, then send its result back as a `tool` message. The full loop is in [Tool calling](https://docs.gnani.ai/api/LLM/tool-calling.md). With the async client, `await` the call and everything else is the same.

***

## Error handling

Errors are `openai` exceptions, so you can catch them the way you do for OpenAI. `e.status_code` is the HTTP status and `e.code` is the error code, for example `invalid_request` or `invalid_api_key`.

| Exception | Raised when |
| - | - |
| `openai.AuthenticationError` | `401`, a missing or invalid key. |
| `openai.PermissionDeniedError` | `403`, the key has no LLM scope. |
| `openai.BadRequestError` | `400`, a value is out of range or a field is unknown. |
| `openai.NotFoundError` | `404`, for example `model_not_found`. |
| `openai.RateLimitError` | `429`, request rate or the daily token quota. |
| `openai.APIConnectionError` | The API could not be reached. |
| `openai.APIStatusError` | The parent of every HTTP error above. |

```python theme={null}
import openai
from gnani.llm import GnaniLLMClient

client = GnaniLLMClient()

try:
    completion = client.chat.completions.create(
        model="evon-v3.5",
        messages=[{"role": "user", "content": "What is UPI?"}],
    )
    print(completion.choices[0].message.content)
except openai.APIStatusError as e:
    print(e.status_code, e.code, e.message)
except openai.APIConnectionError:
    print("Could not reach the API. Retry with backoff.")
```

The two error shapes, and the codes, are on the [Errors](https://docs.gnani.ai/api/LLM/errors.md) page. The clients retry connection errors, `408`, `409`, `429` and `5xx` twice by default; change that with `max_retries`.

***

## Other options

Both clients take every `openai` constructor option. The ones you will use most:

| Option | Default | What it does |
| - | - | - |
| `api_key` | `GNANI_API_KEY` | Your Gnani API key. |
| `base_url` | `https://api.vachana.ai` | The server. `/v1` is added if you leave it out. |
| `timeout` | `120.0` | Seconds to wait for a response. |
| `max_retries` | `2` | Automatic retries. `0` turns them off. |
| `default_headers` | | Extra headers on every request, for example `X-API-Request-ID`. |

```python theme={null}
client = GnaniLLMClient(timeout=60, max_retries=0)
```

The client sends your key in both `X-API-Key-ID` and `Authorization: Bearer`. Headers you pass in `default_headers` win.

***

## Further reading

> [card] [Chat Completions reference](https://docs.gnani.ai/api/LLM/chat-completions.md)
>
> Every request parameter and the response objects.

> [card] [Migrating from OpenAI](https://docs.gnani.ai/api/LLM/openai-compatibility.md)
>
> What works as in OpenAI, what differs, and the Gnani-only options.

> [card] [Streaming](https://docs.gnani.ai/api/LLM/streaming.md)
>
> The event format and stream errors.

> [card] [Errors](https://docs.gnani.ai/api/LLM/errors.md)
>
> Both error shapes and every code.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.