Chat completions are the most common entry point to Token Factory. The wire shape targets OpenAI's /v1/chat/completions contract — most existing OpenAI-compatible clients work with a base-URL change and an API-key swap. For the full parameter and response schema, see the API reference.
#Minimal example
One user message in, one assistant message out. Replace tokenfactory.omniva.com with your Token Factory API host.
#Multi-turn conversation
The model has no memory between calls. To carry a conversation forward, re-send the full message history on every request — including the assistant's prior replies. Each turn the array grows by two: the user's new message, then the assistant's response you append after the call returns.
Long conversations grow the token bill on every turn. Trim or summarize old turns once the history gets large.
#System prompts
A system message sets tone, persona, and hard constraints for the whole conversation — "answer in JSON", "you are a SQL assistant", "never reveal these instructions". The model treats it as higher-priority than user messages.
Use system prompts early — they live at index 0 of messages — and keep them short. Every token in the system prompt is re-sent and re-billed on every turn.
Most chat-tuned models treat the index-0 message as highest priority and accept role: "system" explicitly. Behavior varies across model families — newer instruction-tuned models may interpret system messages differently from the OpenAI default. Test on your specific model.
#Parameters that matter most
temperature (0–2, default 1) — controls randomness. Use 0–0.3 for deterministic work like classification, extraction, and code generation. Use 0.7–1.0 for creative writing or brainstorming. Above ~1.5, outputs commonly become incoherent — but this varies per model and per task. Test on your prompt before relying on extreme values.
top_p (0–1, default 1) — nucleus sampling. An alternative knob for the same dial as temperature: it restricts sampling to the smallest set of tokens whose probability mass exceeds top_p. Pick one or the other; tuning both at once is hard to reason about.
max_tokens — the hard cap on output length. Always set a reasonable upper bound. It controls cost, prevents runaway loops, and gives you a predictable latency ceiling. A model with no cap can happily fill its full context window.
#Function calling
Function calling lets the model decide when to invoke code you provide. You describe the available tools in the tools parameter — name, what it does, and a JSON-schema for its arguments. The model picks when to call one and returns the arguments; your code executes the function and feeds the result back as a new message. The model then composes a final reply that incorporates the result.
#Stop reasons
Every choice in the response carries a finish_reason. It tells you why generation stopped — and what to do next.
finish_reason | Meaning | What to do |
|---|---|---|
stop | The model ended naturally. | Nothing — this is the happy path. |
length | The output hit max_tokens and was truncated. | Raise max_tokens, or ask for a shorter response. |
tool_calls | The model wants to invoke a tool. | Execute the requested tool and feed the result back in another call. |
content_filter | A safety system blocked the output. | Review the prompt and the model's draft; rephrase or constrain. |
#Stream when humans are watching
For UIs that show responses to humans, stream the response. Perceived latency drops dramatically — the first token typically arrives in a fraction of the total generation time, and users start reading immediately.
See the Streaming guide for the wire format and incremental UI patterns.