OpenAI-compatible API

What it is

https://oai.anex.sh/v1 answers OpenAI-compatible requests right away: chat completions (streamed or not), embeddings, typed decisions and a model list. Point any OpenAI SDK at it with an API key from the console's Keys page, sent as Authorization: Bearer <api-key>. Requests bill from the same credit balance as batch jobs. The same host runs files of requests through an OpenAI-compatible batch API.

from openai import OpenAI

client = OpenAI(base_url="https://oai.anex.sh/v1", api_key="<api-key>")
reply = client.chat.completions.create(
    model="gemma4-12b",
    messages=[{"role": "user", "content": "Name three rivers in Europe."}],
    max_tokens=200,
)
print(reply.choices[0].message.content, reply.usage)

Models

GET /v1/models lists the chat and embedding models with their prices, context length, supported parameters, media formats and limits. It is public: no API key needed. GET /v1/models/{model} returns one model's entry from that list (the OpenAI SDK's models.retrieve), and 404 for a model not on it. laya is not on that list: its price is in the realtime price table and its limits under Decisions.

Chat completions

POST /v1/chat/completions takes the OpenAI request body. Supported parameters: temperature (0 to 2), top_p (0 to 1), max_tokens (or max_completion_tokens, at most 32,768), stop (up to 4 sequences), seed, frequency_penalty and presence_penalty (-2 to 2), logprobs and top_logprobs (0 to 20; above 0 needs logprobs true), top_k (0 or more; 0 turns it off), min_p (0 to 1), repetition_penalty (above 0, at most 2), response_format (text, json_object, or json_schema with a JSON schema), tools, tool_choice and parallel_tool_calls, reasoning (and OpenAI's reasoning_effort), and chat_template_kwargs as in batch input files. A value outside these ranges is refused with a 400 that names the parameter. Taken and ignored: OpenAI fields that leave the answer as it is (user, store, metadata, service_tier, prompt_cache_key, safety_identifier), the routing fields model routers send (provider, models, route, transforms, plugins, usage), and structured_outputs as a true or false flag. Any other field, such as logit_bias, ignore_eos or min_tokens, is refused with a 400 that names it. A message's role is system, developer, user, assistant or tool; any other role is refused with a 400 that names it. n may be left out or set to 1; a request for more choices is refused with a 400, so send separate requests instead. Log probabilities cover every generated token, the thinking included, so logprobs together with reasoning.exclude is refused with a 400 that names both. They leave out the token that ends the model's turn and the newline that some models write before the answer and the answer drops, so with thinking off they match content token for token; with thinking on they also carry the thinking and the markers around it. The context is 32,768 tokens; a request that does not fit is refused with a 400 whose code is context_length_exceeded and whose message gives the context and, where the model server reports them, the tokens asked for, split into the messages and the completion when both are known. Counting stops once a prompt passes the context, so the messages' count can read "at least" a number.

tool_choice takes "auto" (the default), "required", a named function ({"type": "function", "function": {"name": "get_weather"}}) and "none". With "required" the answer is one or more tool calls, and with a named function a call to that function, on both chat models; parallel_tool_calls: false keeps a "required" answer to one call. With "none" the model is kept from starting a tool call and answers in text; earlier tool calls and results in messages are still read. Both chat models answer a "none" request without thinking, whatever reasoning asks. On gemma4-12b the tools are left out of the prompt. On nemotron-3-nano-omni-30b they are left out until messages holds a tool call or a tool result, and stay in from then on, and a short instruction to answer in plain text is put at the top of the system prompt (up to about 80 tokens, counted in prompt_tokens and against the context length). A "required" answer finishes with finish_reason: "tool_calls" and a named function's with "stop", as OpenAI's do; "tool_calls" only ever comes with tool_calls. A forced call ("required" or a named function) cut off by the output cap comes back as the partial call text in content, with finish_reason: "length"; with thinking on, leave room for the thinking in max_tokens. A forced call takes the place of a response_format sent beside it, which is ignored. With a named function every tool stays in the prompt and the call's arguments follow the named function's schema, so give similar tools distinct names and descriptions. A tool_choice that names a function missing from tools, or asks for a call without tools, is refused with a 400.

A non-streamed answer carries usage. A long answer is better streamed: one that is not streamed and takes longer than 280 seconds, media downloads included, ends in a 504.

Streaming

Send "stream": true for server-sent events (SSE) in the OpenAI chunk format, ending with data: [DONE]. To get the token counts, also send "stream_options": {"include_usage": true}: the stream then ends with a usage chunk, as OpenAI's does. While the model is silent (thinking before its first token, for example), the stream sends a comment line, : keep-alive, every 10 seconds; SSE clients skip it. If the model fails part way through, the last chunk carries an error object and finish_reason: "error".

Closing the connection mid-stream stops the answer. You pay for the tokens generated up to that point.

Images, audio and video in chat

Media goes in the message content as image_url, input_audio and video_url parts, by URL or inline as base64. Audio by URL goes in input_audio.data or as an audio_url part ({"type": "audio_url", "audio_url": {"url": "https://..."}}). A request takes at most 2 images, 1 audio clip and 1 video. Sizes are in the media limits below.

Accepted formats, by media type: images image/png, image/jpeg, image/webp, image/gif, image/bmp and image/tiff; audio audio/wav, audio/mpeg (MP3), audio/mp4 (M4A), audio/aac, audio/ogg, audio/opus, audio/flac and audio/aiff; video video/mp4, video/webm, video/quicktime (MOV), video/x-matroska (MKV) and video/x-msvideo (AVI). The embeddings route takes the same image and video formats. The format is read from input_audio.format, a data URI's media type or a URL's file extension, and checked against the content; any other format is refused with a 400. GET /v1/models names the formats its schema has a value for (PNG, JPEG, WebP and GIF images, WAV and MP3 audio, MP4 and WebM video); this list is the full one.

Reasoning

Both chat models can think before they answer. nemotron-3-nano-omni-30b thinks unless you send "reasoning": {"enabled": false}. gemma4-12b answers straight away unless you send {"enabled": true}, an effort or a max_tokens. On gemma4-12b, {"enabled": true, "max_tokens": 1000} caps the thinking at 1,000 tokens, including the short note that closes it when the cap is reached (about 24 tokens, so a cap below that can be passed by the note). The cap is not applied beside a response_format. nemotron-3-nano-omni-30b takes no cap and ignores max_tokens. {"exclude": true} leaves the thinking text out of the answer, and cannot be combined with logprobs, whose log probabilities would carry the thinking tokens. An effort other than "none" turns thinking on without capping it; use max_tokens for a cap. OpenAI's top-level reasoning_effort takes the same values (none, minimal, low, medium, high, xhigh, listed under supported_parameters in /v1/models): "none" turns thinking off and any other level turns it on. When it is returned, the thinking text is in reasoning_content. Thinking tokens bill as output tokens.

Embeddings

POST /v1/embeddings is the OpenAI embeddings call. dimensions cuts the vector to any width from 64 to 2048, and encoding_format is float (the default) or base64 (little-endian float32 bytes). input is a string, a list of up to 128 strings, or content arrays. Token ids (an array of integers, or an array of such arrays) are not accepted and get a 400; send the text itself. Content arrays look like this:

client.embeddings.create(
    model="qwen3-vl-embedding-2b",
    input=[[
        {"type": "text", "text": "a red bicycle"},
        {"type": "image_url", "image_url": {"url": "https://example.com/bike.jpg"}},
    ]],
    dimensions=1024,
)

Each content array is one input, embedded together into one vector, and holds text, image_url and input_video parts ({"type": "input_video", "input_video": {"data": "<base64 or URL>", "format": "mp4"}}). A request takes at most 16 content arrays and 16 image and video parts across them. One content array holds at most 4 images and 1 video; send more as separate content arrays. Text is at most 262,144 characters per input and 2,097,152 in all. Sizes are in the media limits below. Audio and file parts are refused. usage.prompt_tokens counts every input token, including the tokens the images and video become; each image and each second of video adds its fee (see how media is billed). A video is sampled at 1 frame per second and every frame is read at up to 156,800 pixels, 81 tokens a frame. A video embedded before this change was read at that size for the whole clip, so it gives a different vector now: re-embed stored video vectors before comparing them with new ones.

Media limits

A request body is at most 32 MiB, so an item sent inline as base64 can be up to about 24 MiB. Larger media goes by URL. By URL, each image may be up to 20 MiB, each video up to 32 MiB, and each audio clip up to 20 MiB on gemma4-12b and 32 MiB on nemotron-3-nano-omni-30b, with 64 MiB of media in one request. Each URL is downloaded once. A media URL must answer with the file itself: a redirect (an HTTP 3xx answer, as from a short link or some CDNs) is not followed and the request is refused with a 400, so send the final URL. This holds for chat and embeddings alike. All of a request's URLs must arrive within 300 seconds, and an image within 10; past that the request answers 400. GET /v1/models lists, per model and medium, the largest item every route accepts.

Decisions

POST /v1/systemone answers typed questions about a state, with a probability per option. The request and answer have the same shape as on the realtime endpoint; see realtime decisions. The model reads each question together with the state in a window of 512 tokens for English and other Latin-script text, or 1,024 when it reads the text with its multilingual checkpoint (other scripts, or a lang naming another language). On this host a state too long for a question is refused with a 400 that names the question and says how many tokens to cut. Batches, and the realtime API on api.anex.sh, cut such a state to the window instead. A question the model cannot answer as written (an unknown type, no instructions, yes-or-no criteria keyed other than true and false) is refused with a 422 that names it.

As in a batch decision row, a request carries model, state, questions and an optional lang; any other field (a temperature, say) is refused with a 400 that names it. The answer carries, per question, type, the answer itself (choice, noul or score, with legend for a score), probabilities, confidence and answer_confidence, plus usage and routing with its model and reason. Batch results from this host carry the same fields.

Prices

Requests on this host bill at the realtime prices and batches at the batch prices, from the same credit balance and at the same prices as on api.anex.sh. GET /v1/models returns the same realtime figures.

Images, audio and video add a fee to their tokens: how media is billed.

Limits and status codes

Each organization may have 8 requests in flight at once, shared with the realtime API on api.anex.sh (ask support for more), and each model has a concurrency limit, listed in /v1/models (laya: 16 requests). Past either limit the answer is an immediate 429 with a Retry-After header; requests are never queued.

Error bodies follow the OpenAI shape: {"error": {"message": ..., "type": ..., "code": ...}}. The code is always a string: the name shown in the table where an error has one, else a name for the status, such as invalid_request, not_found, rate_limit_exceeded or upstream_error. A malformed request (a field of the wrong type, out of range, missing or unknown, or a body that is not a JSON object) is a 400 with code invalid_request whose message names the field as you write it, for example messages[0].content must be a string or an array of content parts.

Your data

We do not store the prompts, media or answers of realtime requests, and we never use them to train models. For billing we record each request's time, model, token counts, media size or length, status and duration. Requests are served from North America (the United States, Canada and Mexico). Files you send to the batch API are stored for the times given there.

Batch API

The same host serves an OpenAI-compatible batch API. Upload a JSONL file of requests, create a batch from it, poll the batch, and download the results. Any OpenAI SDK works with the base URL https://oai.anex.sh/v1 and your API key. Batches bill at the batch prices, the same as a batch job on api.anex.sh.

import time
from openai import OpenAI

client = OpenAI(base_url="https://oai.anex.sh/v1", api_key="<api-key>")
with open("requests.jsonl", "rb") as fh:
    f = client.files.create(file=fh, purpose="batch")
batch = client.batches.create(input_file_id=f.id,
                              endpoint="/v1/chat/completions",
                              completion_window="24h")
while batch.status not in ("completed", "failed", "expired", "cancelled"):
    time.sleep(60)
    batch = client.batches.retrieve(batch.id)
print(batch.status, batch.request_counts)
if batch.output_file_id:
    client.files.content(batch.output_file_id).write_to_file("output.jsonl")
if batch.error_file_id:
    client.files.content(batch.error_file_id).write_to_file("errors.jsonl")

Batch input file

One request per line, each with custom_id (a non-empty string, unique in the file), method (POST), url and body. Every line of a file names the same url and the same body.model: a batch runs one model on one endpoint, /v1/chat/completions, /v1/embeddings or /v1/systemone. Save the file as UTF-8 without a byte order mark.

{"custom_id":"row-1","method":"POST","url":"/v1/chat/completions","body":{"model":"gemma4-12b","messages":[{"role":"user","content":"Say hello in French."}],"max_tokens":200}}

Each chat request keeps its own output cap, max_tokens or max_completion_tokens. A request without one gets 4,096 tokens, or the model's maximum output if that is lower. From a chat request's body, messages, response_format, chat_template_kwargs, reasoning and the output cap are used, as described for batch input files, along with the sampling fields temperature (0 to 2), top_p (0 to 1, as on the realtime route), seed and stop (up to 4 strings). Other fields, including n, are not carried through. Embedding requests take input and dimensions, with encoding_format left out or "float". Decision requests take state, questions and lang.

reasoning works per request. {"enabled": false} or {"effort": "none"} turns thinking off. {"effort": "minimal" | "low" | "medium" | "high" | "xhigh"} turns it on. Neither chat model has effort levels of its own, so every level other than "none" thinks the same way. {"max_tokens": N} caps the thinking at N tokens, including the short note that closes it, on gemma4-12b (nemotron-3-nano-omni-30b ignores the cap); the cap always leaves 64 tokens of the request's output cap for the answer, and a larger N is lowered to fit. A request with a response_format thinks without a budget. {"exclude": true} leaves the reasoning text out of the result. A request's own chat_template_kwargs.enable_thinking or chat_template_kwargs.reasoning_effort wins over reasoning.

A file can be up to 1 GiB and 10,000,000 requests. The upload checks every line and refuses the file with the line number of the first problem. A line too long to hold its media inline is refused with a 413 that says how large media may be; send larger images, audio and video by URL. A repeated custom_id and a malformed message are found when the batch runs, and fail the batch.

Each organization may upload 2 files at a time. Past that, or while the service is busy with other uploads, an upload is answered 429 with a Retry-After header. An upload that sends nothing for 60 seconds, averages under 256 KiB/s after its first minute, or takes longer than 30 minutes is stopped with a 408; send it again.

Batch endpoints

POST /v1/files

Upload a file, as multipart form data with purpose=batch and the file in file.

GET /v1/files

List your files, newest first, with purpose, limit, order and after.

GET /v1/files/{file_id}

A file's details.

GET /v1/files/{file_id}/content

Download a file, including a batch's output and error files.

DELETE /v1/files/{file_id}

Not offered: answers 405 (file_deletion_not_supported). Files are deleted on the schedule in Batch results; to delete a batch's data sooner, delete its job in the console.

POST /v1/batches

Create a batch: input_file_id, endpoint (the url of the file's requests), completion_window ("24h") and optional metadata (up to 16 pairs, keys up to 64 and values up to 512 characters).

GET /v1/batches/{batch_id}

A batch's status, request counts and file ids, and usage: the input, cached input and output tokens of the requests finished so far (null until the first ones finish).

POST /v1/batches/{batch_id}/cancel

Cancel a batch. Requests that already finished are delivered and billed.

GET /v1/batches

List your batches, newest first, with limit (up to 100) and after.

A batch's anex_job_id is the same batch as a job in the console. If creating a batch answers with a 500, list your batches before you retry: the batch may exist, and a retry creates a second one.

Batch price and approval

A batch is quoted when it is created. The quote prices the output of every chat request at the largest output cap in the file (a request without a cap counts at its default), so the output part of a quote is the number of requests times that cap. The bill counts the tokens each request really used. The quote and the bill are never below the model's minimum job price.

A batch is approved as soon as it is priced when your credit balance or your organization's credit limit covers the quote, and then it starts. If it cannot be approved yet (the balance or the credit limit does not cover the quote, the organization is suspended, or the approval check is briefly unavailable), the batch stays in validating and approval is retried every few minutes. It starts once the cause is resolved, for example after a top-up, and expires without a charge if that has not happened 24 hours after it was created.

Batch status

validating

The file is being checked and priced, or the batch is waiting to be approved (see Batch price and approval).

in_progress

The requests are running.

finalizing

The output and error files are being written.

completed

Every request has a result in the output or the error file.

failed

The batch could not run; errors says why.

cancelling, cancelled

The batch was cancelled; requests that finished first are in its files.

expired

The batch had not finished 24 hours after it was created, so it was stopped. Requests that finished are in its files and are billed. A batch already in finalizing when its window closes completes.

Batch results

output_file_id holds the requests that succeeded and error_file_id the ones that failed, one line each in the result line format. Either is null when it would be empty. A chat result's message has the realtime answer's shape: its thinking text, when there is any, is in reasoning_content. Uploaded files are kept 7 days, and a batch's output and error files 30 days after the batch ends. Downloading a file after that answers 404 with a message that says it was deleted.

Batch status codes

Errors from these routes use the OpenAI shape, {"error": {"message", "type", "param", "code"}}.