anex.sh docs: output format, structured output, thinking

The result file

A job produces one result file, in the same format as its input: a JSONL job returns JSONL, a parquet job returns parquet. Rows keep the identity they came in with, so the result can be joined back onto your data.

A job that stopped early but had already finished some rows delivers those rows in exactly the same file shape - just fewer rows.

JSONL output

One line per row, in the OpenAI batch output envelope, keyed by the custom_id you supplied:

{"id":"batch_req_row-1","custom_id":"row-1",
 "response":{"status_code":200,"request_id":"req_row-1",
             "body":{ ...the chat completion... }},
 "error":null}

response.body: The completion object, as the model server returned it. The assistant's text is at choices[0].message.content.
response.status_code: 200 when the row produced a completion. On a failed row, the status the serving layer returned for it - for example 400 when the model rejected your request. It is null only when no status was reported at all, which means "unknown", not "failed": read error to decide whether a row succeeded.
error: null on a row that produced a completion; the error object otherwise, and then response.body is null.
error.message: What went wrong, in the words of the layer that raised it.
error.type: The class of failure. media_fetch_error means we could not download media your row referenced by URL; see media referenced by URL below.
error.code: A machine-readable code when one is available, otherwise null. On a media_fetch_error this is the HTTP status the remote host returned - 429 if it rate limited us, 404 if the file was not there.
error.param: The offending field, when the failure points at one; otherwise null.

A failed row is still a delivered row: it occupies its line in the output file, keyed by its custom_id, so the output always lines up with the input.

Media referenced by URL

When a row points at an image, audio or video file by URL, we download that file from wherever it is hosted. A row whose media cannot be downloaded fails on its own, with error.type of media_fetch_error and the remote host's HTTP status in error.code; the rest of the job runs normally.

A batch job is fast because it runs many rows at once, which means many downloads at once. If the host serving your media rate limits us, you will see rows failing with error.code of 429. Two things help:

Send the media inside the input file. Embedding each file as base64 data in the row removes the download entirely, so nothing can rate limit it. This is the reliable fix, and the one to reach for if you control how the input file is produced.
Allow our traffic through. If the media lives on a host you control, relaxing the limit for us avoids the problem without changing your input file. We are working on serving these downloads from a single fixed address you can add to an allow-list.

Downloads are cached per machine while your job runs, so the same URL repeated across many rows is fetched once rather than once per row. Note that a slow or heavily throttled host makes a job take longer, and a batch job is priced on the machine time it occupies.

Parquet output

The columns of your input file, plus:

id: The row identity. Your own id column if the input had one; otherwise a row_index column is added in its place.
output: The answer text - the assistant's message content. If a completion comes back in a shape that has no such field, the whole completion object is written as JSON instead.
error_type: Null on a row that succeeded. Otherwise a short category: a marker for a generation that stopped on the output-token cap, or the class of the failure for a row that did not produce an answer. The same categories the JSONL output uses in error.type, including media_fetch_error.

The delivered parquet carries error_type only - the error message text is dropped when the per-row results are merged into the final file. Use the job's failure_reason and its row counts to tell what went wrong at the job level.

Embedding output

An embedding job (see embedding jobs) uses the same two result shapes as above, with a vector in place of the text answer.

JSONL - the same output envelope as JSONL output, whose response.body is an embeddings response instead of a chat completion:

{"data":[{"embedding":[0.013,-0.048,"..."]}],
 "usage":{"prompt_tokens":8,"total_tokens":8}}

The vector is at data[0].embedding.

Parquet - the same input columns and id as parquet output, but an embedding column (a list of 32-bit floats, one row's vector per cell) in place of output - an embedding job's result file never has an output column. error_type is still there for a row that failed, same as a chat job's.

Every vector comes back L2 normalized to unit length, so cosine similarity and dot product rank identically and no further normalization is needed. Embedding the same text twice can still return numbers that differ in their last digits, because floating point results depend on the GPU and on the batch the request landed in; the cosine between two such vectors should be very close to 1.0. See how embedding inputs are processed for what reaches the model and how the vector is pooled.

Structured output

A job-level response_format constrains every row's answer to JSON. It works for both input formats. Two shapes are accepted, matching the OpenAI API:

{"type":"json_object"}

{"type":"json_schema",
 "json_schema":{"name":"person",
                "schema":{"type":"object",
                          "properties":{"name":{"type":"string"},
                                        "age":{"type":"integer"}},
                          "required":["name","age"]}}}

Note the nesting: for json_schema the schema itself sits at json_schema.schema. The shape is checked when the job is submitted - a wrong type, or a json_schema without that nested schema object, is rejected before the job is created. The check is structural: the schema's own contents are not validated against the JSON Schema spec, so a schema that is well-formed but unsatisfiable still reaches the model.

A JSONL row may carry its own response_format in its body. It then replaces the job-level one for that row whole - it is all or nothing, never a field-by-field merge of the two. A row that carries none uses the job-level format. Parquet rows have no per-row override.

Enforcement happens during generation, so a constrained row can still fail to satisfy the format. Those rows are counted as format_error_rows on the job, separately from error_rows. When output length is estimated automatically, the sample is generated under the same response_format, so the estimate reflects the constraint.

Thinking and reasoning

Some models reason before they answer. That reasoning consumes output tokens, so a small output cap can be spent entirely on thinking and leave the answer empty.

The job-level switch is reasoning (true or false) on submit. Leave it out to keep the model's own default. In the console it is a thinking / reasoning checkbox, shown next to the model picker for the models that support it.

A JSONL row can set it per row with chat_template_kwargs.enable_thinking in its body. The row's own setting always wins; the job-level value only fills in for rows that say nothing, and any other chat_template_kwargs keys on the row are left alone. Parquet rows have no per-row setting, so the job-level value applies to all of them: on a reasoning model, the estimated reasoning_budget below keeps thinking from swallowing the whole cap by default; set reasoning: false at submit if you want no thinking at all.

Left unset, reasoning_budget is estimated from the same sample as the output cap: the thinking length every row is expected to stay under with 95% confidence. Set it yourself on submit (or the thinking-budget field in the console), or send 0 (the console's no-thinking-cap checkbox) for no cap. A row whose thinking reaches the budget has its trace closed by the server: the trace's last line is a short notice that the budget is exhausted and the model is answering with what it has, and the answer follows as usual. In the result you see that notice at the end of the message's reasoning field; content holds the answer. Budget tokens and the notice are ordinary output tokens for max_tokens and billing.

{"custom_id":"row-1","method":"POST","url":"/v1/chat/completions",
 "body":{"messages":[{"role":"user","content":"Say hello in French."}],
         "chat_template_kwargs":{"enable_thinking":false}}}

When output length is estimated automatically, the sample is generated with the same thinking setting the real run will use, so the estimate accounts for the reasoning tokens.