Batch inference · open models · estimates up front
Batch inference you don't have to babysit.
Batch inference for open models like GLM, Qwen and Gemma. All modalities including audio, video, embeddings. Your job gets its own GPUs, so it isn't throttled or queued behind others, thousands of tokens per second, parallelized across the fleet. Anex prices the job before it runs, and an approved job never costs more than its estimate. 10–20× below Google and Anthropic.
We're in private beta - email hello@anex.sh to ask for access.
Run massive jobs faster
Per-token APIs are built for chat traffic: every token carries a margin, and big jobs pay it millions of times over. anex.sh sizes the whole job against the compute it actually needs instead.
Thousands of tokens per second
Every job runs on GPUs dedicated to it, so it moves at the speed of the hardware instead of a shared queue. No rate limits to spread a large dataset over days.
Capped estimate with no dynamic pricing
We tokenize your dataset and estimate input, output, images, and audio/video minutes in credits before you commit. Per token prices are fixed, no dynamic pricing causing surprising bills.
No hidden quantization
We never serve a quantized model without clearly specifying it.
Pricing
You buy prepaid credits (1 credit = 1 US cent, USD) and jobs consume them. Estimates are free, and the credit amount shown at approval is a hard cap - an approved job never consumes more, and unused credits stay on your balance.
Text output models
| model | input modality | input / 1M tokens | cached input / 1M | output / 1M tokens | price per image |
|---|---|---|---|---|---|
| qwen3.8-27b | text + image + video | $0.18 | $0.045 | $0.90 | $0.0001 |
| glm-5.2 | text | $0.70 | $0.175 | $2.20 | - |
| qwen3.6-27b | text + image | $0.20 | $0.05 | $1.00 | $0.002 |
| gemma4-31b | text + image | $0.20 | $0.05 | $1.00 | $0.0003 |
| qwen3-14b | text | $0.025 | $0.006 | $0.125 | - |
| qwen3-omni-30b | text | $0.12 | $0.03 | $0.60 | - |
| qwen3-omni-30b | image + audio + video | $0.20 | $0.05 | $1.00 | $0.0003 |
Embedding output models
| model | embeds | text rows / 1M input tokens | image rows / 1M | video rows / 1M |
|---|---|---|---|---|
| qwen3-vl-embedding-8b | text + image + video | $0.02 | $0.03 | $0.10 |
| qwen3-vl-embedding-2b | text + image + video | $0.005 | $0.01 | $0.05 |
| qwen3-embedding-8b | text | $0.02 | - | - |
Realtime embedding requests (see realtime embeddings) bill at the listed text input rate above plus 10 percent, from the same credit balance as batch jobs. Available in realtime today: qwen3-embedding-8b and qwen3-vl-embedding-2b (text queries only - image and video embedding stays on the batch API).
What jobs actually cost
| example job | model | assumed per row | estimate |
|---|---|---|---|
| Classify 1M support tickets | qwen3-14b | ≈300 tokens in, 10 out | ≈ $9 |
| Caption 100,000 photos | gemma4-31b | 1 image, ≈200 tokens in, 60 out | ≈ $40 |
| Transcribe 1,000 hours of audio | qwen3-omni-30b | audio at ≈13 tokens/s, ≈12k transcript tokens out per hour | ≈ $25 |
| Describe 10,000 one-minute videos | qwen3-omni-30b | video at ≈72 tokens/s, ≈500 tokens out | ≈ $17 |
Your job gets its own estimate based on current prices; these are representative examples. What the columns mean and what else shapes a bill is on the pricing page. The exact version (weights build) and quantization behind every model name here are on the model list.
The batch lifecycle
You can use our OpenAI compatible API or submit the jobs through your user dashboard.
Submit a dataset
Point the API at a Parquet or JSONL file - upload it to us, or have us ingest it from any HTTPS URL.
Get the estimate
We tokenize your rows and measure real output length, then hold the job here with a priced estimate: input, output, and media credits. Nothing has run yet.
Approve it
Nothing runs until you (or your auto-approve rule) accept the estimate. This is the spend gate.
Collect results
The job runs on the lowest-cost hardware that fits it, and the merged result file, with per-row usage, is ready to download. A signed webhook tells you it's ready.
You can see all job statuses in the job lifecycle reference.
Submit, review the estimate, approve
Mint an API key in the dashboard. Estimates are free; approval is what starts paid compute.
# submit a batch job against glm-5.2
curl -X POST https://api.anex.sh/batch/v1/jobs \
-H "Authorization: Bearer $ANEX_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"input_s3_uri": "s3://.../in/ingest/.../input.jsonl",
"model": "glm-5.2",
"max_tokens": 512,
"response_format": {"type": "json_object"}
}'
# poll until "status": "cost_ready", review est_cost_cents, then approve
curl -X POST https://api.anex.sh/batch/v1/jobs/<job-id>/approve \
-H "Authorization: Bearer $ANEX_API_KEY"About us
We are a small team of AI and infrastructure engineers (Tonda, Radek, Jan) based in Prague, with 30+ years of combined experience building production ML systems. We built anex.sh because we kept paying API prices for dataset work, and knew from running our own GPUs exactly how much cheaper the same tokens could be.
The GPU fleet itself runs on ANEX, our open-source engine for integrating GPU containers into Kubernetes clusters - the project this domain started as, and still free and open on GitHub.
When you write to support@anex.sh, you talk to the people who wrote the code.
FAQ
How are you so cheap?
We automatically analyze each job and then select the best way to run it. We optimize model serving extensively to achieve high GPU utilization, which makes our low prices possible.
How quickly do you provide the results?
We offer a standard 24h SLA, but we are able to deliver the results much faster, even in less than 1h.
If you have specific needs for delivery speed guarantees per job, we are happy to discuss it - just message us at support@anex.sh.
How does batch differ from a standard real-time API?
Real-time APIs are built for use cases where you need the response immediately because the user is waiting for it: chatbots, copilots, interactive AI applications.
At the same time, they are expensive and have strict rate limits that slow down the processing of large amounts of data. This is exactly where a batch API becomes useful: it is both cheaper and faster at scale.
What are the use cases for the batch API?
The batch API is optimized to process large amounts of data efficiently. If you have any case where you would be repeatedly calling a standard real-time API for a list of inputs (text, audio, video), then the batch API is the perfect fit.
Some examples:
- Categorizing and tagging millions of products.
- Generating product descriptions.
- Summarizing news articles, videos, or audio.
- Call transcription or categorization for sales and customer support teams.
- Vectorizing product data to implement AI search in ecommerce.
- Generating video descriptions or vectorizing video segments for an AI video search.
- Evaluating your product on a large dataset or benchmark.
Can you keep our data inside the EEA?
Yes, on request. We can run a job so that everything it touches stays inside the EEA: the dataset you upload, the pre-processing, the inference itself, and the stored input and output.
This is not the default for every job, so agree it with us before you send us data. Write to support@anex.sh with what you need and we will confirm in writing what applies to your jobs.
Know the price. Then run it.
Create an organization, invite your team, and submit your first batch job. Estimates are free.
Create accountWe are onboarding organizations personally during the beta. Email hello@anex.sh for access.