Jev rate limits and context window

What the limits mean in practice

Two ceilings run at once: 250,000 tokens per second and 1,200 requests per minute. Convert both to a minute and they cross at a single number. 1,200 requests a minute against 15,000,000 tokens a minute is 12,500 input tokens per request. Below that, the request cap binds and the token budget goes unused. Above it, the token cap binds and you never reach 1,200 requests.

The caps are TypeSafe's published figures. The throughput column is our arithmetic on them, not a measurement.

Social content sits at the far left of that table. A post at X's 280-character limit is somewhere around 70 tokens, and a seven-question request over one is about 430 input tokens. At that size you are 29 times under the token ceiling and pinned to the request cap, which works out on the published caps at 1,200 posts a minute, 72,000 an hour, one post per request.

The fix for the request cap is questions, not requests. All the questions in one request are evaluated in the same parallel pass and share one copy of the state, so attaching an eighth question to an existing call costs you its own tokens and nothing else. TypeSafe's fan-out cookbook measured 13 questions in one request at 12.2x cheaper and 10.0x faster than asking them separately, with no change in the answers. If you are near 1,200 requests a minute, look for two calls about the same state that could be one.

What it costs at those volumes

Our arithmetic on TypeSafe's published $0.042 per 1M input tokens, at about 430 input tokens for a short post carrying seven questions. Output tokens are free, so they are not in the arithmetic.

The number worth carrying around: running out of rate limit costs about $1.30 an hour. Cost is not the constraint at social volume. The request cap is, and after that, whether the answers are right. That question is a different page.

// 429 and 529 are the only two worth retrying.
// 401 and 422 are your request, not the service.
const RETRYABLE = new Set([429, 529]);

async function ask(body: unknown, attempt = 0): Promise<Response> {
  const res = await fetch('https://api.typesafe.ai/v1/systemone', {
    method: 'POST',
    headers: {
      Authorization: `Bearer ${process.env.TYPESAFE_API_KEY}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify(body),
  });

  if (RETRYABLE.has(res.status) && attempt < 3) {
    await new Promise((r) => setTimeout(r, 200 * 2 ** attempt));
    return ask(body, attempt + 1);
  }

  return res;
}