Jev guide

Jev sentiment analysisuse Score, not Choice

Sentiment is an ordered rubric. Hostile sits below negative, which sits below neutral, which sits below positive. That ordering is the whole signal, and a Choice throws it away. A Jev Score keeps it and returns a probability-weighted float, which means you can rank by it, average it over a week, and move the line between "ignore" and "answer this today" by editing one number in your own code.

Last updated: September 2026

Jev decides. An LLM writes. OpenTweet publishes.

The short version

A Choice hands you the word "negative". A Score hands you 1.4, and 1.4 versus 1.9 is the difference between a customer you are about to lose and a customer having a bad morning. One of those numbers is actionable. The word is not.

The same mention, two primitives

Both are valid Jev questions. Only one of them survives contact with the second week of running it in production, when somebody asks whether things are getting worse.

	Choice over 5 labels	Score over 5 ordered levels
What comes back	One label, for example "negative"	A float, for example 2.968, plus the full distribution
Ordering	Discarded. The options do not know they are a ladder.	Preserved. Level 4 is more than level 3 by construction.
Moving the line later	Rewrite the criteria and re-validate	Change one number in your own code
Ranking a day of mentions	Impossible within a label	Sort by score
Averaging over time	Only as counts per bucket	A real mean you can plot
Borderline inputs	Probability splits between two labels and confidence drops	The float lands between the two levels and stays informative

The last row is the one that decides it. Real mention streams are mostly borderline. A Choice answers a borderline mention by splitting probability across two adjacent labels, which drags confidence down and makes the gate you built reject exactly the middle of your distribution. A Score answers the same mention with a float that lands between the two levels and carries the ambiguity forward instead of destroying it.

A five-level rubric, written properly

Score takes an ordered array of 2 to 10 level descriptions. The descriptions are the classifier. Most bad sentiment scores are bad rubrics, not bad models.

sentiment-request

{
  "state": {
    "mention_text": "Used this for a month. Scheduling works, the analytics are genuinely useful, and support answered on a Sunday. Switching the whole team over.",
    "platform": "X",
    "is_reply": false
  },
  "model": "jev-1.13.0",
  "questions": {
    "sentiment": {
      "type": "score",
      "instructions": "How favourably this mention speaks about the brand it names",
      "criteria": [
        "Hostile. Attacks the brand or a person at it. Accusations of fraud, slurs, or telling others to leave.",
        "Negative. Names a specific problem or disappointment, without attacking anyone.",
        "Neutral. A question, a factual statement, or a mention that carries no evaluation either way.",
        "Positive. Says something worked, or recommends it, in ordinary language.",
        "Enthusiastic. Unprompted advocacy. Tells other people to use it, or calls it the best option available."
      ]
    }
  }
}

Rules the rubric above follows

One dimension per Score. If a level mixes "angry" with "at risk of churning", you have two questions wearing one coat.
Monotone wording. Higher level means more of the thing, always. TypeSafe documents contradictory instructions and criteria as something that degrades answers.
Put the boundary case in the level, not the instructions. "Names a specific problem" is a boundary. "Is negative" is not.
Between 2 and 10 levels. Five is usually right: two bad, one neutral, two good.
Do not label the levels with numbers you plan to reuse elsewhere. Scores from different questions are not on a shared scale.

What comes back

Four fields, and the one you probably ignore first is the one worth keeping. An illustrative response: the field shapes are the documented ones, the values are written to show them.

response

{
  "model": "jev-1.13.0",
  "answers": {
    "sentiment": {
      "type": "score",
      "score": 2.968,
      "legend": {
        "0": "Hostile. Attacks the brand or a person at it...",
        "1": "Negative. Names a specific problem or disappointment...",
        "2": "Neutral. A question, a factual statement...",
        "3": "Positive. Says something worked, or recommends it...",
        "4": "Enthusiastic. Unprompted advocacy..."
      },
      "probabilities": { "0": 0.002, "1": 0.028, "2": 0.150, "3": 0.640, "4": 0.180 },
      "confidence": 0.714
    }
  },
  "usage": { "input_tokens": 244, "output_tokens": 16 }
}

Check the arithmetic yourself

score is the probability-weighted mean of the level indices:0(0.002) + 1(0.028) + 2(0.150) + 3(0.640) + 4(0.180) = 2.968So 2.968 is not "positive, 97 percent sure". It is the centre of mass of a distribution that put 64 percent on Positive and 18 percent on Enthusiastic. Store probabilities alongside the float. It is the only record of what the model nearly said, and you will want it the first time somebody disputes a score.

The float is for ranking, not measuring

TypeSafe's Score documentation is explicit that the values are not linearly interpolatable in meaning. 2.968 is not "96.8 percent of the way from Neutral to Positive", and the gap between 1 and 2 is not guaranteed to be the same size as the gap between 3 and 4. Use it to sort, to threshold and to watch a trend. Do not multiply it by a weight and call the result a percentage.

Thresholding, which is the entire point

A label can only be compared for equality. A float can be compared against a line, and the line lives in your code where you can move it on a Tuesday afternoon without touching the model.

route.ts

const HOSTILE = 0.6;
const NEGATIVE = 1.8;
const POSITIVE = 3.2;
const ADVOCATE = 3.6;
const MIN_CONFIDENCE = 0.55;

type Route = 'escalate' | 'reply_today' | 'ignore' | 'thank' | 'ask_for_testimonial' | 'human_review';

function route(score: number, confidence: number): Route {
  if (confidence < MIN_CONFIDENCE) return 'human_review';
  if (score < HOSTILE) return 'escalate';
  if (score < NEGATIVE) return 'reply_today';
  if (score < POSITIVE) return 'ignore';
  if (score < ADVOCATE) return 'thank';
  return 'ask_for_testimonial';
}

Different actions, different lines. Escalating to a founder is expensive to get wrong, so it sits behind both a low score and a decent confidence. Ignoring something is cheap to get wrong, so it does not need a gate at all.
MIN_CONFIDENCE is not a universal number. TypeSafe's own confidence documentation refuses to give one and says to start conservative, test on your own data and adjust. 0.55 here is a starting point to replace, not a recommendation to copy.
Pin the model version. jev-1.13.0 rather than jev-latest, so that the thresholds you calibrated do not shift underneath you when the alias moves.

Aspect sentiment costs you nothing extra

One blended sentiment number tells you a mention was negative. Three aspect Scores tell you it was negative about pricing and positive about support, which is the difference between a discount and a bug fix. All of them go in the same request.

score-mention.ts

import { TypeSafeClient, score } from '@typesafe-ai/sdk';

const client = new TypeSafeClient({ timeout: 4000 });

const LEVELS = [
  'Hostile about it.',
  'Complains about it specifically.',
  'Mentions it without judgement, or does not mention it at all.',
  'Says it works.',
  'Singles it out as the reason to use the product.',
];

const QUESTIONS = {
  overall: score('How favourably this mention speaks about the brand it names', [
    'Hostile. Attacks the brand or a person at it.',
    'Negative. Names a specific problem or disappointment.',
    'Neutral. Carries no evaluation either way.',
    'Positive. Says something worked, or recommends it.',
    'Enthusiastic. Unprompted advocacy.',
  ]),
  pricing: score('How the mention speaks about price or value for money', LEVELS),
  reliability: score('How the mention speaks about uptime, bugs, or things breaking', LEVELS),
  support: score('How the mention speaks about the support the author received', LEVELS),
};

export async function scoreMention(text: string, platform: string) {
  const { answers, usage } = await client.systemOne({
    state: { mention_text: text, platform },
    questions: QUESTIONS,
  });

  return {
    overall: answers.overall.score,
    confidence: answers.overall.confidence,
    distribution: answers.overall.probabilities,
    aspects: {
      pricing: answers.pricing.score,
      reliability: answers.reliability.score,
      support: answers.support.score,
    },
    inputTokens: usage.input_tokens,
  };
}

Jev evaluates all declared questions in one parallel pass, so four Scores cost one round trip and one copy of the state rather than four of each. TypeSafe calls this fan-out, and its own cookbook reports a 13-question version of it at 12.2x cheaper and 10.0x faster than asking one question at a time.

Scores from different questions do not share a scale

A 3.1 on pricing and a 3.1 on support are not comparable magnitudes, and TypeSafe documents that no structural invariants hold across separate answers. Compare each aspect against its own history, not against its siblings.

Languages, honestly

There is less published here than you want, so here is exactly what exists.

TypeSafe's model documentation states that English is primary, and that other languages, including CJK, are handled but "not equally well". That is the whole of the public record. No per-language accuracy figures, no supported-language list, and no independent multilingual evaluation has been published as of September 2026.

So the practical position, rather than a claim:

Keep the rubric in English. The levels are instructions to the model, not content shown to a user, and English is the documented primary.
Tag the language in state. Detect it in code, put it on the state object, and store it on the result so you can slice accuracy by language later.
Calibrate per language. Label a couple of hundred mentions in each language you care about and set a separate confidence threshold for each. Assuming one threshold transfers is the failure mode here.

Cost against running sentiment on an LLM

The reason to move sentiment off an LLM is not that the LLM is bad at it. It is that sentiment has to run on every mention, and cost per call is what decides whether "every mention" is affordable.

	Jev	GPT-5.6 Terra
Price per 1M input tokens	$0.042	$2.00
Price per 1M output tokens	Not metered	$12.00
100,000 mentions at ~250 input tokens	$1.05	about $74.00 with a short label generated per mention
Cost per case, TypeSafe workflow eval	$0.0004	$0.0304
Latency per case, same eval	0.4s	10.1s
Accuracy, same eval	67.8%	67.9%

Token prices for Jev from docs.typesafe.ai; the Terra prices are as stated by The Register. The last three rows are TypeSafe's own workflow eval at evals.typesafe.ai, which is a general workflow benchmark and not a sentiment benchmark. The 100,000-mention row is arithmetic on list prices, September 2026.

Read the accuracy row carefully

67.8% against 67.9% is a tie on TypeSafe's own evaluation, not a win, and on that same table GPT-6 Sol (74.1%) and Claude Opus 5 (73.1%) both score higher than Jev. The reference labels are the average of two other frontier models rather than human judgements, which TypeSafe states plainly. The case for Jev on sentiment is cost and latency at comparable accuracy, and anybody telling you it is more accurate than a frontier LLM is reading the table wrong.

The checkable version of this arithmetic is public: the free post scorer runs four Scores and three Nouls on a draft. No signup, so you can run the arithmetic against your own token counts.

Where the score goes next

A number in a table is not a product. Two things are worth doing with it.

Watch the trend, not the mention

A single 0.4 is a bad day for one person. Forty of them in an hour is a different event. Brand crisis detection covers the baseline and spike method that turns these scores into an alert worth waking up for.

Answer the ones that earned it

Jev cannot write the response. An LLM or a person writes it, and OpenTweet publishes it to X, Bluesky and LinkedIn in one call, while a one-to-one reply goes through the browser extension into X's own composer. Replies is that feature, and the end-to-end how-to has the working code.

Frequently asked questions

Should I use Choice or Score for sentiment analysis with Jev?

Score. Sentiment is an ordered rubric: hostile sits below negative, which sits below neutral, which sits below positive. A Choice treats those four as unrelated competing options and throws the ordering away. A Score keeps it and returns a probability-weighted float between the level indices, which you can rank by, average over time, and move a threshold on without touching the question.

What does Jev return for a sentiment Score?

Four fields. score is the probability-weighted mean of the level indices. legend maps each index back to the level text you wrote. probabilities gives the full distribution across your levels. confidence collapses that distribution into one number from 0 to 1. The score is always inside the range you declared, because Jev cannot answer outside the space you gave it.

Does Jev support sentiment analysis in other languages?

TypeSafe’s model documentation says English is primary and that other languages, including CJK, are handled but not equally well. It publishes no per-language accuracy numbers, so there is nothing to quote beyond that. In practice: keep the rubric text in English, label a sample per language yourself, and set a separate confidence threshold for each language rather than assuming one number transfers.

How much cheaper is Jev than an LLM for sentiment?

At published list prices, roughly 70x on a 100,000-mention run. TypeSafe charges $0.042 per 1M input tokens and does not meter output, so 100,000 mentions at 250 input tokens each is $1.05. The Register lists GPT-5.6 Terra at $2.00 per 1M input and $12.00 per 1M output, which puts the same run near $74 once a short label is generated per mention. That is arithmetic on list prices, not a benchmark.

Can Jev explain why it scored a mention as negative?

No. Jev cannot generate text at all, so there is no rationale field and there never will be. What you get instead is the distribution: which levels it considered and how much weight each one carried. If you need a written explanation, send only the small fraction of mentions that matter to an LLM, which is far cheaper than sending all of them.

Is a Jev sentiment score accurate enough to act on?

Nobody has published an independent sentiment benchmark for it, so the honest answer is that you have to measure it on your own data. TypeSafe’s own workflow eval, published at evals.typesafe.ai, puts Jev at 67.8% against GPT-5.6 Terra’s 67.9%, with GPT-6 Sol at 74.1% and Claude Opus 5 at 73.1% both ahead, and the reference labels in that eval are the average of two other models rather than human judgements. It is a general workflow benchmark, not a sentiment benchmark. Score a few hundred mentions by hand, plot confidence against agreement, and set the automatic-action threshold from that curve.

Keep exploring

The primitives, the thresholds, and what to do once the scores start arriving.

Jev, explained

What a System One model is, what Jev can and cannot do, and where the numbers come from.

Jev classification, worked end to end

The same task solved with Choice, Score and Noul, with the exact request and response JSON.

Choice, Score and Noul explained

What each primitive accepts, what it returns, and the limits on each.

Brand crisis detection with Jev

Score every mention, hold a rolling baseline, and alert on the spike rather than any one mention.

Setting a confidence threshold

Where to gate an automatic action, and how to find the number from your own labelled data.

Will it go viral, a free Jev scorer

Seven Jev questions on your draft. Free, no signup.

Score every mention. Publish the ones that deserve an answer.

Jev grades the inbound. OpenTweet ships the response to X, Bluesky and LinkedIn through one REST call. Plans start at $11.99 a month with the API and the MCP server on every one.

7-day free trial
No X developer account needed
Cancel anytime