Build the best classifier. Then prove it.

Unsure is three calls and a loop. decide turns a prompt and a JSON Schema into a filled object, with Jev's probability distribution for every field. feedbackrecords what the answer should have been, whether that came from a reviewer or from what a user actually chose downstream. prototype replays a rewritten prompt against every reviewed decision and scores it against the prompt in production.

Jev is a decision model: it answers booleans, choices and scores from text, and returns probabilities instead of tokens. It does not read images or write prose. Pipelines put the models that can on either side of it, so one call can take a screenshot in and hand a written reply back while the decision in the middle stays measurable.

1. Create an account. Every account starts with $1 of Jev credit.
2. Open Account & credits and create a named API key. The secret is shown once.
3. Make a decision.

from typing import Literal
from pydantic import BaseModel, Field
from unsure_client import Unsure


class Triage(BaseModel):
    urgent: bool = Field(description="Does this need a reply within the hour?")
    team: Literal["billing", "support", "sales"]


client = Unsure(api_key="YOUR_UNSURE_KEY")
decision = client.decide(
    prompt="I paid the invoice, but my account is still locked.",
    schema=Triage,
)
decision.data.team     # "support", typed as the Literal
decision.answers       # Jev's probabilities for each field
decision.latency_ms    # milliseconds, end to end

import { Unsure } from "@unsure.dev/client";

const client = new Unsure({ apiKey: process.env.VERDICT_API_KEY });
const decision = await client.decide({
  prompt: "I paid the invoice, but my account is still locked.",
  schema: {
    type: "object",
    properties: {
      urgent: { type: "boolean", description: "Does this need a reply within the hour?" },
      team: { type: "string", enum: ["billing", "support", "sales"] },
    },
    required: ["urgent", "team"],
    additionalProperties: false,
  },
});
decision.output.team; // "support"
decision.answers;     // Jev's probabilities for each field

curl https://app.unsure.dev/api/decisions \
  -H "Authorization: Bearer $VERDICT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "I paid the invoice, but my account is still locked.",
    "schema": {
      "type": "object",
      "properties": {
        "urgent": {"type": "boolean", "description": "Does this need a reply within the hour?"},
        "team": {"type": "string", "enum": ["billing", "support", "sales"]}
      },
      "required": ["urgent", "team"],
      "additionalProperties": false
    }
  }'

Every surface returns the same envelope:

{
  "id": "5f1c3b2e-…",
  "group_id": "9a2e77c0-…",
  "output": { "urgent": true, "team": "support", "decision_id": "5f1c3b2e-…" },
  "output_schema": { "type": "object", "properties": { "…": "…" } },
  "answers": {
    "urgent": { "type": "noul", "noul": 0.91 },
    "team": {
      "type": "choice",
      "choice": "support",
      "probabilities": { "billing": 0.31, "support": 0.66, "sales": 0.03 },
      "confidence": 0.66
    }
  },
  "model": "jev-1.13.0",
  "usage": { "input_tokens": 212, "output_tokens": 0 },
  "latency_ms": 118
}

output is your schema filled in, plus a server-added decision_id. answers is the part most SDKs throw away: the probability behind each field. group_id is the prompt group Unsure filed this call under, which is where review and replay happen.

Send an API key as a Bearer token on every request. Keys are owned by the account that created them and charge that account. Create, name and revoke them under Account & credits. Only a hash is stored; a key created after the workspace secret was set can be re-shown to a signed-in session, a revoked key cannot.

Authorization: Bearer $VERDICT_API_KEY

Both SDKs also support a password session: login holds the server's HttpOnly cookie and sends the X-Verdict-CSRF: 1 header the server requires on cookie-authenticated writes. Creating, listing and revoking keys needs a session, not a key. Cross-origin browser writes are rejected outright, so keep keys in server-side code.

A decision is one prompt, one object schema, one Jev call. Every leaf in the schema becomes a question; Jev answers all of them in a single forward pass, so a schema with twelve fields costs about the same time as one with one.

Unsure files each decision into a prompt group automatically by aligning it against recent prompts with the same schema and extracting the shared template and its {{input_1}} slots. You do not declare groups; you get them by keeping the fixed part of the prompt stable.

Jev has three native question types. Unsure maps JSON Schema leaves onto them and adds a boolean convenience.

{
  "type": "object",
  "description": "A support ticket, read as the customer wrote it.",
  "properties": {
    "team": {
      "type": "string",
      "enum": ["billing", "support", "sales"],
      "x-criteria": {
        "billing": "Invoices, charges, refunds, plan changes",
        "support": "Something is broken or the customer is stuck",
        "sales": "Pricing questions before a purchase"
      }
    },
    "urgent": {
      "type": "boolean",
      "description": "Does this need a reply within the hour?",
      "x-threshold": 0.7
    },
    "needs_human": {
      "type": "number", "format": "noul",
      "x-criteria": {
        "true": "Needs judgment, an apology, or an exception",
        "false": "A templated reply would fully answer it"
      }
    },
    "severity": {
      "type": "number", "format": "score",
      "x-criteria": ["Cosmetic", "Degraded, workaround exists", "Blocking, no workaround"]
    }
  },
  "required": ["team", "urgent", "needs_human", "severity"],
  "additionalProperties": false
}

The same schema, written as types. Both SDKs export Noul and Score; Python models are converted with model_json_schema(), and local $defs references are resolved.

from typing import Annotated, Literal
from pydantic import BaseModel, Field
from unsure_client import Noul, Score


class Ticket(BaseModel):
    team: Literal["billing", "support", "sales"]
    urgent: bool = Field(description="Does this need a reply within the hour?")
    needs_human: Noul = Field(description="Does this need a human?")
    severity: Annotated[
        float, Score("Cosmetic", "Degraded, workaround exists", "Blocking, no workaround")
    ]

import { Noul, Score } from "@unsure.dev/client";

const schema = {
  type: "object",
  properties: {
    team: { type: "string", enum: ["billing", "support", "sales"] },
    urgent: { type: "boolean", description: "Does this need a reply within the hour?" },
    needs_human: Noul,
    severity: Score("Cosmetic", "Degraded, workaround exists", "Blocking, no workaround"),
  },
  required: ["team", "urgent", "needs_human", "severity"],
  additionalProperties: false,
};

Extensions

• description on a field is the question. On the root, it is context for every question. Write them as questions a careful reviewer could answer.
• x-criteria holds rubrics. For Choice, a map from option to rubric. For Noul and booleans, a map with true and false keys. For Score, a required ordered list of 2 to 10 level descriptions. Rubrics can be strings, objects or arrays.
• x-threshold on a boolean moves the cut from 0.5. Use it when a false positive costs more than a false negative, or the other way round.
• x-instructions replaces the generated question for a field entirely, when the name and description are not enough.

Shape rules

• The root is an object. Nested objects are fine; nested fields are corrected with dot paths.
• Every property is required and objects are closed. Missing required and additionalProperties are normalized for you.
• Free strings, unbounded numbers, arrays, unions and optional fields are rejected. If a field cannot be a probability, it is not a decision. Use a pipeline decoder for prose.
• decision_id is reserved at every level. The server adds it after inference and returns the augmented schema as output_schema.

answers is the reason Unsure exists. Each field carries what Jev returned natively:

To rank decisions for review, Unsure reduces each field to a certainty between 0 and 1. For Choice and Score it is the confidence. For Noul and booleans it is the distance from the threshold, scaled by the room on that side: a Noul of 0.52 with a threshold of 0.5 is almost a coin flip, and scores near 0. A decision's certainty is its least certain field. The review queue is sorted by that number, so the first item you see is the one the model was least sure about.

Use the same signal in your own code. If you are gating an action on a decision, read the distribution and escalate below a threshold you choose, rather than trusting the label.

Review is where labels come from. A label is a correction, or an affirmation, of one field on one decision. It can come from a reviewer in the queue, or from your own system when the truth shows up later: the category a user picked, the team an agent reassigned to, the refund that was actually issued. Post it either way. The API calls this feedback; the UI calls it review.

# The customer picked a team in your UI: that is the label.
client.feedback(decision_id=decision.id, team="billing")

# Affirm an answer that was right, with a note for the next reviewer.
client.feedback(decision_id=decision.id, urgent=True, note="Outage, not a question.")

# Nested fields use dot paths.
client.feedback(decision_id=decision.id, **{"routing.team": "support"})

await client.feedback({ decision_id: decision.id, team: "billing" });
await client.feedback({
  decision_id: decision.id,
  urgent: true,
  note: "Outage, not a question.",
});
await client.feedback({ decision_id: decision.id, "routing.team": "support" });

curl https://app.unsure.dev/api/feedback \
  -H "Authorization: Bearer $VERDICT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"decision_id": "5f1c3b2e-…", "key": "team", "expected": "billing"}'

• Feedback is append-only. The latest label per decision and field is what replay uses; earlier ones stay in the history with their author.
• You can flip a boolean, affirm the selected option, choose any unselected enum option, or give a Noul or Score its target value. Unknown keys and out-of-schema values are rejected.
• Pipeline runs expose decision_ids. Feedback on those corrects the Jev step inside the run.

A prompt group is every decision made with the same schema and the same template. The group page shows the template with its input slots, every decision, the reviewed subset, and a history of replays. Replay is how you find out whether a rewritten prompt is better before it takes traffic.

group = client.group(decision.group_id)["group"]

# Keep the group's {{input_1}} slots; change the instructions around them.
run = client.prototype(
    decision.group_id,
    template=group["template"] + "\nA locked account after payment is a support issue, not billing.",
)

# Replays run in the background. Poll until status is no longer queued or running.
summary = client.prototype_run(run["id"])
summary["baseline_score"]  # the production prompt on the reviewed set
summary["score"]           # the candidate on the same examples
summary["results"]         # per-example before/after, with Jev's answers

const { group } = await client.group(decision.group_id);
const run = await client.prototype(decision.group_id, {
  template: group.template + "\nA locked account after payment is a support issue, not billing.",
  limit: null, // every reviewed decision, not just the latest 100
});
const summary = await client.prototypeRun(run.id);
console.log(summary.status, summary.baseline_score, summary.score);

How a replay is scored

Each reviewed example is rendered into the candidate template with its original inputs and sent to Jev. Agreement with the latest label is exact match for enums and booleans, 1 − |predicted − expected| for Noul, and 1 − |predicted − expected| / (levels − 1) for Score, averaged over labeled fields. The production prompt is scored on the same successful examples, so baseline_score and score are directly comparable. Failed examples are shown, not hidden.

Jev takes text and returns structured fields. A pipeline extends that in both directions. Encoders take an image or a video and hand Jev a text reading of it. Decoders take Jev's fields and write the richer string you need: a summary, a reply, a reason. The decision in the middle is still a Jev decision, still in the review queue, still replayable.

Define the record

A pipeline starts with two models: the input each record arrives with and the output you want back. Write them as TypeScript types, Pydantic models or dataclasses, or raw JSON Schema. Source is parsed, never executed; only the two resulting schemas are stored. Image and Video mark media inputs, and ? or | None marks an optional field.

JSON schemas use draft 2020-12 with objects, arrays, scalar constraints and local #/$defs/name references. Network references, regular expressions, schema composition and uniqueness checks are not supported. Schemas are limited to 100 KB, 12 levels and 1,024 expanded schema nodes.

type TicketInput = {
  message: string;
  receipt?: Image; // http(s) or data: URL at run time
};

type TicketOutput = {
  wants_refund: boolean;
  category: "billing" | "bug" | "question";
  summary: string;          // written by a text model
  refund_reason?: string;   // only when wants_refund is true
};

from typing import Literal
from pydantic import BaseModel

# Pasted into the pipeline editor. Source is parsed by name, never imported,
# so Image needs no import here.


class TicketInput(BaseModel):
    message: str
    receipt: Image | None = None


class TicketOutput(BaseModel):
    wants_refund: bool
    category: Literal["billing", "bug", "question"]
    summary: str
    refund_reason: str | None = None

Outputs are routed by type. Booleans, choices and scores go to the Jev Classifier. Free-text fields go to a Text LLM decoder. An image input makes a round trip through an Image LLM encoder on the way in; a video input through a Video LLM. On the canvas you can add steps, reconnect fields, write instructions per connection, and make a text field conditional on a classifier result: refund_reason only when wants_refund is true.

Encoder and decoder steps pick their model from OpenRouter's catalog, filtered to what the step needs to read, with prices per million tokens. The catalog is synced hourly.

Run it

result = client.run(
    pipeline="PIPELINE_ID",
    input={
        "message": "Charged twice for September, receipt attached.",
        "receipt": "https://example.com/receipts/1187.png",
    },
)
result["output"]        # the filled TicketOutput
result["readings"]      # what the image model read from each file
result["decision_ids"]  # Jev decisions inside the run, for feedback()
result["steps"]         # per-step model, tokens, latency

const result = await client.run({
  pipeline: "PIPELINE_ID",
  input: {
    message: "Charged twice for September, receipt attached.",
    receipt: "https://example.com/receipts/1187.png",
  },
});
console.log(result.output, result.readings, result.decision_ids, result.steps);

{
  "id": "c0ffee12-…",
  "pipeline_id": "…",
  "input": { "message": "Charged twice for September, receipt attached.", "receipt": "https://…" },
  "output": {
    "wants_refund": true,
    "category": "billing",
    "summary": "Customer was charged twice for September and wants one charge reversed.",
    "refund_reason": "Duplicate charge on 2026-09-03 for $48.00."
  },
  "readings": { "receipt": "Receipt #1187, 2026-09-03, two line items of $48.00 …" },
  "decision_ids": ["5f1c3b2e-…"],
  "steps": [
    { "node": "…", "type": "image_llm", "label": "Image LLM", "model": "…", "fields": ["receipt"], "skipped": [], "latency_ms": 1420 },
    { "node": "…", "type": "classifier", "label": "Classifier", "model": "jev-1.13.0", "fields": ["wants_refund", "category"], "skipped": [], "decision_id": "5f1c3b2e-…", "latency_ms": 121 },
    { "node": "…", "type": "text_llm", "label": "Text LLM", "model": "…", "fields": ["summary", "refund_reason"], "skipped": [], "latency_ms": 2210 }
  ],
  "latency_ms": 3790,
  "created_at": "2026-09-22T14:03:11Z"
}

A run executes in three phases. Encoders read every media field concurrently. Each Classifier step makes one Jev decision over the fields routed to it. Decoders then write their fields as constrained JSON with the decision in hand, skipping any field whose condition was not met. The run keeps the input, the output, every reading, and a per-step trace with model, tokens and latency.

If you already call Chat Completions with a json_schema response format, change the base URL and the model name. The same schema rules apply, and the response carries the decision ID and Jev's answers alongside the standard envelope.

from openai import OpenAI
from pydantic import BaseModel

class Triage(BaseModel):
    urgent: bool

client = OpenAI(base_url="https://app.unsure.dev/v1", api_key="YOUR_UNSURE_KEY")
response = client.chat.completions.parse(
    model="verdict-latest",
    messages=[{"role": "user", "content": "Our checkout is down."}],
    response_format=Triage,
)
output = response.choices[0].message.parsed
print(output)
decision_id = response.model_extra["decision_id"]  # keep this for corrections

The response adds decision_id, output_schema and answers at the top level, and the JSON content includes decision_id. The SDK builds the schema from your Pydantic model and returns its instance in message.parsed. Keep Pydantic's default extra-field handling; extra="forbid" rejects the added ID. Raw JSON Schema is still supported with .create(). Streaming, tools, temperature, image parts and the Responses API are rejected with a 4xx rather than silently ignored. Errors on /v1 use OpenAI's error shape.

Agents make classification decisions constantly: is this message spam, which tool handles this, did that step succeed, is this safe to auto-approve. Most of them are made by asking a chat model for a label and trusting the string that comes back. Unsure gives an agent a faster way to make those calls and, more importantly, a way to find out afterwards how often they were right.

The pattern

from unsure_client import Unsure

unsure = Unsure(api_key=UNSURE_KEY)

def handle(ticket):
    # 1. Decide. One call, several fields, a probability for each.
    d = unsure.decide(prompt=TEMPLATE.format(ticket=ticket.text), schema=Triage)

    # 2. Act on the answer, but escalate when Jev is not sure.
    if min_certainty(d) < 0.4:
        return ask_human(ticket, suggested=d.data)   # the review queue holds it too
    route(ticket, d.data.team)

    # 3. Later, when the truth shows up, record it against the decision.
    # A reassignment by an agent, a user-picked category, a closed ticket:
    # any of these is a label.
    ticket.on_reassigned(lambda team: unsure.feedback(decision_id=d.id, team=team))

• Decide inline. A decision is a single round trip in the low hundreds of milliseconds, so it can sit in the request path or inside a tool call. Bundle every question about one input into one schema; a call with twenty fields is one forward pass.
• Read the distribution, not just the label. answers tells the agent when to act and when to ask. A Noul near the threshold or a Choice with two close options is a reason to escalate to a human, retry with more context, or take the reversible path.
• Keep the decision ID. Store it on the ticket, the row, the trace. The truth usually arrives later, from a different part of the system, and the ID is how it finds its way back.
• Post feedback from outcomes. A user overriding the agent's pick, a tool call that failed after the agent judged it safe, a reviewer changing a category: each is a label. Send it with feedback and the review queue empties itself.
• Improve with evidence. When the agent's prompt changes, replay it against the reviewed set. Ship when the score goes up. This is the same discipline as a test suite, for the fuzzy part of the agent.
• Do not retry blindly. The SDKs never retry on their own, because a retry creates a second decision and a second charge. Treat 5xx as a signal to fall back, and 429 as a reason to wait.

Instructions for your agent

Paste this into the agent's system prompt, tool description, or repository instructions.

## Classifying with Unsure

- Use Unsure for any yes/no, pick-one, or rate-it question inside a task.
  Do not ask a chat model for a label when a Jev decision will do; it is faster and returns probabilities.
- Put every question about the same input into ONE schema and ONE decide call (up to 100 fields).
- Write field descriptions as questions. Put rubrics in x-criteria. Keep the prompt template stable
  and put the variable text at the end so calls group together for replay.
- Keep decision_id from every response. When the outcome becomes known (a user picked, a tool
  succeeded, a reviewer changed it) send POST /api/feedback with that id, the field, and the value.
- Read answers.<field>: a noul near 0.5, a choice confidence under ~0.6, or a score with a flat
  distribution means the model is unsure. Escalate or ask instead of acting.
- For images, video, or prose outputs, run a saved pipeline (POST /api/pipelines/{id}/runs)
  instead of a raw decision. Feedback goes to the run's decision_ids.
- Never retry a decide call blindly: each call is billed and creates a new decision.
- Machine-readable reference: https://app.unsure.dev/openapi.json (needs a key).

Prompt templates that group well

Grouping is lexical. It aligns a new prompt against recent prompts with the same schema, extracts the shared text, and treats what differs as input slots. To get one group per use case, keep the instructions fixed and put the variable text at the end, ideally under a heading. Prompts over about 1,000 tokens of shared text, or with the variable text scattered through the instructions, may land in separate groups.

Where the reference lives

Every route is described in the generated OpenAPI document at https://app.unsure.dev/openapi.json, and the interactive version at https://app.unsure.dev/docs. Both require a session or an API key. An agent with a key can read the document and build requests from it directly.

Unsure is deliberately narrow about what a decision is, and broad about how you get to one. The rule: if a field can be answered with a probability, Jev answers it. Everything else goes through a pipeline step, or is not supported yet.

Both clients cover the whole API. Python methods are synchronous and use snake_case; JavaScript methods return promises and use camelCase. Request and response fields keep the API's snake_case spelling in both.

• Python. Unsure(base_url=, api_key=) defaults to https://app.unsure.dev. Depends only on httpx and Pydantic. Errors raise httpx.HTTPStatusError. Call close() on a long-lived client or use it as a context manager.
• JavaScript. new Unsure({ apiKey, baseUrl?, timeoutMs?, fetch? }). ESM and CommonJS. Errors throw UnsureError with status, parsed body and the response. Generics on decide<T> and run<T> type the output without runtime validation.

Everything under /api returns plain JSON and errors as { "detail": "…" }. Everything under /v1 follows OpenAI's shapes.

Every account starts with $1 of credit for Jev. Buy more in packs of $20, $40, $60, $80 or $100 under Account & credits. Each purchase carries a 5% platform fee, itemized as a second line at checkout, so $20 of credit is charged as $21. Credit has no expiry date. The first confirmed purchase unlocks encoder and decoder steps on OpenRouter.

Usage is metered in billionths of a dollar, rounded up at most one billionth per call. One final call may take the balance below zero; after that, requests return 402 until it is topped up. GET /api/billing returns the balance.

Oversized requests are rejected with a 400 or 422, never truncated. These are Unsure's limits, not Jev's.

Native routes return { "detail": "message" }. Routes under /v1 return { "error": { "message", "type", "param", "code" } } with type set to invalid_request_error below 500 and server_error above.

Jev answers every field in one forward pass and returns probabilities directly, so there is no token generation to wait for. Measured 2026-09-21 from a laptop: 20 sequential calls to jev-1.13.0, each with 3 questions (Noul, Choice, Score), wall clock including the network.

Pipeline runs add the encoder and decoder models on top, typically one to several seconds each depending on the model chosen. The per-step trace on every run shows where the time went.

Unsure stores what the loop needs: the prompt, the schema, the output, Jev's answers, every label with its author, and for pipelines the input record, media readings and step traces. Prompts go to Jev for inference; encoder and decoder inputs go to the OpenRouter model you chose. The privacy policy lists every provider and what each receives.