Jev classifies. Unsure closes the loop.

Feedback loops for a better classifier.

You can't improve a classifier without knowing where it goes wrong. Follow one ticket through a Jev decision, a human correction, and a prompt change measured against the reviewed set.

1. 01

   Classify on Jev

   A customer has paid, but their account is still locked. Jev classifies the ticket as urgent and routes it to billing. Its confidence in that team is only 54%.

   Three fields round-trip in 111 ms median from a laptop. Unsure records every answer and its confidence under one decision_id, so this uncertain route can be reviewed.

   class Triage(BaseModel):
       urgent: bool
       team: Literal["billing", "support", "sales"]
       severity: Annotated[float, Score("Low", "Mid", "High")]
   
   decision = client.decide(
       prompt=(
           "Payment questions go to billing. "
           f"Ticket: {ticket.text}"
       ),
       schema=Triage,
   )
   ticket.decision_id = decision.id

   inbox · support@example.comticket #1042

   MR

   Maya Ruiz <maya@northwind.co>

   9:41 AM

   to support@example.com

   Paid, but still locked out

   Hi there,

   I paid the invoice, but my account is still locked. The receipt came through an hour ago and I have a demo at noon.

   — Maya

   client.decide · 111 ms

   decision3 fields · classified

   • urgenttrue0.92
   • teambilling0.54
     Review the team. Billing is the least certain answer, at 54% confidence.
   • severity1.7 / 20.85
2. 02

   Correct the route

   The review queue brings the least certain decisions to the top. On this ticket, a reviewer sees that payment succeeded: restoring access belongs to support. They correct the team from billing to support.

   A reassignment in your own app can supply the same correction. Send it with the original decision ID. Now this ticket has an expected answer to test the next prompt against.

   # Same ticket: payment cleared, but access is blocked.
   # The reviewer moves it from billing to support.
   client.feedback(
       decision_id=ticket.decision_id,
       team="support",
   )

   review queueleast certain first

   • MR

     Maya Ruiz

     Paid, but still locked out

     team

     0.54

   • JO

     Jonah Okafor

     Invoice shows the wrong plan

     team

     0.61

   • LS

     Lena Sørensen

     Export keeps timing out

     urgent

     0.66

   reviewer opens ticket #1042

   correctionticket #1042

   reviewer note

   “Payment cleared. Support needs to restore access.”

   team

   billingmodel · 0.54

   supportreviewer

   sales 

   Saved as the expected answer. The next prompt is measured against support on this ticket.
3. 03

   Prove the prompt improved

   The first prompt sent payment questions to billing. Add the missing rule: if the customer has paid but is locked out, route to support. Replay it on the reviewed set, including this ticket.

   This time, the ticket goes to support and matches the saved correction. Compare the original and revised prompts across the same reviewed set to check whether the change helped overall before you ship it.

   group = client.group(decision.group_id)["group"]
   new_template = group["template"] + (
       "\nPaid but locked out? Route to support."
   )
   run = client.prototype(
       decision.group_id,
       template=new_template,
   )

   prompt+1 line

   1. Payment questions go to billing.
   2. +Paid but locked out? Route to support.
   3. Ticket: {ticket.text}

   client.prototype · replay on the reviewed set

   ticket #1042replayed

   MR

   Maya Ruiz

   Paid, but still locked out

   team

   originalbilling

   revisedsupport

   expectedsupport

   Matches the reviewer's correction. The revised prompt gets this ticket right without being told the answer.

   score · same reviewed set84.2%→91.0%