Here I built a little game to demonstrate how classifications work, perform, and how they can be integrated into any workflow: https://woodstockops.com/jev

For each scenario in the game, I wrote three sets of criteria outlining what I think makes a good, mid, or crazy answer. Jev looks at the scenario and your response against those three criteria, then returns how likely your response matches the outcomes. It also checks whether you're answer is nonsense in order to stop spammy submissions.

The story itself is written by me. Jev's job is only to assess your choice, not invent what happens next.

For comparison, a more typical model (Llama 3.3) assesses your answer too. Llama is a more typical model, like what you might use normally for summarising things or answering questions. Here, I've limited it to a one word verdict so it isn't spending time and money writing an explanation.

The table then lets you compare their decisions, response times and estimated costs. These scenarios involve judgement, so it isn't necessarily a definitive test of which model is better.. It's more like a way to see how two different types of tools handle the same task.

The game itself is a fun way to learn the tool, but the automation underneath is actually useful to anyone:

Read some text.

Check it against criteria.

Decide what happens next.

That could mean routing a support ticket, flagging a comment for review, or approving if a request meets a policy. Your criteria and scenario will be unique, but an AI judging a quick and cheap outcome can be applied to anything.

For a decision making AI like Jev, the question is.. do you need AI to write a proper text response, or simply to make an assessment? If you don't need something written, a "System One model" model is worth testing, particularly when the task happens hundreds or thousands of times a day.

If your team spends time reading, sorting or checking the same kinds of information on repeat, I'd love to hear about it, and share how I can save you time and money.