TypeSafe's Jev is a new kind of model. It does not write text: you give it a situation and a set of options, and it returns a probability for each option, quickly. I wanted to see how that plays out in practice, so I put it against Haiku, a fast conventional LLM, in four simple games.
- Models. Jev 1.13 through TypeSafe's API; Claude Haiku 4.5 without extended thinking.
- Games. Each one tests a different side of Jev:
- Snake: how well does it read space and danger on a grid?
- Chess: how well does it judge the value of a move?
- Minesweeper: how well does it judge risk from local clues?
- Codenames: how well does it judge semantic closeness between words?
- Ground rules. Both models get the same text and the same options. Haiku answers in JSON with a confidence it states itself; Jev returns a probability for every option.
- Limits. 100 rounds for Codenames (20 boards × 5 clues) and 20 games for the others.
Every move was logged with its timing and the full game state, so the replays below let you watch each decision of both models side by side, in the time it actually took.
At a glance
| GameMeasure | Result | Time per decision (median) | Cost per 1,000 decisions | |||
|---|---|---|---|---|---|---|
| Jev | Haiku | Jev | Haiku | Jev | Haiku | |
| Snakefood per game, turn-based | 38.3 | 39.1 | 0.23 s | 0.64 s | $0.02 | $0.54 |
| Chesswon / drawn / lost | 2 / 17 / 1 | 1 / 17 / 2 | 0.23 s | 0.83 s | $0.03 | $0.84 |
| Minesweepergames won | 6 of 20 | 0 of 20 | 0.29 s | 8.53 s | $0.37 | $11.21 |
| Codenamesintended words found | 61.4% | 55.8% | 0.24 s | 1.31 s | $0.04 | $1.91 |
| Overall | Jev ahead in 2, level in 2 | Jev 2.8–29× faster | Jev 27–45× cheaper | |||
Jev was faster in every game and 27 to 45 times cheaper per decision, and in every game except chess its probabilities ranked its own decisions well: when it was confident it was usually right. In Snake, Haiku said 0.95 on almost every move, right or wrong. Where there was no clock and code did the counting, Haiku drew level, and in turn-based Snake it was slightly ahead.
Snake
A 16 × 16 board, one food item at a time, 500 moves at most. Both models see the board as text and choose a direction. Turn-based, with the move facts, they are level: Jev ate 38.3 food per game and Haiku 39.1. Switch the pace to real time and the snake moves twice a second whether or not an answer has arrived. Jev answers in about 0.23 s and keeps up; Haiku takes 0.6–0.7 s and never once answers in time, so its snake runs straight into the wall.
Chess
Twenty lightning games, Jev against Haiku directly, 10 seconds each plus 1 second added after every move. Stockfish graded every move afterwards. Both played the same quality of chess, weak and even (on average about 1.3 pawns of value lost per move), and most games ended with too few pieces to mate. Jev moved 3.6 times faster for a twenty-seventh of the cost. Switch the clock to 30 seconds, then 1 second per move (byo-yomi, as in Go), and replay the same moves: Haiku runs out of time in 9 games and Jev in 1, after a single 30-second stall of the API.
Minesweeper
On the raw board neither model could tell mines from safe cells. With the counting done by code (each number's hidden neighbours and how many mines they still hold), both became useful. A rule opened only cells whose calibrated mine chance was at most 5%; for Jev those cells really were mines 3.7% of the time. Jev won 6 of 20 games in about 16 seconds each. Haiku won none and took about 8 minutes a game.
Codenames
The guesser's job in Codenames: given a one-word clue and a number, find the words on the board it points to. This is the closest of the four to the text-classification work Jev is built for. Over 100 rounds written by a Sonnet 5 spymaster, Jev found 61.4% of the intended words and Haiku 55.8%, with Jev 5 times faster.