I taught a 322M-parameter model to play Snake. It lost to its own confidence.
Every pellet is a real support message in one of 12 languages. Laya — an
open-source non-autoregressive decision model — reads each one in a single
forward pass and types it: billing, technical, sales, or other. The model picks
the target; a plain BFS does the steering.
No text generation. Nothing to parse. One pass, 25 ms.
Then it got interesting.
Watch the rounds where the target is "sales." The snake starves. Laya reads
"price for 200 seats" as billing — and in Hindi, "कीमत बताइए" comes back as
billing at 0.99 confidence. Confidently wrong. A confidence gate can't save you,
because the confidence is high.
Second surprise: latency collapsed under sparse traffic. Back-to-back calls ran
25 ms. The identical calls 2.5 s apart ran 323 ms — 13x slower, purely from idle
gaps letting the GPU kernels go cold. A 400 ms keep-warm heartbeat brought it
back to 58 ms.
You don't find either of those in a benchmark table. You find them by making the
model do something in real time, on a loop, where you can watch it fail.
Laya by Nandha Kishor M / Convai Innovations. Ran locally on an M-series Mac —
these numbers are mine, not the published T4 benchmarks.
#MachineLearning #NLP #OpenSource #AIEngineering