#llm #ml #ai #transformers
To my understanding, TypeSafe's Jev might be a clever repackaging of an ordinary transformer — an encoder with a classification head instead of a decoder (without the quadratic compute cost associated with decode), sold differently. Of course, it is just a hypothesis.

Encoder-only classifiers are a known thing. They can be treated as an LLM reading out at a single position, with logits over the option set - and without the need to feed it back.

Processing input tokens is roughly O(N²) for large enough N, but highly parallel - given unlimited compute, the wall clock is theoretically just O(log N), and N is capped at 32k.

TypeSafe's API lets you pass the “state” and a bunch of classifier-style questions. The same state across multiple questions tells a prefix-caching story - the state's KV is computed once and reused. The rest of computation parallelizes per question. And they advise keeping questions small, so the state dominates until the questions' total size exceeds the state.

The LLM as product packaging is “sell input tokens for X, output at 3–5X" — where both cost them quadratic compute, with wall clock linear in the tokens generated.

Assuming that architecture, TypeSafe similarly sells "input tokens for X, output free", which is honest: there is no decode. State costs O(N²) in compute once, each question adds O(M·N), they kind of encourage you to implement caching with their API shape. Wall clock stays O(log N) in theory, and adding questions doesn't move it.

And that N² is bounded, since state is capped at 32k, holding down N².

Reply to the post by Denis Sexy IT 🤖:

Наконец-то поигрался с Jev, новой моделью которая не генерирует текст как обычные LLM, а условно, очень быстро, почти забесплатно и при это качественно – заполняет JSON поля; технически вот тут можно почитать как устроена – нужна она чтобы быстро ставить чему-то оценку или классифицировать, или нажимать какие-то кнопки заданные заранее

В видео тест в реальном времени – попросил ее оценить 4 мегабайта текста (~1 597 постов) этого канала, и разбить по категориям

Справилась за ~1 минуту в 16 потоков, стоило все 0.08$ долларов

Я всего на 3% шутник, понятно вам? В канале приколов нет, теперь официально 🗿