Not every AI call needs to generate text: ej, an 11 MB model for typed decisions

Описание и применение

ej – лёгкая модель 11 МБ, которая принимает текст и типизированные вопросы, выдаёт распределения вероятностей без генерации текста, позволяя быстро принимать решения в пайплайнах.

Задача

Decisions in production pipelines often require only a probability distribution over options, not text generation, yet existing solutions force costly text generation or multiple model passes.

Результат

ej reduces compute and cost by avoiding text generation, provides direct probability outputs for thresholding, and fits entirely within a service’s CPU resources.

Рабочий процесс

• 1. Prepare a record with "state" (text or JSON) and a set of questions of types choice, noul, or score.
• 2. Load the ej model via ej.load("5ak3t/ej", revision="v0.0.1").
• 3. Call model.predict([record]) to obtain a dictionary of probability distributions.
• 4. Apply threshold logic on the highest probability of each question to decide routing or human escalation.
• 5. Optionally use adapt with labeled examples to improve performance on specific workflows.

Доказательства

It returns one probability distribution per question, in one forward pass, without generating a single token.

Источник

The whole model is one 11 MB file that runs on a CPU.

Источник