Not every AI call needs to generate text: ej, an 11 MB model for typed decisions

Description and use

ej is a lightweight 11 MB model that takes text and typed questions, outputs probability distributions without generating text, enabling fast decision‑making in pipelines.

Problem

Decisions in production pipelines often require only a probability distribution over options, not text generation, yet existing solutions force costly text generation or multiple model passes.

Outcome

ej reduces compute and cost by avoiding text generation, provides direct probability outputs for thresholding, and fits entirely within a service’s CPU resources.

Workflow

• 1. Prepare a record with "state" (text or JSON) and a set of questions of types choice, noul, or score.
• 2. Load the ej model via ej.load("5ak3t/ej", revision="v0.0.1").
• 3. Call model.predict([record]) to obtain a dictionary of probability distributions.
• 4. Apply threshold logic on the highest probability of each question to decide routing or human escalation.
• 5. Optionally use adapt with labeled examples to improve performance on specific workflows.

Evidence

It returns one probability distribution per question, in one forward pass, without generating a single token.

Source

The whole model is one 11 MB file that runs on a CPU.

Source