Building Amit.AI, Part 1: The Frontend

Description and use

Fast client‑side FAQ answers, deferred LLM calls, token streaming via fetch, backend warm‑up ping, safe rendering.

Problem

Provide instant, free FAQ answers in a browser while deferring to an LLM only when necessary, avoiding free‑tier quota waste and backend latency.

Outcome

Instant FAQ answers, reduced LLM usage, mitigated backend sleep latency, and safe streaming rendering.

Workflow

• 1. Detect if user message is confident using a token‑overlap matcher with thresholds and stopword filtering.
• 2. If confident, return FAQ answer immediately.
• 3. If not confident, send message to backend via POST and stream tokens using a hand‑rolled fetch reader.
• 4. Render tokens incrementally, keeping typing indicator until first token.
• 5. Warm backend with a health‑endpoint ping during boot animation.
• 6. Apply guardrails to block jailbreak‑style or disallowed content before any answer is produced.

Evidence

'Most questions never reach an LLM. A tiny client-side matcher answers them instantly, and when it is unsure, it stays quiet and defers.'

Source

The matcher returns one of three things: guard, faq or defer. There is deliberately no fourth "I'm not sure" answer. A defer is silent.

Source

SSE frames are separated by a blank line, and a network chunk can end in the middle of one. So you buffer, split on \n\n, and keep the leftover for the next read.

Source