Building a Secure LLM Proxy with Fine‑Tuned Gemma 3 for Production Applications

Description and use

AEGIS is a lightweight proxy that protects LLM endpoints from PII leakage, prompt injection, and malicious files, using MIME verification, OCR, PII scrubbing, a Gemma 3 safety classifier, and web context sanitization.

Problem

Freelance developers and small agencies need a low‑cost, lightweight security layer to protect client‑facing LLM endpoints from accidental PII leakage, prompt injection, and malicious payloads in documents or web context.

Outcome

AEGIS provides a drop‑in security proxy that sanitizes PII, neutralizes prompt injections, de‑weaponizes uploaded documents, and filters web context, enabling independent developers to offer a compliance layer on modest budgets.

Workflow

• Receive untrusted payload (text, PDF, image, web search)
• Layer 1: Magika deep MIME & polyglot verification
• Layer 2: Dual‑stream OCR & hidden text extraction
• Layer 3: Context‑aware Presidio + regex PII scrubber (redacts <EMAIL>, <PHONE>, <API_KEY>)
• Layer 4: Fine‑tuned Gemma 3 / LoRA safety classifier (detects prompt injections & malicious payloads)
• Layer 5: Sanitized web retrieval via SerpApi (filters poisoned knowledge)
• Forward clean, verified context to target LLM
• Return response to user

Evidence

AEGIS is a lightweight, drop‑in security proxy that developers can place in front of their LLM endpoints.

Source

It acts as an intermediary gateway that sanitizes personal data in real time, inspects and OCRs uploaded documents, evaluates prompt safety using an open‑source fine‑tuned Gemma 3 model, and provides sanitized web retrieval through SerpApi.

Source

The gateway handles inbound requests across multiple input types. Plain text prompts are stripped of credentials and personal identifiers before being evaluated by the safety classifier.

Source

For file uploads undergo deep MIME verification and visual OCR to detect obscured text before text extraction proceeds.

Source