Before an LLM request leaves your app, inspect what it contains

Description and use

Sether replaces sensitive values in LLM prompts with tokens to reduce data leakage, enabling developers to verify what leaves the application, though it does not ensure full compliance.

Problem

Sensitive data can leak from application to LLMs when requests are sent without proper redaction, risking privacy violations.

Outcome

Sether enables controlled redaction of sensitive data in LLM prompts, allowing developers to verify what data leaves the application and to restore tokens in responses, though it does not guarantee full compliance.

Workflow

• 1. Create fictional messages with task instructions and sensitive values.
• 2. Run Sether to redact sensitive values, producing a tokenized prompt.
• 3. Inspect the redacted request, counting missed values and unnecessary replacements.
• 4. Send the redacted prompt to the LLM and evaluate task quality.
• 5. Receive the response, check for preserved tokens, and restore original values using the mapping.
• 6. Test edge cases: altered tokens, missing mappings, and streaming where values split across chunks.

Evidence

I am building Sether around that boundary. It is an open-source library that replaces detected sensitive values with stable tokens before text goes to an LLM.

Source

The useful question is not “did we add a privacy tool?” It is “what actually left the application?”

Source

A practical evaluation has four parts: ...

Source

Streaming adds another case: a value may be split across chunks. Include that in your tests rather than relying only on complete strings.

Source

Sether is not a promise that every sensitive value will be detected, and installing it does not establish regulatory compliance.

Source