Do you log your LLM prompts and responses? Four answers, and the one I'd pick

Description and use

<p>Friday AI Question. This week's question is one every team shipping an LLM feature has to answer, usually after the first weird bug report:</p> <p><strong>Do you log your LLM prompts and responses?</strong></p> <ul> <li> <strong>A. No.</strong> We don't store prompts or outputs at all.</li> <li> <strong>B. Metadata only.</strong> Model, token counts, latency, cost, finish reason, user/session IDs. No text.</li> <li> <strong>C. Full text, redacted, short retention.</strong> Prompts and outputs are kept, with obvious personal data masked, deleted after a fixed window, and access is restricted

Problem

<p>Friday AI Question. This week's question is one every team shipping an LLM feature has to answer, usually after the first weird bug report:</p> <p><strong>Do you log your LLM prompts and responses?</strong></p> <ul> <li> <strong>A. No.</strong> We don't store prompts or outputs at all.</li> <li> <strong>B. Metadata only.</strong> Model, token counts, latency, cost, finish reason, user/session IDs. No text.</li> <li> <strong>C. Full text, redacted, short retention.</strong> Prompts and outputs are kept, with obvious personal data masked, deleted after a fixed window, and access is restricted

Outcome

<p>Friday AI Question. This week's question is one every team shipping an LLM feature has to answer, usually after the first weird bug report:</p> <p><strong>Do you log your LLM prompts and responses?</strong></p> <ul> <li> <strong>A. No.</strong> We don't store prompts or outputs at all.</li> <li> <strong>B. Metadata only.</strong> Model, token counts, latency, cost, finish reason, user/session IDs. No text.</li> <li> <strong>C. Full text, redacted, short retention.</strong> Prompts and outputs are kept, with obvious personal data masked, deleted after a fixed window, and access is restricted

Evidence

Do you log your LLM prompts and responses? Four answers, and the one I'd pick

Source