Giving tools to an AI agent: why I set the limit outside the prompt

Описание и применение

<p>When I give tools to an agent, I treat it like a third-party dependency: I don't trust it, I contain it.</p> <p>The rule that matters: the constraint is set <strong>when the process starts</strong>, not in the prompt. A prompt is a request. The model can ignore it, or be turned around by malicious text slipped into what it reads.</p> <p>On an agent I built, the tool list is closed in advance: twelve read tools allowed, fifteen write tools refused. It can't write a file or send a message, even if it tries.</p> <p>Three more protections:</p> <ul> <li>an anti-injection section in the prompt, b

Задача

<p>When I give tools to an agent, I treat it like a third-party dependency: I don't trust it, I contain it.</p> <p>The rule that matters: the constraint is set <strong>when the process starts</strong>, not in the prompt. A prompt is a request. The model can ignore it, or be turned around by malicious text slipped into what it reads.</p> <p>On an agent I built, the tool list is closed in advance: twelve read tools allowed, fifteen write tools refused. It can't write a file or send a message, even if it tries.</p> <p>Three more protections:</p> <ul> <li>an anti-injection section in the prompt, b

Результат

<p>When I give tools to an agent, I treat it like a third-party dependency: I don't trust it, I contain it.</p> <p>The rule that matters: the constraint is set <strong>when the process starts</strong>, not in the prompt. A prompt is a request. The model can ignore it, or be turned around by malicious text slipped into what it reads.</p> <p>On an agent I built, the tool list is closed in advance: twelve read tools allowed, fifteen write tools refused. It can't write a file or send a message, even if it tries.</p> <p>Three more protections:</p> <ul> <li>an anti-injection section in the prompt, b

Доказательства

Giving tools to an AI agent: why I set the limit outside the prompt

Источник