Your Java LLM agent works in the demo. Friday night in production is another story.

Description and use

In production, an LLM refund agent failed due to provider outages, malformed JSON, and unapproved large refunds. Implementing ARA added deterministic output validation, provider failover, cost caps, and a human approval gate, resolving the incidents.

Problem

An LLM‑driven refund agent failed in production: provider 503s, malformed JSON, and unapproved large refunds.

Outcome

The agent now validates outputs deterministically, retries malformed responses, requires human approval for critical actions, and handles provider failures and cost caps, preventing the production incidents described.

Workflow

• 1. Configure AgentContract with input sanitizers, content truncator, output schema, Markdown fence stripper, and repair attempts.
• 2. Create AraRuntime with llmClient and approvalGate.
• 3. Define AraAgent with humanApprovalRequired=true and enabledTools=["refund"].
• 4. When agent calls refund, runtime parks the call and creates an approval request.
• 5. Operator reviews request and submits Approved, Rejected, or Modified decision via gate.submit().
• 6. Agent proceeds based on decision or retries if rejected.

Evidence

ARA is an open source (Apache 2.0) Java 21 runtime for LLM agents, built on LangChain4j.

Source

The model wraps its JSON in a Markdown fence, or forgets a required field.

Source

An agent can have access to the right tool, understand the request and even write a convincing justification.

Source