Verifying that an approved correction reaches the next agent run

Description and use

Tests confirm that a user‑approved correction is persisted and automatically applied in subsequent agent runs, improving output accuracy.

Problem

A corrected answer isn’t guaranteed to influence future agent runs; a test must trace the correction from approval, through persistent storage, into the next run’s context, and finally into the output.

Outcome

Fixture tests confirm that the alias persists across runs and affects output; the system automatically approves repeated misspellings in this controlled scenario.

Workflow

• 1. User submits misspelled vendor “Cobolt Cloud”.
• 2. Agent replies with correction “Cobalt Cloud” and requests approval.
• 3. User approves the correction; reply handler saves alias when decision is approval, state is revised, and corrected vendor differs from original.
• 4. Alias relation mg:alias_of is stored in client’s vendor namespace.
• 5. Next agent run triggers extract_ctx query, assembling context with organizational facts and recent thread events.
• 6. Extraction tool builds alias lookup from grains where relation is mg:alias_of and replaces vendor if matched, raising confidence to 0.95.
• 7. Agent processes request using updated vendor and outputs corrected result.
• 8. Smoke and improve scripts run to assert that the alias persists and auto‑approval occurs on repeated misspelling.

Evidence

A corrected answer isn’t guaranteed to influence future agent runs. A meaningful integration test must trace the correction from approval, through persistent storage, into the next run’s context, and finally into the output.

Source

The reply handler, the alias is only saved when the decision is approval, the state is marked as revised, and the corrected vendor differs from the original input.

Source

The extraction tool processes the provided grains and constructs an alias lookup... if vendor in aliases: vendor = aliases[vendor] confidence = max(confidence, 0.95)

Source

assert again[0]["vendor"] == "Cobalt Cloud", again[0] assert again[0]["approved_by"] == "auto", "it should not have needed a person this time: %r" % again[0]

Source