Our Service Desk and a Customer's Spent a Weekend Opening Tickets for Each Other
Description and use
Automatic acknowledgements from two support systems created an infinite loop, generating over 11,000 duplicate tickets. Implementing header checks, rate limiting, and routing known customer emails stopped the loop.
Problem
Automatic acknowledgements from two service desk systems triggered an infinite loop, creating over 11,000 duplicate tickets on each side.
Outcome
The loop created 11,400 tickets on each side; after implementing header checks, rate limiting, and queue adjustments, the system now prevents similar loops and controls ticket volume.
Workflow
- 1. Customer’s portal notifications sent from customer’s service desk mailbox.
- 2. Company’s service desk received notification and created a ticket, sending an acknowledgement.
- 3. Customer’s service desk treated acknowledgement as a new request and created a ticket, sending another acknowledgement.
- 4. Company’s service desk repeated the process, creating a loop of tickets.
- 5. After the incident, company’s service desk added automatic message header detection and rate limiting.
- 6. Configured queues to ignore acknowledgements from known customer/supplier mailboxes.
- 7. Implemented surge handling to alert on‑call engineer after ten minutes of high volume.
Evidence
Our service desk tool sends an automatic acknowledgement to every email it receives, with a ticket number and a line saying we will be in touch.
By Monday morning our queue held eleven thousand four hundred tickets and theirs held about the same.
Our desk now marks every automatic message with the standard header that tells other systems it was generated automatically, and it no longer acknowledges incoming mail that carries the same header or its common equivalents.
No sender receives more than five acknowledgements in an hour. A sixth message from the same address in an hour opens one ticket for a person to look at, with the rest attached to it.
- Record date: 2026-10-13T06:40:56.222Z