With love... from Exchange
YellowOnline last edited by
An e-mail arrived in the support mailbox of a small country's Ministry of Lost Cases. A reminder that the services of a branch office are interrupted at night because of a neighbour's complaint that the servers make too much noise. Amazed that a very light sleeping individual can actually force the state to shut down servers overnight, Juan did not notice at first the looming disaster and continued the daily routine of trying to solve tickets faster than they were created. In vain.
Someone replies to the previous e-mail: "OK".
"What a stupid reaction" he thought before it dawned on him. How come he gets the reply to that message anyway?
Just from the size of the scroll bar it was clear that someone in the communications department made a mistake: BING a mail that had to be sent to a distribution list containing the whole organization had BING somehow been expanded and all of the addresses were BING in the To: field of the e-mail. Juan didn't count it, and there BING might be a limit to how many recipients Exchange allows, but in theory at least, this mail had been BING sent to 25 000 people. And they started to do the feared "reply to all".
The next fifteen minutes Juan spent looking for the Exchange admin who was on lunch break. Meanwhile, the mailbox had already more than 100 replies, most of them people answering (to 20 000 people) "Hey, why do I get this e-mail?" or simply "Stop sending me this irrelevant mail!" and a support employee trying a well meant e-mail (to 20 000 people) "Stop replying. You are causing a snowball effect for the mail system!".
By the time the Exchange admin came around, there wasn't much Exchange 5.5 left. The system was before this sudden increase in load already running at its limits, or rather past those limits, as it handled more traffic and had more storage than could possible be imagined when 5.5 came out.
Exchange crashed and in its fall somehow managed to drag SentMail, OCS and Lotus into the void. It took a lot of money and some Microsoft experts flown in from across the Atlantic to get the e-mail system running again in about three days.
But all's Well That Ends Well: a year later after this débacle, and the bad press that came with it, the organization would switch to Exchange 2010 and get a real Exchange Admin instead of a public servant who had once done a course on Windows NT.
Legend says that the brave support guy who sacrificed his mailbox for the common good meanwhile also got through all the OOF and NDR messages he received that day.
After Gone Phishing I decided to only send stuff in "Error'd" and write my own (true) stories, e.g. Confession: Proactive wiring). I'm happy if they entertain you long enough to eat a sandwich. And don't mention Hanzo.
Magus last edited by
I always love it when the 10th "Hey unsubscribe me from this list as well" comes through, and especially when the 10th person tells everyone to stop replying, because they're making it worse.
Gąska last edited by
Something similar, but on a much, much, much less scale happened to me recently. Basically, there's this JIRA instance used internally by one of projects (by project, I mean a several hundred man department, and there's few dozen of those over here), and a ticket was created there. For some strange reason, an email notice about it was sent to a thousand or two developers and testers, some by name, some by various mailing groups, but about 99% of them were unneeded. And within this majority of people who don't care at all, there's a very vocal minority of people who don't care so much that they need to say that aloud. Within an hour before it was deleted, the ticket received about a thousand comments, all of them being a variation of "ABC has nothing to do with it" or "DEF is not responsible for this use case" or "why was GHI attached in discussion?" or "not JFK problem".
This wasn't the first time, and probably won't be the last.
"I got 99 problems, but a bullet ain't—"
<I'm going to hell for that one, aren't I?