On human greed



  • In 2007 and early 2008, I happened to work as a server guy at a local radio station. I had to mostly maintain existing infrastructure, but was free to do my thing on two of the big servers. One of which was not so big, in fact, and it would happen to be the mail server for the place and a firewall/gateway to the outer world.

    The boss had objections (like, "We're a bit short on cash") for upgrading the old Celeron 1.2 GHz box with 256 MB of RAM and a 40 GB PATA drive (it didn't have any SATA slots inside, you see). They had a 1 Mbps network connection to the outside world, and they were happy. When one Sunday a FreeBSD 4.11 installation didn't bring itself up because someone happened to crack it from the outside and change all passwords which were there (before then, I was told not to change anything, even the root password, because "if it works, don't fix it"), I was more than happy to replace it with Debian which at least I do have some knowledge of. I put there postfix, tied it up with MySQL, and hardened it where I could, because people kept complaining about spam. Then I installed there SpamAssassin, for good measure, and everyone was happy again.

    When I complained to the boss that the machine is running near its limits, I was told that nothing is going to be replaced until there's a need to, and there's no need.

    Then it happened: you know, 1 Mbps/s Internet is not anywhere near broad enough for 40 people. So a broader channel was ordered, four times that bandwidth. As you may predict, if you host a mail server there, most of your inbound traffic will eventually be spam. And boy oh boy, what a lot of spam was there. A poor machine with its scarce RAM couldn't become instantly four times as efficient. It was a DNS server, it was a mail server, it was a web proxy, and it had big fat SpamAssassin running, preventing people from having to read about breathtaking deals on v1@gra and C1@L15. And the most important thing was that they had their web site, yes, running on that same machine! Naturally, their domain was popular enough both among the local audience and among the spammers to saturate the bandwidth and resources on the gateway...

    The load average, at rough estimate, was just slightly over 9000 and I could not even log in on the physical console, SSH constantly timing out: it was so busy that getty and friends got swapped out and were beyond hope to ever swap in.

    It's no surprise that after some time spent on fixing emergencies which happened due to the management making us use almost failing hardware with no hope of replacement, I waved them goodbye.



  • While this is a WTF, it really sounds like the same pattern of how a small shop handles its IT in general. In an ideal world no shop should handle their IT this way, but in the real world this is frequently what happens.



  • This is ridiculous! A new and better machine would've cost a few hundred dollars. Probably less than the monthly cost of coffee and office supplies.

    This isn't greed, since the amount of money involved is minimal, just plain stupidity.



  • @wf_tmro said:

    This is ridiculous! A new and better machine would've cost a few hundred dollars. Probably less than the monthly cost of coffee and office supplies.

    This isn't greed, since the amount of money involved is minimal, just plain stupidity.

     

    But it ain't broke!!!

    Where I work is just like this. Trying to explain to them that a $1k machine would save me way more than $1k worth of time in a year is like trying to explain... something simple to a stupid person, sorry I'm terrible with analogies. They hire the absolute dirt-cheapest offshore programmers because hey, a programmer is a programmer right? We all use keyboards, screens, and mice, so it must all end up the same. I believe management sees programmers like manual labor, just give them enough time and they'll dig you a ditch worth of code. They don't understand (or don't want to) that coding requires skill and artisanship. So I have to explain to them why the code they spent $30 on sucks donkey balls and ends up taking me 40 hours to get to do what they wanted in the first place, and then they're happy with the end result after I work it over, so they look at it as "hey I only spent $30 on this, I'm an awesome manager!". I feel like a stupidity enabler sometimes, but I do like my paycheck, and I'm so jaded by it at this point that I don't give a fuck.

    EDIT:

    Another stupid thing they do is not hire anybody to do IT. The programmers are IT, because it's all computers and we know how to do all that stuff right? I've started saying "no" to stuff when I know I could probably figure it out, but it's way outside of my comfort and knowledge level.  So we have a bunch of ill-maintained "servers", maybe we have backups if someone remembered to, etc. 



  • I still remember showing top(1) screen to the boss. After briefly explaining the figures, especially for memory and LA, I told that the machine has just too many functions on it, and is cranking near its limits.

    “But it does keep up,” the boss said.

    “But it won't be too long,” me and the CTO replied in unison.

    “But it does work and I see no reason to replace it,” that was the reply.

    And then they ordered four times their current bandwidth. It was just as if, after it was confirmed that you can handle a cocktail straw, you were forced to drink from a thick tube plugged to a hydrant. At least SpamAssassin behaved so.

    Now I have to say that SpamAssassin is a creepy semi-WTF, but there was nothing better at filtering at the time. I offered an option to move everyone to Google Apps for business, but extra ~2000 bucks a year, you know, that was much for a radio station...



  • @EJ_ said:

    Another stupid thing they do is not hire anybody to do IT. The programmers are IT, because it's all computers and we know how to do all that stuff right? I've started saying "no" to stuff when I know I could probably figure it out, but it's way outside of my comfort and knowledge level.  So we have a bunch of ill-maintained "servers", maybe we have backups if someone remembered to, etc.

     

    QFT. My job is officially "web developer", but extends to both one-man "IT staff" and "tech support". It's difficult to finish projects when someone is constantly complaining that they can't connect to the fileserver without trying to click on the icon, or that they deleted the shortcut to the Desktop, or that they don't get an IP because they unplugged the ethernet cable (that's all just today).



  • @Schlagwerk said:

    @EJ_ said:

    Another stupid thing they do is not hire anybody to do IT. The programmers are IT, because it's all computers and we know how to do all that stuff right? I've started saying "no" to stuff when I know I could probably figure it out, but it's way outside of my comfort and knowledge level.  So we have a bunch of ill-maintained "servers", maybe we have backups if someone remembered to, etc.

     

    QFT. My job is officially "web developer", but extends to both one-man "IT staff" and "tech support". It's difficult to finish projects when someone is constantly complaining...

    We were like that when I started my current job. We now have 4 people in 'IT' and they are still struggling to keep up with new stuff needed, let alone trying to fix existing stuff.



  • FWIW, if I were in OP's situation I'd have done the following: (And I've seen this done many times before)

    • Make postfix/exim/whatever stop. This sends people into a state of panic.
    • Corallary to this is
    • dd out enough of a few random files so that the rest of the disk is filled, save for a few megs.
    • or worse
    • Set /var/mail to be owed by nobody:nobdy, with a permissions block of 330
    • Stop apache.
    • Watch the chaos ensue. Explain that this is caused by a critical hardware failiure.
    • start MTA again, People have email, people are not /quite/ as unhappy,at least until they can't send or receive mail..
    • Write up the cost of everyone leaving for 2 days. Present this as a bill, with a comparison for building a new box
    • ???
    • Profit.

    The interesting point is that people are not promoted on skill, but on lack of competence. Good job, Upper Managment!



  • The title seems a little bit inaccurate to me.  I hardly doubt that "human greed" was the reason for a radio station to have a shitty budget in 2008.  It's not like everybody in that business is swimmingin a Scrooge McDuck style pool full of gold coins.



  • Management probably had a nagging thought that they were going to shell out for a new server when the old one probably would have coped - and that someone higher than them would kick up a stink. The only way they know they need a new one is when the current one breaks.

    I don't think (sadly) that this mode of thinking is unusual but it's still a WTF.

    There's no excuse for taking a machine that they are told is at breaking point and giving it more work to do.



  • I wonder how much of the blame for situations like this rests at our feet? Maybe as an industry we need to get better at writing CBA (cost benefit analysis) documents?



  • +1 to Indrora, but I wouldn't go to those extremes. The easiest way to solve a problem is to make it someone else's problem. While it's all "we are going to have problems", they can easily respond with "we will worry about it later". When it affects them, it's harder for them to not think about it.

    Show them the problems. Not the ones you're having. The ones that are caused by your problems.



  • @fourchan said:

    +1 to Indrora, but I wouldn't go to those extremes. The easiest way to solve a problem is to make it someone else's problem. While it's all "we are going to have problems", they can easily respond with "we will worry about it later". When it affects them, it's harder for them to not think about it.

    Show them the problems. Not the ones you're having. The ones that are caused by your problems.

    I think what Mr Furry is suggesting is otherwise known as "sabotage." No, it's not a good idea.

    If your communication skills are not up to snuff to explain the problem in such a way that boss-man understands it, then you have two choices:
    1) Find a way to communicate to him
    2) Leave before the system dies and you're running around in a panic trying to cope with it

    I've had to take option 2 before, but option 1 is far better for everybody involved. Sabotaging your employer's systems to make some kind of point is *never* the right way to do it, and is probably illegal.



  • @Indrora said:

    The interesting point is that people are not promoted on skill, but on lack of competence. Good job, Upper Managment!
     

    Even if people are promoted based on competence, it tends to competend for the job they're in, not the job they're being promoted into. Hence the "Peter Principle": people get promoted into jobs they aren't good at and stay there because they're not competent enough to get promoted out.



  •  Hey I heard someone talking about dicks and jobs and I thought I'd say something.



  • @blakeyrat said:

    I think what Mr Furry is suggesting is otherwise known as "sabotage." No, it's not a good idea.

    What I'm describing isn't /sabotage/ its a means by which for people to listen. Sabotage implies permenant and irreprable damage.

    @blakeyrat said:

    If your communication skills are not up to snuff to explain the problem in such a way that boss-man understands it, then you have two choices:

    1) Find a way to communicate to him

    2) Leave before the system dies and you're running around in a panic trying to cope with it

    How about I put this into perspective. this is the equivelant to your boss having their car keyed (after someone gets paid $50 under the table) when they deny that there's a vandalism problem In the parking lot. its not /sabotage/ its a /conversation catalyst./

    @blakeyrat said:

    I've had to take option 2 before, but option 1 is far better for everybody involved. Sabotaging your employer's systems to make some kind of point is never the right way to do it, and is probably illegal.

    Again, not sabotage



  • @Indrora said:

    @blakeyrat said:
    I think what Mr Furry is suggesting is otherwise known as "sabotage." No, it's not a good idea.

    What I'm describing isn't /sabotage/ its a means by which for people to listen. Sabotage implies permenant and irreprable damage.

    1. No it doesn't

      2) No matter what you call it, it's still almost certainly illegal in almost all jurisdictions

    @Indrora said:

    @blakeyrat said:

    If your communication skills are not up to snuff to explain the problem in such a way that boss-man understands it, then you have two choices:

    1) Find a way to communicate to him

    2) Leave before the system dies and you're running around in a panic trying to cope with it

    How about I put this into perspective. this is the equivelant to your boss having their car keyed (after someone gets paid $50 under the table) when they deny that there's a vandalism problem In the parking lot. its not /sabotage/ its a /conversation catalyst./

    That's also illegal, both parts. (The paying, and the keying.) Again, regardless of what word you use to describe it.

    I can't find a way to work the furry mocking into the post itself, so I'll just have to close with: haha, go stroke your tail, furry. I apologize for the lack of wit.



  •  I would call it an unannounced stress test.



  • @Indrora said:

    this is the equivelant to your boss having their car keyed (after someone gets paid $50 under the table) when they deny that there's a vandalism problem In the parking lot. its not /sabotage/ its a /conversation catalyst./
     

    You're not serious, are you?



  • @Indrora said:

    @blakeyrat said:
    I think what Mr Furry is suggesting is otherwise known as "sabotage." No, it's not a good idea.

    What I'm describing isn't /sabotage/ its a means by which for people to listen. Sabotage implies permenant and irreprable damage.

    @blakeyrat said:

    If your communication skills are not up to snuff to explain the problem in such a way that boss-man understands it, then you have two choices:

    1) Find a way to communicate to him

    2) Leave before the system dies and you're running around in a panic trying to cope with it

    How about I put this into perspective. this is the equivelant to your boss having their car keyed (after someone gets paid $50 under the table) when they deny that there's a vandalism problem In the parking lot. its not /sabotage/ its a /conversation catalyst./

    @blakeyrat said:

    I've had to take option 2 before, but option 1 is far better for everybody involved. Sabotaging your employer's systems to make some kind of point is never the right way to do it, and is probably illegal.

    Again, not sabotage

     

    Call it what you will; it doesn't make it (a) ethical, (b) likely to succeed, or (c) career-enhancing.  Sure, it will be a "conversation catalyst", but if they know that you did what you described, the conversation will be extremely short:

    You: Here's the report/result/etc.

    Boss: You're fired.


Log in to reply
 

Looks like your connection to What the Daily WTF? was lost, please wait while we try to reconnect.