A few details lost in the implementation



  • My client has a weekly report that shows to top management various indicators about the organization. Here is the process for the preparation of the report:

    1. Department managers update numbers in a shared Excel spreadsheet
    2. Since there are 25+ people updating the file, contention is a big issue and multiple copies are created by annoyed managers. A manual reconciliation of all those copies is done by the helpdesk... at least twice during a typical week.
    3. The spreadsheet is loaded in an Access database that is used by some assistant to visualize (and possibly "tune") the data using Access queries.
    4. The spreadsheet is loaded in Oracle using a macro (which of course is in a second spreadsheet linked to the first).
    5. If the assistant has made changes in Access, either before or after the spreadsheet has been loaded in Oracle, she emails the new numbers to the DBA team so they can update the database.
    6. Thursday night aggregation jobs are executed in Oracle to update totals (they frequently fail because most numbers are in VARCHAR fields that sometimes contain spaces or other non-numeric values).*
    7. On Friday, The Communications Manager (a glorified secretary) fires up her browser to access a custom web application where the data pulled from Oracle is displayed. She copy-paste it in a Word document and sends the document by email to top management.
    * yeah, pre-calculated totals.

    Everybody in IT hates this report because every bit keeps breaking and they have to drop everything to fix it. Two months ago they hired an intern, a pretty sharp kid, and on his spare time he built a web application that would allow managers to enter their numbers directly in Oracle, without requiring the nasty spreadsheet; this application also allows the assistant to make changes and includes valuable features like audit and rollback. The web application is designed to be consumed by top management but there is also an "Export to Word" button designed specifically for the Communications Manager. Everything is all textbook restful/ajax/jQuery/ORM/etc. and the dude even used the intranet stylesheet to "brand" the application properly.

    Beta users loved the application and today a demo was scheduled to get the ok from IT management to formally deploy it. The demo went well, then the proud intern was excused so IT management could "iron out implementation details". I had the privilege to remain in the room, being the rent-a-bad-cop for one of the managers.

    The discussion lasted at most 5 minutes before switching to 30 minutes of idle chat (mostly JP Morgan bashing). Because a critical piece of the organization could not be trusted in the hands of an intern (what would happen if he left? who would have to maintain the beast?), someone suggested to use a "proven product" with "similar features" instead. So they decided to buy Cognos and agreed on a $80k budget to deploy it; after all, Cognos has the "Export to Word" button and is in Gartner's Magic Quadrant. For the first iteration of the project, only the Communications manager will have access to the Cognos but she will use Cognos instead of building the Word document manually (i.e.: big win). None of the actual problems are solved and they will spent probably much more than expected to deploy an overkill report viewer, but to them it's Mission Accomplished.

    I suggested to the guy in charge of the intern to wait until Monday before letting him know what "the implementation" will be. He did an excellent job so IMHO he deserves to live in Utopia one more weekend before discovering what real life is like in IT.



  • @Speakerphone Dude said:

    So they decided to buy Cognos and agreed on a $80k budget to deploy it
    We use Cognos.  All I know about it is:  when accessing it from a modern Windows 7 machine (8 GB RAM), it is very fast.  When accessing it from a Windows XP machine (32 bit, 4 GB RAM) + PGP encryption . . . I get complaints about slowness.  At the same time as the Windows 7 machine is accessing it just fine.  Both clients on the same subnet.  But somehow it's still a network problem.  (No, really, it's not . . .)



  • God that was depressing. I feel bad for the guy.



  • We can only offer our sincere condolences, Dude. And hope that this has been a good lesson for the poor intern.



  • When I got to the end, I was thinking "hope he didn't do that in his own time", because this scenario sounds horribly familiar. Alas, a quick scan back through the article, and sure enough...



  • @Speakerphone Dude said:

    rent-a-bad-cop for one of the managers
    That's an awesome title to put on your business card.

    @TGV said:

    We can only offer our sincere condolences, Dude.
    +1



  • I'm pretty sure that this has happened to every single person on this forum and we're used to it being "par for the course". This is how spirits are broken and great employees become average employees.



  • @Speakerphone Dude said:

    what would happen if he left? who would have to maintain the beast?
     

    Surely this is so commonplace in the development world that there are defined handover procedures involving documentation and operational manuals - that which is invented must be maintainable.

    Did anyone suggest getting him to properly document it then purchasing it off him? If this isn't an issue, tell him to document it then he can sell it onto someone else - it is, after all, his work. Let him take it with him when he leaves.

    Truly a sad story given that the requirements are fundamentally simple (capture metrics from various sources, collate them then produce reports) and not unique to your organisation, yet the (original) manner in which they're done is highly inefficient and not particularly cost-effective. It's definitely crying out for automation but a meeting of "implemention details" actually being "rejection of tried and tested + approval of alternative system" is the WTF for me.

    Rifle + clock tower. It's the only way to be sure.



  • @Cassidy said:

    @Speakerphone Dude said:

    what would happen if he left? who would have to maintain the beast?
     

    Surely this is so commonplace in the development world that there are defined handover procedures involving documentation and operational manuals - that which is invented must be maintainable.

    Unfortunately, developers that are clueless and blame their lack of ability to maintain a simple web application on the lack of documentation from the author are more common.

    I'm going through this right now at work. I'm being pressured to spend an inordinate amount of time training the rest of the team because they can't fix a bug in less than four months. This persists even though I have demonstrated that solving 90% of those bugs requires nothing more than reading the stack trace and knowing how to use the debugger.



  • @Jaime said:

    This persists even though I have demonstrated that solving 90% of those bugs requires nothing more than reading the stack trace and knowing how to use the debugger.
     

    Mmmm... I was thinking of maintanence more in terms of enhancement changes, rather than bug fixes, but I get yer point.

    @Jaime said:

    Unfortunately, developers that are clueless and blame their lack of ability to maintain a simple web application on the lack of documentation from the author are more common.

    Won't argue with that, but documentation does mean you (at least) have some understanding of the app at hand. Lack of it means you've got to spend time and effort just getting to that point. I sympathise with their POV, but lack of documentation doesn't mean an unmaintainable app, it just makes the job easier (and quicker, thus cheaper).

    The flipside is clueless developers that don't bother with documentation and believe you can understand the codebase by peeking at the source[1] - I've yet to experience across any code that's so well-written it's self-describing and requires no documentation at all. Some classes have come close, but I've still had to scrawl my own diagrams out to make more sense of it.

    [1] IME, the majority of these tend to be PHP "developers"[2].

    [2] I use that term loosely. "Butchers" would be more appropriate.



  • At my previous job a lot of my time was spent writing web apps to replace convoluted legacy systems like the one you describe. Fortunately I had a boss with enough clout to make my improvements stick. Oh happy days.



  • As stated above, we've all had these moments and we all swiftly learn how the world really works. When I was young and stupid I experienced this many times and ranted to everyone who would listen, I couldn't see why they didn't understand, of course now I can see they must have thought exactly the same thing about me. Out there in the real world getting a job done is the last thing on anyone's mind, its all about power games and social wrangling, I often compared it to working in a playground full of apes.

    ... <snip> ... I started writing a rant here, then suddenly realised it would just be that, I've done it before and you don't even need to read it because its the same rant that goes on inside your head all the time.

    Its easier to just recall the example of The DHCP Office and let it speak for itself.

    The bigger the organisation the LESS of a benefit good clean efficient processes are appreciated because every script interferes with someone's social struggle or puts someone's friend out of a job, after drifting between business and public sector I find that money makes it even worst. At least in a small business savings in manhours are appreciated on the bottom line but when you basically have a blank cheque and make money without the need to compete, there is nothing to discourage this sort of process bloat.

    I hate it because, I can see how IT could do so much and streamline so many processes but then see layers of management and more useless process be built around the IT, completely defeating the original point. Just as I remove one tedious admin job through the use of IT, someone invents three more tedious admin jobs that have to be done to manage the computers and processes I used.

    I feel like I've been swimming through custard, the harder I tried the stiffer it got and so a couple of years back I just stopped trying altogether.



  • @EncoreSpod said:

    I find that money makes it even worst. At least in a small business savings in manhours are appreciated on the bottom line but when you basically have a blank cheque and make money without the need to compete, there is nothing to discourage this sort of process bloat.

    I agree. As absurd as it seems to people stuck in small companies hurting for money, bottomless IT budget makes it difficult to promote grassroot innovation and leads to a culture of incompetence where people flock to vendor- or product-driven architecture.



  • @Cassidy said:

    @Speakerphone Dude said:

    what would happen if he left? who would have to maintain the beast?
     

    Surely this is so commonplace in the development world that there are defined handover procedures involving documentation and operational manuals - that which is invented must be maintainable.

    Did anyone suggest getting him to properly document it then purchasing it off him? If this isn't an issue, tell him to document it then he can sell it onto someone else - it is, after all, his work. Let him take it with him when he leaves.

    Truly a sad story given that the requirements are fundamentally simple (capture metrics from various sources, collate them then produce reports) and not unique to your organisation, yet the (original) manner in which they're done is highly inefficient and not particularly cost-effective. It's definitely crying out for automation but a meeting of "implemention details" actually being "rejection of tried and tested + approval of alternative system" is the WTF for me.

    Rifle + clock tower. It's the only way to be sure.

    This situation is not about process, it's about risk management. In many organization, nobody is "guilty" of the current state, no matter how messed up it is, but if someone brings in a solution to a known problem, they become a sponsor; this means that they put their name on it and they will look bad if the solution does not work. The only way around this is to use the power of Mainstream and bring in a known product/vendor/consultant, hence the expression nobody ever got fired for buying IBM. The solution made by an intern, no matter how brilliant, does not carry this weight and is therefore a liability.

    The proper way to solve this problem would have been to make a business case, outline the issues, and propose a solution built on a known platform or at least a known approach (such as SOA). Obviously a business case providing a solution costing less than $80k would have been accepted. Faced with a liability but unable to keep the lid on a known problem, management did as one would expect - they ran to IBM.



  • @Speakerphone Dude said:

    This situation is not about process, it's about risk management.
     

    Hmm.. well....

    The first stage in risk management is risk analysis - and the risk here was one concerning ongoing maintenance and support of said beast, as you stated.  However, the choice they faced was:

    1. taking ownership and responsibility for this task in-house
    2. outsourcing maintenance/support to a third party, opting for their solution instead.

    It sounds like they didn't analyse option (1) but decided upon (2) and went running to IBM.. that right? What message does that send to internal devs? "we don't trust you lot to develop and maintain some home brewed app"...?

    @Speakerphone Dude said:

    if someone brings in a solution to a known problem, they become a sponsor

    This at your place, or in general? I've seen some organisations where the creator automatically becomes the owner, but other places where there's some formalised transition process (deliverables handover).



  • @Cassidy said:

    The first stage in risk management is risk analysis - and the risk here was one concerning ongoing maintenance and support of said beast, as you stated.  However, the choice they faced was:

    1. taking ownership and responsibility for this task in-house
    2. outsourcing maintenance/support to a third party, opting for their solution instead.

    Yes, but option 2 was really:

    2.a Ignore the perfectly-suited-for-the-task custom system and buy a general-purpose* reporting system.
    2.b Believe the salesman when he says that the business users can maintain the system without bothering the IT department.
    2.c Train only one of said business users.
    2.d The user chosen is the most experienced but least senior person who couldn't avoid being ordered to go to the training.
    2.e After configuring the system, the trained person then retires.
    2.f Call the IT department** for any problems (even if the IT department is one guy who hauls around monitors and network cables.)
    2.g Have the developer who originally developed option 1 support the multi-tiered general-purpose* system, with no training or backup
    2.h C-level executives think they got a good deal because they also got a good golf shirt that says "Cognos" on it.



  • @Qwerty said:

    @Cassidy said:

    The first stage in risk management is risk analysis - and the risk here was one concerning ongoing maintenance and support of said beast, as you stated.  However, the choice they faced was:

    1. taking ownership and responsibility for this task in-house
    2. outsourcing maintenance/support to a third party, opting for their solution instead.

    Yes, but option 2 was really:

    2.a Ignore the perfectly-suited-for-the-task custom system and buy a general-purpose* reporting system.
    2.b Believe the salesman when he says that the business users can maintain the system without bothering the IT department.
    2.c Train only one of said business users.
    2.d The user chosen is the most experienced but least senior person who couldn't avoid being ordered to go to the training.
    2.e After configuring the system, the trained person then retires.
    2.f Call the IT department** for any problems (even if the IT department is one guy who hauls around monitors and network cables.)
    2.g Have the developer who originally developed option 1 support the multi-tiered general-purpose* system, with no training or backup
    2.h C-level executives think they got a good deal because they also got a good golf shirt that says "Cognos" on it.

    I see you have actual experience... Sounds like what is about to happen, except for one detail - it's senior management that will get Cognos golf shirts from the vendor. C-Level people will instead get iPads preloaded with a shortcut for the report URL (of course iPads will be bought with the IT budget).



  • @Speakerphone Dude said:

    I agree. As absurd as it seems to people stuck in small companies hurting for money, bottomless IT budget makes it difficult to promote grassroot innovation and leads to a culture of incompetence where people flock to vendor- or product-driven architecture.
     

    Wow I completley agree. Central IT were I work was incapable making the "WAMP server" connect to Oracle (were the W in WAMP is a WTF itself).  Even after a whole year and me sending them screenshots of every single step. We now have our own web server at local IT and I'm the admin of it. Oracle is mandatory because the product(s) we use requires it...But after all it's actually a good thing because now I'm in control and can release "web applications" (speak simple data entry and retrieval forms) without going through central IT. They would be incapable of this

    The we don't get access to our applications servers so and updates must be installed by the responsible tech and they are incapable of following simple written instructions.I once followed such an upgrade while he shared the screen of the server and I had to correct him multiple times even so he took about 10 times longer than me to process each instruction. At least that explained why every update of every applications always takes forever. before I always thought the software is shitty. I'm nto so sure anymore...

     



  • @beginner_ said:

    updates must be installed by the responsible tech and they are incapable of following simple written instructions
     

    When I get bottlenecks of the "requires skilled and qualified personnel to perform such operations" flavour, I tend to hang back and leave them with enough ethernet string to garotte themselves.

    If they're responsible, let them take responsibility. If they're not responsible enough, you're outside the blast radius.



  • @Speakerphone Dude said:

    someone suggested to use a "proven product" with "similar features" instead. So they decided to buy Cognos and agreed on a $80k budget to deploy it

    That's got to be the worst case of Reverse Not-Invented-Here Syndrome I've ever read about. Do your company's shareholders have a thing for burning money?



  • @Renan said:

    @Speakerphone Dude said:

    someone suggested to use a "proven product" with "similar features" instead. So they decided to buy Cognos and agreed on a $80k budget to deploy it

    That's got to be the worst case of Reverse Not-Invented-Here Syndrome I've ever read about. Do your company's shareholders have a thing for burning money?

    It's not my employer, it's my client. Big difference (especially for the paycheck) and since they hired me, I would suspect they do have a thing for burning money...


Log in to reply