Tell me it's better at other companies



  • I've worked at my place long enough, and am familiar with the moronic requests from the end-users who usually don't realize what they truly want. I've learned to politely ask probing questions to minimize my headaches. But this most recent problem boggles my mind.

    Someone comes to me saying "Hey there, we have these query results we got from a government website, a bunch of HTML files. Can you convert them to DBF?"

    "Sure", says I. But I get the feeling this is not the entire scope of their issue. After looking at a few sample files, I begin my questions.

    "Do these have to be in DBF?"  They assure me that no other format will work. Not Access, not SQL Server...it MUST be DBF.  "What are they being used for?" For use with their software which is known to play very well with DBF files.

    No problem, I make a cute little script that reads in the HTML, finds the <table> of data, and makes a pretty DBF file. Which works fine for about 3 days.

    They come back saying "Now we need something that will merge the DBFs together". OK my mistake, I overlooked this possibility. Given the nature of their work I assumed they would want individual files for simplicity. So I make a second script that will take the DBFs and do some simple SELECT and INSERT statements to make one big DBF. This works for another 3 days.

    They come back reporting some errors to me. Apparently the query results don't necessarily produce the same column output every time. A few of the files have columns that no others have, and this crashes the script. I try to figure out what kind of a system produces these results, but it doesn't matter. I have to find a way around this. DBFs don't like to be changed once created, so I come up with an elaborate third script that ... well, I'll skip the ugliness and just say that it works.

    Not that they use it. Instead they prefer to use Excel to copy and paste data, create new columns, etc, and then act surprised when their systems don't like the resulting files. This works for another few days.

    They come back with a new error message. From what I can determine, their merged DBF has gotten too large, too many fields that all end up being CHAR(250) to potentially hold the rare text strings that push the limits of DBF. I found out how to make MEMO fields, but at this point it's a waste of time to have them go back and redo everything.

    And the icing on the cake... "Now we want to convert this DBF to an Access database". Two weeks of work, numerous rewrites (with some of the more annoying requests edited out to keep the story somewhat short), a handful of ugly scripts, all for the end result of a format that I could have converted the HTML to with no data loss or funky error messages.

    That thumping sound? It's just me, slamming my forehead into the desk.



  • In a word, no

    That's pretty much normal. Hope you knew what field you were getting into when you decided to work in it. (I try and warn people, but they very rarely listen)



  • Oh, it is better at other companies.

    At least... I've heard storys where it's better, I've not seen it for myself yet, but there are places out there...

    ...I hope; it's the only thought that keeps me going :/



  • If I had a dollar for each one of these types of situations I have been in here in just the 2 years I have been here. I would retire. Just to give a quick example I created a fairly simple report about 4 months after I started one year later i was on close to my 12 revision. The worst part is being spread over time like that you forget what is all in the report making it more difficult each time to dig back into it.



  • Pretty much sounds like every internal project i've ever worked on.

    So here's my short list of internal project musts

    1. Vague specs:  even a diagram scrawled on a napkin is too much, preferabling the specs are explained by someone in marketing who doesn't quite "get" computers

    2. Scope creep: After getting an initial version of the software up, which is to the spec(as far as you can tell), you then get heaps of feature requests once they start thinking of what they might actually need.  Also because it was initially a little "quick and dirty" application you now need to hack on all the features onto your "quick and dirty" framework making for some good WTFs.

    3. Integrating the App into critical systems:  After the 6th round of change requests the application now becomes a critical part of the business architecture, even though you were never told that would be the case.  And now because you wrote the horrible app you get to maintain it whenever anything goes wrong and the horrible hacked together code becomes the bane of your working life for the rest of your time at the company.

    I like to call these the 3 steps to success on any internal it project, i hope they help you out.

     


     



  • You're way ahead of me. I would have converted them to PDF.



  • Ah, good ol' design-by-debugging-the-developers-to-write-random-code-and-see-if-it-turns-out-to-be-what-we-want. 

    Also see: http://www.neopoleon.com/home/blogs/neo/archive/2003/09/29/5458.aspx

    They didn't try putting it on a wooden table before taking a photo of it, though.
     



  • At least at my company there's a 2-year cycle.  Keep this HTML-to-DBF-to-Access converter somewhere safe for 2 years and I guarantee someone else will come up to your desk with the same brilliant idea:

    "You'll have to go back to your own desk now."

    "Why, can't you do it?"

    "No, I've just emailed it to you."



  • Usually, you don't get the spec straight from the person who needs it, because that person is too important to work directly with the developer.  So not only is the original spec vague, but after it s communicated "telephone" style through three or four levels of management, it is borderline impossible.

     

    I'd say the final piece to your html to dbf to access converter should be to convert it back to html. 



  • @Manni said:

    Someone comes to me saying "Hey there, we have these query results we got from a government website, a bunch of HTML files. 

     

    I'm pretty sure that this website gives an extra option to save those queries to a csv file :o) 



  • Not only is this completely normal, but I think it is the most important part of our jobs.

    We just need to face it, we will always have changing requirements.  (I've been working on a project for about 5 months which is now in beta.  Last week, my boss's boss came to us and wanted make a fundamental change to the underlying data model that would have set the project back by at least a month.  Luckily we talked him into putting it off until version 2).  Learning the mental stamina and coding skill to deal with this aspect of software development accounts for 90% of the difficulty of creating software in the first place.

    It might not necessarily apply in this case, but for any programmers who are afraid of making changes to working software, check out the book Refactoring by Martin Fowler.  It is pretty ubiquitous - it might be lying around your office somewhere.  Really read (don't skim) the first two chapters.  That book has had a larger effect on the way I program day to day than any other programming resource.
     



  • It's the programmers axiom to Murphy's Law.

    It is always better at the other company, until you work for the other company, then the company you just left becomes the better of the two.



  • @element[0] said:

    Pretty much sounds like every internal project i've ever worked on.

    So here's my short list of internal project musts

    1. Vague specs:  even a diagram scrawled on a napkin is too much, preferabling the specs are explained by someone in marketing who doesn't quite "get" computers

    2. Scope creep: After getting an initial version of the software up, which is to the spec(as far as you can tell), you then get heaps of feature requests once they start thinking of what they might actually need.  Also because it was initially a little "quick and dirty" application you now need to hack on all the features onto your "quick and dirty" framework making for some good WTFs.

    3. Integrating the App into critical systems:  After the 6th round of change requests the application now becomes a critical part of the business architecture, even though you were never told that would be the case.  And now because you wrote the horrible app you get to maintain it whenever anything goes wrong and the horrible hacked together code becomes the bane of your working life for the rest of your time at the company.

    I like to call these the 3 steps to success on any internal it project, i hope they help you out.

     

     

     

    I agree.  Im working on one of those projects right now :-/

    Thankfully though i decided to rewrite the entire thing after the 5th or so time i needed to add new features because the code was a complete mess, now its manageable but im the only one that understands it enought to quickly update it. Its sad because i made this project as an intern with little experience with the language and now it runs on all of our clients sites.



  • @Salami said:

    I'm pretty sure that this website gives an extra option to save those queries to a csv file :o)

    Wayne Brady may have to choke a bitch.

     

    @Salami said:

    I'd say the final piece to your html to dbf to access converter should be to convert it back to html. 

    Ladies and gents, we have a thread winner. I guffawed at reading this one.



  • Don't take this badly, but...

    I think this type of thing happens at lots of companies, but it is a bit unfair to blame the users. I don't want to be too mean here, but it sure seems like you should have asked a lot more questions, especially of the "Why?" sort. At least the way you recounted the story, I saw several places where I wouldn't have settled for the answer you got.

     This looks like another example where the programmer didn't do the legwork to understand the problem before coding away. I understand there is normally a lot of pressure to get to work, and you normally get bad requirements, but getting clear requirements is YOUR responsibility not theirs. They are trying to do their job and are not technology professionals, so they can only see the problem through a very limited view of the world. You were the one that understood the consequences of the utility you were building and apparently didn't communicate them clearly.

    Think about it outside the world of programming and put yourself in the shoes of the person with a need and little technical knowledge.

     If you went to the doctor with an earache and insisted he give you antibiotic drops for your ear. The Doctor suspects a tumor, but has trouble explaining it to you without medical jargon so that you can understand. He gets frustrated and just gives you the ear drops because you are too stupid to "get it". Later when the tumor kills you, who's fault is it?



  • @element[0] said:

    Pretty much sounds like every internal project i've ever worked on.

    Yes.  It's the same everywhere.  The only variable is whether your boss is a good one, or a not good one.



  • jfuex, I did admit basically "whoops my bad, I should have asked more in-depth questions". You must have missed that. My point was that the one thing I tried to fully understand was what format they wanted for the output. I even got them to describe what they were ultimately using the data for.

    I've been burned before by wasting time on a task just to find out what I'm doing is unnecessary and the user didn't know any better. This is not one of those situations, because I got pretty conrete specs in that area, and after two weeks they changed their mind.

    Feel free to tell me how I can prevent users from being fickle in the future. But since there are 14 replies and only yours disagrees with me, I'll let the majority speak for itself.


  • Considered Harmful

    @Manni it's worse


  • Discourse touched me in a no-no place

    @Gribnit bravo


  • BINNED

    @Gribnit
    Soon


  • Considered Harmful

    @Luhmann said in Tell me it's better at other companies:

    @Gribnit
    Soon

    I'm still out of candy. I mean, I have some. But not for handing out.


Log in to reply