So where is the balance between cowboy and waterfall?



  •  I'm asking you guys this seriously, because I've worked in both (but mostly in cowboy land); I can see the pros and cons for both, and I know there has to be a balance point in the middle somewhere but I don't know where.  When I was an engineer, all I cared about was doing my coding/testing/getting today's job done.  Now that I'm a BA for a team of developers, my job is managing processes to enable the devs to get their jobs done.  The one thing I'm struggling with is the sort of power-pull between principals of quality (which entails planning and good requirements that are documented, and controlling changes and scope creep) and cowboyness.  The cowboy pressure comes from two directions:  management pressure to deliver yesterday, and engineers who want to dive into coding, self-test, and not be bothered with anything they perceive as beaucracy.

    More specifically, here is what I have:  The culture is pretty nicely in the middle of the two extremes.  It's agile-like but they refuse to be pigeonholed into a methodology.  We have twice-a-month meetings with all the dev teams to quickly review our coming releases to make sure everybody is aware of potential impacts on each other's systems, security, finance, operations.  We have formal QA with a really good QA team.  At the beginning of each year, the release management team publishes the release schedule for the entire coming year.    That's the extent of our beaurocracy, everything else is left to our discretion.

    Here's what I struggle with:

    It's my job to keep QA informed of our projects and when we will drop code to them for testing, so that they can plan their work schedule.  Everybody understands this.  However, my team leads seem to be allergic to providing work estimates, so they can never tell me when we can drop to QA until the day before.  The best I've been able to do is to let QA know approximately which two-to-three-week period the drop may happen. One thing my manager requires of me is that I pull a report that compares task estimates to work logged.  The principal of this exercise is to help the dev leads improve their estimating abilities.  But the leads instead seem to feel resentful about it, they get really defensive when I review the report with them.  I'm not sure how to make them understand that it's not about questioning their expertise.  

    The other major thing I'm strugging with is one particular project that's gone so far out of control that it's not even in the Wild West, it's on Mars.  The web UI team, which was brought in from outside our company, somehow has more clout than the project manager, so he's completely set aside all project management activities in favor of full-time cat-herding.  He's dropped all attempts at scheduling, change control, risk monitoring, documenting requirements.  He does nothing to help us enforce the release schedule, QA practices and all that, so the web UI team has decided to dismiss all that hogwash.  They cut code in the morning and release it to production the same afternoon.  Because they work that way, and the project is a bunch of dev teams integrating systems, it's caused alot of pressure on the other teams to work the same way.  They've gotten QA scrambling to try to complete end to end testing AND regression testing in 1 day, with 1 day's advance notice.  They've pushed us to release code after that, and finish testing it in production.  The kicker is that the project manager is frustrated and instead of doing what he KNOWS will fix it, just rants at me.  He blamed me (or rather the dev teams) for not questioning the requirements, for missing requirements, for having to spend time in rework, etc.  He rants that he doesn't know why this keeps happening.  It's all I can do not to rant back:  YOU KNOW HOW TO FIX THIS, YOU JUST REFUSE TO FOR SOME REASON I DON'T UNDERSTAND.  THE SOLUTION IS TO ENFORCE THE PROJECT SCHEDULE, DEMAND DOCUMENTED REQUIREMENTS AND HAVE ALL THE TEAMS REVIEW AND SIGN OFF ON THEM, ENFORCE CHANGE CONTROL, ENFORCE QA.....

    And then in performing this rant in my head, I realize that what I'm saying is that the solution is to add beaurocracy.  I'm not saying we need to go waterfall, but we need some middle ground somewhere.  What I don't know is how to identify that middle ground.  Suggestions?



  • @jetcitywoman said:

    However, my team leads seem to be allergic to providing work estimates
     

     You have to make your programmers give estimates. They have to.

    They will get it wrong at first, and they will get it wrong a lot, and they will use this as an argument to support that estimating is bullshit. The reality is that they don't know how to do it, and it's a skill that takes time to learn.

    So until then, fudge the numbers to 150-200% of what they say.

    @jetcitywoman said:

    what I'm saying is that the solution is to add beaurocracy.

    Some is good. It's not possible to control a group of people without adding or promoting a person to the group's controller. There is no way around this.  After all, you yourself are now a cog in the bureaucratic machine.


  • 🚽 Regular

    One of the features of agile development is Scrum. Instead of asking your programmers to say "how long will X take?" try asking them, "Which tasks out of this prioritized list do you think can be done in two weeks?" It might sound like you're asking the same thing, but it gives them a different perspective of how they're going to schedule their work. I, personally, find that there's a bit less pressure (until the last two days of the sprint) and there's still some wiggle room for adjustments during the sprint. You're more able to gauge how well things are progressing when you have this relatively short timespan. If on the second week you find there's a few tasks that you realize is going to take more than a few days to complete, then you can relay those new expectations sooner than later.

    Obviously, as dhromed says, there can and will still be slip. The difference here is often times the slip is more in the form of missing features for the bi-weekly QA test. If, in two weeks, a programmer says he can get 6 tasks done but only gets 4 done, it's not necessarily the end of the world and you at least have something to provide the QA team with, and they won't feel as though their schedule is completely shot because you don't have to delay their timetable. There is, of course, a methodology involved, but if they abhor even this level of methodology, then if you're in the US perhaps you should take advantage of the high unemployment rate and find some more qualified engineers.



  • @dhromed said:

    They will get it wrong at first, and they will get it wrong a lot, and they will use this as an argument to support that estimating is bullshit. The reality is that they don't know how to do it, and it's a skill that takes time to learn.

    So until then, fudge the numbers to 150-200% of what they say.

     

    Interestingly the issue isn't that they provide estimates and then slip milestones - well, they do sometimes, but we're not that rigid that people get hung for it.  It's a pretty reasonable organization.  But as a quality exercise, one of the VP's (the one we report to) has asked the dev teams to compare their estimates to the logged work hours as a way to learn how to estimate better.  It's that report that they deeply resent.  Simply showing them that ticket 321 had an estimate of 43 hours but they logged 10 (or 500, it doesn't matter which way it goes) hours against the ticket gets them all in a huff.  

    What puzzles me is how people get so defensive about things.  Am I more mature than most, because that kind of thing doesn't make me defensive.  Sure, if somebody gets in my face and demands to know why I screwed up, I'll respond in kind.  But someone politely showing me that I made a mistake, isn't a problem.  I don't understand why I have to dance on eggshells, wearing kid gloves when dealing with most people.

     



  • @RHuckster said:

    Instead of asking your programmers to say "how long will X take?" try asking them, "Which tasks out of this prioritized list do you think can be done in two weeks?" It might sound like you're asking the same thing, but it gives them a different perspective of how they're going to schedule their work. I, personally, find that there's a bit less pressure (until the last two days of the sprint) and there's still some wiggle room for adjustments during the sprint. You're more able to gauge how well things are progressing when you have this relatively short timespan. If on the second week you find there's a few tasks that you realize is going to take more than a few days to complete, then you can relay those new expectations sooner than later.
     

    Hmm, that's interesting.  Thanks, I'll try this approach.  I have noticed that dev teams in the other end of the business (front of house vs back of house) tend to schedule bundles of work that are all tested and released as a bundle.  This allows them to tell business folks when they can provide deliverables, but most interestingly I've also seen them use it as a club.  "Sorry Mr. Late-to-the-table Manager, our bundle for this month is already full.  We can't work on your project until the next bundle releasing the following month."  I've been pondering why this works for them while when I try to tell business "we can fit your request in next month", they strong-arm me/my team into doing what they demand.

    I've also noticed that if I ask my dev lead for an LOE on a request, his answer is always "I'll have to think about it and get back to you", but occasionally I've asked him "can you get this done in this release" he'll say yes or no.  I don't quite trust him, though, because I don't know if he knows how much work the team can take on at once.  I hate to sound racist, but he's an ethnic person with a very soft persona - he knows his stuff, but perhaps because of the language barrier he comes across as hesitant and unsure.  I guess more time working with him will give me more trust in him.



  • @jetcitywoman said:

    Simply showing [their] hours against the ticket gets them all in a huff.  
     

    That is dumb and they should feel dumb. It's just programming. Could be either arrogance or fear of prosecution, depending on the person. Have you asked them directly one-on-one why they don't like it?

    Hell, maybe they have this super duper idea but they don't tell you because nobody ever listens to them.

     



  • @jetcitywoman said:

    What puzzles me is how people get so defensive about things.  Am I more mature than most, because that kind of thing doesn't make me defensive.  Sure, if somebody gets in my face and demands to know why I screwed up, I'll respond in kind.  But someone politely showing me that I made a mistake, isn't a problem.  I don't understand why I have to dance on eggshells, wearing kid gloves when dealing with most people.

    Yes, you are more mature than most. But then, given what you describe as your job, you sound like a senior manager, so that's probably to be expected - you've got more work experience to draw on! Actually, your job sounds kind of like what my dad does (sorry if I made you feel old! Don't worry, I'm still rather young!) - he's in "project governance" at a large corporation with its HQ here in Cincinnati - despite being in a director-level position and regularly hobnobbing with vice-presidents and other executives, he has very few direct subordinates; instead, he's more of an adviser to the various project managers in the company, checking up on their status to make sure they're following proper procedures and the like...



  • @jetcitywoman said:

    I've also noticed that if I ask my dev lead for an LOE on a request, his answer is always "I'll have to think about it and get back to you", but occasionally I've asked him "can you get this done in this release" he'll say yes or no.
     

    Those most removed from the business often do not understand the impact upon the business their tasks have. And stressing "it's important" often doesn't work, because the business uses this as a clout to artifically raise the priority of something that's been overlooked until it's too late.

    A technique I tried is "information exposure": I had a list of tasks drawn up on a whiteboard with numbers against them showing importance (priority, rather than urgency). When discussing additional tasks given to me, I asked the requester to add it to the board and either cross something out or assign it what they deem to be a priority, in the "grand scheme of things".

    Sometimes this raises questions about developers' priorities, or rather their way of prioritising tasks:  what happens if you ask some dev at random "what tasks are you working on, in order of importance?" Similarly, giving the task to a dev lead and discussing ROI should then put them in a position to decide upon it priority: "this could save us 10K in the first month alone, but only if performed this quarter: could this be scheduled, and when? Or should I tell the powers that be that you've more important tasks to work upon?"

    Hope that helps.



  • @dhromed said:

    @jetcitywoman said:

    Simply showing [their] hours against the ticket gets them all in a huff.  
     

    That is dumb and they should feel dumb. It's just programming. Could be either arrogance or fear of prosecution, depending on the person. Have you asked them directly one-on-one why they don't like it?

     

    That.

    Don't expect to be liked by those you manage, but expect them to engage and deliver what is expected of them as the business requires.

    Sometimes it could quite simply be the way the question is phrased. Always give people an escape route: explain it in terms of "I don't understand this" or "can you explain this" rather than "why the fuck do you think this?" (I know you don't - but there's some reason they're guarded, possibly due to the perceptions createdby the previous manager, and that baggage needs to be unloaded first).

     



  • @dhromed said:

    You have to make your programmers give estimates. They have to.

    They will get it wrong at first, and they will get it wrong a lot, and they will use this as an argument to support that estimating is bullshit. The reality is that they don't know how to do it, and it's a skill that takes time to learn.

    So until then, fudge the numbers to 150-200% of what they say.

    The rule that always seemed to work for me was to add four to whatever time estimate was given, then change the units to the next larger size.  So if they thought something would take ten minutes, you put the estimate down as fourteen hours.  Three days -> seven weeks.  Six months -> ten years.

    On the higher ranges, this has the added benefit of making many projects unfundable, so they get cancelled long before anyone has a chance to find out if the estimate was any good or not.

     



  • @da Doctah said:

    The rule that always seemed to work for me was to add four to whatever time estimate was given, then change the units to the next larger size.  So if they thought something would take ten minutes, you put the estimate down as fourteen hours.  Three days -> seven weeks.  Six months -> ten years.
     

    Deliberate padding is one of the project Deadly Sins.

    Estimates are over-inflated to build in unnecessary slack, leading to a higher project cost and very generous timescales.  If the project is signed off, this generous estimate finds its way back to someone assigned to the task, then Parkinson's Law takes over.

    The upshot is that someone - the customer - either ends up paying over the odds for the product due to the low net output level, or the supplier prices themselves out of the market because the customer has found another supplier with lower padding and therefore cheaper costs and shorter timescales.

    It's also one of the reasons IT has become a prima donna industry with the self-belief that society will happily pay high costs for low output. Actions like this can only serve to perpetuate that myth.



  • @Cassidy said:

    Parkinson's Law
     

    This law is true!



  • @Cassidy said:

    Sometimes it could quite simply be the way the question is phrased. Always give people an escape route: explain it in terms of "I don't understand this" or "can you explain this" rather than "why the fuck do you think this?" (I know you don't - but there's some reason they're guarded, possibly due to the perceptions createdby the previous manager, and that baggage needs to be unloaded first).
     

    Everybody's input is helpful, but this I think gets closer to the situation.  I've only been with this company for a year, in a culture where people tend to hang around for around 10 years, so I'm sure part of it is that I haven't "earned" their respect yet.  They know I have alot of experience as a developer, but that's not the same as seeing it demonstrated.  I have no problem with that. 

    Also, part of the issue is the way we break work into tasks that are logged on tickets.  Management likes to see neat monthly reports of the work completed for the month, but our projects (obviously as everybody's do) span months.  Work just doesn't start and end like that in the real world.   Also, before I started here, they made a "cheatsheet" of estimates they can use for all projects.  For example x type of project should take x (small estimate), y (medium estimate), and z (large estimate).  Some of the work types make sense to me, but some of them are esoteric breakdowns that I'd never in a million years know, like "setting config parameters for Pisby routines".  When they actually create tickets, do you think they include the words "setting config parameters for Pisby routines" in either the title or description?  Heck no.  (I tried to categorize the tickets into general software development work types like requirements, analysis, prototyping, development, qa support, production support, etc.  It was okay with one dev lead, but the others had a fit.

    I think what I need to do is make the dev leads run and analyze their own estimate/actual reports. That way they'll do it however they feel comfortable and however they feel is a valid way to analyze their work, so they'll own it.  The problem with this is that it's just that much more administrative work that they don't really have time for.  Or is it something you should just suck up and do when you're a lead?


  • ♿ (Parody)

    @jetcitywoman said:

    I think what I need to do is make the dev leads run and analyze their own estimate/actual reports. That way they'll do it however they feel comfortable and however they feel is a valid way to analyze their work, so they'll own it.  The problem with this is that it's just that much more administrative work that they don't really have time for.  Or is it something you should just suck up and do when you're a lead?

    Being a lead means that you've entered the realm of management. If you can't measure something, how can you know if you're improving? Or even if you need to improve? They hopefully understand this concept when it comes to optimizing code (i.e., profile!), so it's unfortunate that they view this as just more bureaucracy instead of a useful and necessary (at least for non-trivial projects) tool. This sort of thing should have been explained as part of the deal when they got the gig to be a leader. This sounds like a failure of project (and possibly line) management.

    We don't actually use time to estimate tasks, but "story points." It's kind of vague, but you can rate tasks as 0.5, 1, 2, 4 or 8 story points. I guess the translation to time is different for each person, but my rule of thumb is that a 0.5 story point is something like fixing a typo. A 1 is a day or less. A 2 is more than a day, but probably not more than a week. A 4 is a multi-week task. I don't think I've ever used 8 (and if I did, it's probably a good indication that the task needs to be broken down). I'm not the lead, but I know that the powers that be have a pretty good feeling for how many story points each person can get through in a single iteration (which is usually about a month, including testing).



  • @jetcitywoman said:

    I'm sure part of it is that I haven't "earned" their respect yet.  They know I have alot of experience as a developer, but that's not the same as seeing it demonstrated.  I have no problem with that.
     

    No problem because that's not what you're currently employed to do? Or no problem because you know you don't feel you have to prove anything?

    @jetcitywoman said:

    Management likes to see neat monthly reports of the work completed for the month, but our projects (obviously as everybody's do) span months.

    So have three metrics: number of tasks complete (closed) this month, number started (open) and number still in progress. Or switch to a more rapid Agile methodology where the work is broken into smaller packages so greater progress appears to be made. I think your challenge here won't be the content or structure of the reports, but the way in which they're received, perceived and interpreted. If management are using specific facts to guide their decision-making then it is possible to shape their opinions with creative reporting; I'm more mindful of removing unnecessary/inaccurate stuff from the report that could influence decisions due to misinterpretation.

    @jetcitywoman said:

    For example x type of project should take x (small estimate), y (medium estimate), and z (large estimate). 

    Question those figures: request evidence from prior projects that they have proven to be accurate, demand to know who arrived at those estimates and when they they last reviewed for appropriateness, who's currently using them now. I know if I were to adopt them as gospel truth then I'd need convincing first, and even then I would see those as general guidance but --

    @jetcitywoman said:

    I think what I need to do is make the dev leads run and analyze their own estimate/actual reports. That way they'll do it however they feel comfortable and however they feel is a valid way to analyze their work, so they'll own it.

    -- but that.

    @jetcitywoman said:

    The problem with this is that it's just that much more administrative work that they don't really have time for.  Or is it something you should just suck up and do when you're a lead?

    Don't let the fact that it's "too much admin work" means it shouldn't be done, just that at present the current facilities will mean an increased admin burden upon them.  Perhaps throwing them the problem of "what technology/processes could assist in the task of estimating?" may engage them better. Programmers won't like using tools that have been foisted upon them; if a few could arrive at some simple way of cutting down the admin work then it'll benefit both sides. It doesn't need to be perfect first off - it can be subject to improvements later - just something that gets the ball rolling and empowers the devs to capture and record their own estimates.

     Anyway, my thoughts from a moran. Dunno if they help or hinder.



  • @Cassidy said:

    @jetcitywoman said:

    I'm sure part of it is that I haven't "earned" their respect yet.  They know I have alot of experience as a developer, but that's not the same as seeing it demonstrated.  I have no problem with that.
     

    No problem because that's not what you're currently employed to do? Or no problem because you know you don't feel you have to prove anything?

    No problem because I know in a new company you have to prove yourself and earn people's respect.  Especially with techies.  Right?  That's all I meant.

    @Cassidy said:

    So have three metrics: number of tasks complete (closed) this month, number started (open) and number still in progress. Or switch to a more rapid Agile methodology where the work is broken into smaller packages so greater progress appears to be made. I think your challenge here won't be the content or structure of the reports, but the way in which they're received, perceived and interpreted. If management are using specific facts to guide their decision-making then it is possible to shape their opinions with creative reporting; I'm more mindful of removing unnecessary/inaccurate stuff from the report that could influence decisions due to misinterpretation.

    We have those metrics also.  But the LOE/actual comparison is one of the tools management thinks should be helpful to the dev leads.  Tell me about creative reporting.  I have noticed that when I have to go chase the leads down to update their ticket hours for the monthly reports, is when one of them does all of that admin work.  I mean he doesn't enter tickets for his work.... until it's time for the report.  Then he enters the ticket, logs the time, sets the estimated hours the same as the time logged, and closes the ticket.  I just haven't said anything about it yet, but I'm not stupid.


     


  • Discourse touched me in a no-no place

    @Cassidy said:

    Anyway, my thoughts from a moran.
    We're back to that Australian chef and Belgian detective aren't we?



  • @boomzilla said:

    you can rate tasks as 0.5, 1, 2, 4 or 8 story points.
     

    I've done that kind of doubling-estimation, except in hours/days:

    1, 2, 4, 8, two days, all week, several weeks.

    The main point is that you don't estimate in between. Nothing takes three hours, or six hours, or three-and-a-half days.



  • @Cassidy said:

    an increased admin burden upon them
     

    Progger's gonna prog. They want to make a plan and put it in code. They detest filling the cumbersome and awkwardly structure timesheet forms of your crappy timesheet software with the messy UI.

    Maybe I'm projecting, but I know I do.

     

    Oh another thing: Real Artists Ship.

    Sometimes people get together and try to make the perfect plan, but that's utterly impossible, and then they get stuck in this infinite loop of trying to tweak the plan. Often you just have to build a rough, flawed thing and use it for a while and then all its flaws — if any — will become apparent almost immediately.



  • @jetcitywoman said:

    Everybody's input is helpful
     

    Man, if TDWTF formed a company we'd be so successful: competently made products, delivered on time! We'd be an example for other It companies! A shining beacon in the night. Heck, I'd even buy Silentrunner a pair of noise-cancelling headphones to shut out all the fucks.



  • @dhromed said:

    Sometimes people get together and try to make the perfect plan, but that's utterly impossible, and then they get stuck in this infinite loop of trying to tweak the plan. Often you just have to build a rough, flawed thing and use it for a while and then all its flaws — if any — will become apparent almost immediately.

    Fred Brooks used to promote the idea, "make one to throw away". Basically, since your first attempt will be flawed no matter what you do, just put the inevitable rewrite into the schedule.



  •  Known these days as prototyping, right?  I like that approach also, but the problem is that management (universally) thinks that if we aren't brilliant enough to do it perfectly the first time out, we need to be fired.  I just finished reading Death March and in one of the later chapters he talks about simulators and war games.  The focus is more on IT projects rather than coding/development, but the principal is the same.  The only way to truly learn how to do something hard and compex is to simulate it - the only way to know how to get through a death march project is by practicing in a war game.  He points out that we (society) love that airline pilots train in simulators and if that ever stopped being part of their training we'd all stop flying.  It's odd that corporate management thinks that IT people can deliver business-critical systems correctly in one shot without ever having practiced - AND in half the time generally necessary (because they always want it delivered yesterday).



  • @jetcitywoman said:

    I mean he doesn't enter tickets for his work.... until it's time for the report.  Then he enters the ticket, logs the time, sets the estimated hours the same as the time logged, and closes the ticket.  I just haven't said anything about it yet, but I'm not stupid.
     

    Have you investigated those reasons? It could be down to two things:

    1. there's no easy way to do it, it's a barrier, it can be done later once the "important" stuff is done
    2. it's not important/compulsory, so it can go on a back-burner.

    There are ways of tackling 2, but it involves coercion and rigidity - such as making some information mandatory during a code commit - so a better method is looking at it from a (1) perspective: it should form part of the normal workflow process and feel natural to the user.

    @dhromed said:

    Progger's gonna prog. ...They detest filling the cumbersome and awkwardly structure timesheet forms of your crappy timesheet software with the messy UI.


    That's partly my point. If some middle ground can be found between the two that serves different audiences but can reuse the same action then it becomes natural. I'm mindful of JavaDocs in this regard.



  • @jetcitywoman said:

    I like that approach also, but the problem is that management (universally) thinks that if we aren't brilliant enough to do it perfectly the first time out, we need to be fired. 
     

    The reason why we did it brilliantly right the first time is because we had the opportunity to have a number of pretend goes at it before attempting the real thing! @jetcitywoman said:

    It's odd that corporate management thinks that IT people can deliver business-critical systems correctly in one shot without ever having practiced

    Delivering requirements and walking on water are similar: both are possible if  both are frozen. Or something.

     


Log in to reply