QA Team



  • I was going to hijack the "Urgent" thread in sidebar, but this seemed like a bette rplace to get answers.  I'm seeing a lot of insistence that absolutely any software shop should have a QA department.  But how big do you have to be before that applies?  If you're a lean startup with a team of two or three developers, would you fire a developer to make room in your budget for a QA person?  Or are startups somehow TRWTF?


  • ♿ (Parody)

    An insistence that "absolutely any X should have Y" is usually wrong.

    It depends on a lot of factors, but most importantly, what's the risk of shipping a defect, and how quickly can you identify, fix, and ship a fix? All a QA person will do is reduce the chance that you'll have a defect. That's it.

    At Inedo, we focus on defect mitigation before coding. Good tools and good design (loose coupling, high cohesion) reduce the chances magnitudally more than a QA person, but both are important as your team grows. For us, it's not so big of a deal if there's a bug -- there's usually a workaround, and if not, we can fix it super fast or our users can downgrade/rollback. Take a look at our release dates:

    In April, for BuildMaster, we shipped four releases within three weeks (two were on the same day).

    So, unless you're doing critical software where defects would kill it, then skip the QA and focus on good design.



  • You don't need a QA department.

    You need QA.



  • There are very talented, intelligent people who wouldn't mind spending their career in QA who can't, because they quickly hit the ceiling and there's no future in it.

    Most shops with QA departments try to change this by making the position more focused on tools-based QA ("QA in test" or whatever term they use for that) with the intent that talented QA people move on to developer positions. Personally I think that's a terrible idea, analogous to moving talented developers into management positions.

    To answer your actual question:

    @sprained said:

    I'm seeing a lot of insistence that absolutely any software shop should have a QA department.

    It doesn't need to be a department.

    @sprained said:

    But how big do you have to be before that applies? If you're a lean startup with a team of two or three developers, would you fire a developer to make room in your budget for a QA person?

    It depends on what you're doing. Some products need more people QAing data than they need coders. (Think about the problem Apple's having because nobody QAed their imported map data, for example.) Some products need a large amount of QA to receive some certification. (Think about medical devices, or the software under your car's hood.) For some products, the developers are already a small minority, and the majority of work needing to be QAed isn't code. (Think about a content-heavy video game made with an off-the-shelf engine.)

    So the answer as always with all advice ever is, "it depends".

    Would I fire one of those developers to free-up a QA position? Probably not. But I can't answer that without making 40,000 assumptions about what your three developers are actually doing and what product your company is making.

    @sprained said:

    Or are startups somehow TRWTF?

    The point of advice isn't to give you a hard-and-fast rule you must follow on penalty of death. Advice is just advice. Digest it and figure out for yourself whether it applies to your decision or not.



  • @blakeyrat said:

    So the answer as always with all advice ever is, "it depends".

    Would I fire one of those developers to free-up a QA position? Probably not. But I can't answer that without making 40,000 assumptions about what your three developers are actually doing and what product your company is making.

    @sprained said:

    Or are startups somehow TRWTF?

    The point of advice isn't to give you a hard-and-fast rule you must follow on penalty of death. Advice is just advice. Digest it and figure out for yourself whether it applies to your decision or not.

     

    Well, of course.  I was just intrigued by the extremely intense reaction on the other thread -- people jumped on the lack of a QA team without knowing whether the OP was in a three-man shop or a company that employs 100 developers.  

    Working in a fast-growing startup, I've been thinking about at what point we add more procedure and what resources we devote to that: procedure (be it QA or a formal code review process or even a formal process for management approval of ticket completion in the issue tracker) adds overhead, but as a team grows it also becomes necessary to avoid bedlam.  A WTF at a megacorp is just the obvious or only way to do things at a small company.

     

    Is pretty much everyone here from a large-corporate background? 

    I guess I was sort of asking: Where is that line in the sand between procedure and excessive bureaucracy? How does that move as a company grows? Does small size disadvantage the company that can't afford a QA tech?  What are some ways to handle a QA process on a small team?

     

     



  • @sprained said:

    But how big do you have to be before that applies?
     

    I think the real question ought to be "how many times do we release untested products then watch our (reputation | bank balance | income) suffer before we do anything about it?"

    @sprained said:

    Or are startups somehow TRWTF?

    No. Any organisation that does QA has typically been burned in the past and is determined to prevent a repeat. Startups may not have the organisational maturity to understand the importance of QA, or may simply not budget for it at the moment and hope for the best. Some startups are lucky enough to get away with it, too, due to the diligence of their devs performing a good level of QA themselves.

    The discussions over in the other thread are related to who actually performs the QA (same developer, different developer, whole dedicated department, independent QA group). As blakey pointed out, "it depends": the amount of - and expense afforded to - QA is usually related to the business criticality (and maybe safety-criticality) of the product. More QA would be performed on health & safety kit due to the risk to life, for instance.



  • @sprained said:

    Well, of course. I was just intrigued by the extremely intense reaction on the other thread -- people jumped on the lack of a QA team without knowing whether the OP was in a three-man shop or a company that employs 100 developers.

    I think you missed my examples. If you're a 3-man shop making software for a medical device, and you don't have a QA person responsible for navigating you through the certification programs for that device, your company is an idiot. So there are cases where a 3-man shop should 100% certainly definitely no-arguments have a QA person, and if they can't afford to pay more than 3 people they should fire one of the existing developers to create the QA position.

    Basically put, QA is required based on the activity the company is performing, not the number of employees it has.

    @sprained said:

    A WTF at a megacorp is just the obvious or only way to do things at a small company.

    I disagree. There's no reason your small 3-man team couldn't do the exact same type of processes that the megacorp is doing.

    @sprained said:

    Is pretty much everyone here from a large-corporate background?

    I usually work in medium sized non-development-centric companies. For example, in my last job I was working in an ad agency with about 600 employees-- but while there I was working as a sole developer on a very silo-ed product. Currently, I'm working at a web analytics company without about 500 employees, but there are only 3 developers on my particular team. Neither of these teams have had any kind of process put in place, so the process is just what I can convince people on the team to do.

    @sprained said:

    I guess I was sort of asking: Where is that line in the sand between procedure and excessive bureaucracy?

    QA doesn't have to mean "bureaucracy". I think you have some false assumptions here...

    @sprained said:

    How does that move as a company grows?

    Have you read the Mythical Man-Month? Do. It covers this.

    @sprained said:

    Does small size disadvantage the company that can't afford a QA tech?

    If you're creating a truly new and original product, it probably doesn't matter if you don't QA it. Look at Tumblr or Twitter for examples-- their products were buggy as shit for years. (I would argue Tumblr still is.)

    If you're catering to a specific and insular clientele, it probably doesn't matter if you don't QA it. If your site is the only one on the web catering to, I dunno, clown porn, people into clown porn will come to your site even if it's buggy.

    If you're competing with established products, and theirs are buggy, like most open source projects, you only need to be less-buggy. If you're competing with the Microsoft, you're in a world of hurt because Microsoft does everything right.

    @sprained said:

    What are some ways to handle a QA process on a small team?

    The number one easiest one you can do right away is to eat your own dogfood.



  • @blakeyrat said:

    @sprained said:
    Well, of course. I was just intrigued by the extremely intense reaction on the other thread -- people jumped on the lack of a QA team without knowing whether the OP was in a three-man shop or a company that employs 100 developers.
    I think you missed my examples. If you're a 3-man shop making software for a medical device, and you don't have a QA person responsible for navigating you through the certification programs for that device, your company is an idiot.

    Another point to keep in mind is that in smaller shops having people doing more than one thing is common.  Dev skills are different from QA skills, but that doesn't mean that devs couldn't be doing QA work.  Of course, how much QA you need to be doing depends on what you are building so we can't say if the right load is one dev spending 10% of their time on it or all spending 50%.



  • @blakeyrat said:

    you're in a world of hurt because Microsoft does everything right.
     

    Oh god this is just begging to be quoted out of context.

     

    ...whoops



  • My coworker asked me to install and run a standalone application he'd built. Supposedly all done.

    5 minutes in we found a glaring issue.

    That's QA as well, and it used exactly 2 developers and 0 dedicated departments.



  • @dhromed said:

    My coworker asked me to install and run a standalone application he'd built. Supposedly all done.

    5 minutes in we found a glaring issue.

    That's QA as well, and it used exactly 2 developers and 0 dedicated departments.

    +1, x10, etc.. As alluded to over in the other thread, testing your own code, while helpful, is marginal QA and should never be your only form of QA unless you are a one-man shop. Testing another developer's code is fair QA, ideal for your 3-man team working on a web app. Having a dedicated QA guy is good QA, ideal for a decent sized dev team. Having a dedicated QA team is excellent QA, ideal for an enterprise or sensitive projects like medical devices, power plants, weapons, car navigation, etc. (Assuming the QA guy/team is competent!)



  • The comments I made in the other thread were based on my experiences in the industry. Maybe I used a bit of hyperbole, but so far, most non-trivial changes to the project I work on have had at least one bug. And, that includes my whole team, not just me. That doesn't mean that we're bad coders; far from it. It just means that edge cases can creep up where you don't expect them, because you "know" what you wrote.

    Anyway, my company is a full-blown web-site media shop. We have an actual QA department, but they don't just test software. They review Emails that we produce for our clients in roughly 700 different email clients, they look at data pulls we do, they write and execute automated front-end testing (not unit testing, though, that's in our domain). And, of course, test our code for flaws.

    Our QA department is about 10 people, compared to about 20 App Dev* and about 15 Web Dev**. My team specifically is three people, and we generally have 1-2 QA working with us, which is usually more than sufficient.

    * - Application Development, we do the backend programming in C#, SQL, etc.

    ** - Web Development, they do most of the front-end HTML, CSS, etc.

    <plug type="shamelss">If you happen to be in the Greater Toronto Area and are looking for a career in one of these departments, it would be worth your while to drop me a PM, because we are always looking for new talent.</plug>


Log in to reply