Senile Argyle Architect
-
The other day I've met a guy who is a kind of senior software architect at WTFCorp (where I happened to work, too). The guy believes in unit testing and Agile practices so much that he seriously thinks that test automation by QA are a duplication of effort and consequently a waste of time, that all testing is really doable by developers, and the QA should confine themselves to writing basic smoke tests and do "exploratory testing" (i. e., manually).
Why? Because unit tests run so fast!
Never mind that our QA department has caught quite a few serious bugs while all unit tests were green.
Never you mind, too, that the development on that particular project only does isolated unit tests, no integration tests at all.
Never you mind that the guy isn't even involved in the project, it's just that a few bigger wigs think his opinion is รผber important...
-
Our team took that path around four years ago and we are still suffering.
Now at each release we need to put the software live into our internal production system for ~3 months and have 4-7 release candidates to address the defects that "testing" did not catch.
That also means that for each release the entire dev team are on regression / defect fixing for that 3 month period.Thankfully, it is now finally seeping into the minds of the gods that testing is important and we are rebuilding the QA department and investing in more automated testing, including integration tests.
"Agile" is great in many respects but it can be used by the unwise to make all sorts of stupid promises to management.
-
How can a Senior Architect think like that? Did he learn nothing? Did he get the job because his CV looked so cool with all his knowledge about open source Javascript frameworks?
If you contract Senior and Agile, you get Senile. Seems appropriate here.
-
so cool with all his knowledge about open source Javascript frameworks?
No, not a hipstery type, over a decade into enterprise programming, all that shit.
-
enterprise programming
That's the style of programming where you care far more about having every development process i dotted and every t crossed and every procedural box ticked than you do about whether the software actually works, yes?
-
The guy believes in unit testing and Agile practices so much that he seriously thinks that test automation by QA are a duplication of effort and consequently a waste of time, that all testing is really doable by developers, and the QA should confine themselves to writing basic smoke tests and do "exploratory testing" (i. e., manually).
Why? Because unit tests run so fast!
Never mind that our QA department has caught quite a few serious bugs while all unit tests were green.
Never you mind, too, that the development on that particular project only does isolated unit tests, no integration tests at all.
Someone needs to slap him upside the head with a Clue-By-Four.
Unit thwack tests thwack are thwack not thwack integration thwack tests. thwack
Any non-trivial system needs unit tests, block tests (module-level functional testing), function tests (for the entire process) and system testing (fully integrated with the rest of its running environment.)
Anyone who thinks you can safely ignore higher level tests just because the low-level tests pass is a fool.
This has nothing whatsoever to do with Agile. Agile simply determines when the tests will be defined and executed and who will define and execute them. It does not eliminate the need for testing. As a matter of fact, Agile practices usually require more testing, and require that all tests be automated for regression-testing purposes.
-
Unit tests are good for verifying that the code does what the developers think the code should do.
QA is good for verifying that what the developers think the code should do matches what the system is supposed to do.
-
-
Really good QA is good for mimicing eratic user behaviour and trying to use it in all the ways the system is never designed for.
I always admire their creativity. /bittersmile
-
If you contract Senior and Agile, you get Senile
In fact when I first saw this thread's title, I initially read it as "Senile Architect"... Probably my expectations of TDWTF combined with my lack of sleep showing.
-
-
Really good QA is good for mimicing eratic user behaviour and trying to use it in all the ways the system is never designed for.
I always admire their creativity. /bittersmile
I once worked with a QA guy who would try things like entering "cat" for a time limit. We really wanted to make the error message for that to be "You can't do that, Bob."
-
INB4: A guy walks into a bar, orders a beer...
-
I once worked with a QA guy who would try things like entering "cat" for a time limit. We really wanted to make the error message for that to be "You can't do that, Bob."
I had a BA testing some code I wrote once--I needed a name and some message text somewhere, so I used Smitty Werben Jaeger Manjansen for the name and "He was number one!" as the text.
-
Unit tests are good for verifying that the code does what the developers think the code should do.
QA is good for verifying that what the developers think the code should do matches what the system is supposed to do.
I agree with the first part, but not so much the second. While "functional verification" can be done by QA, "Classic" QA skill sets can not "assure quality".
As a general rule, ANY failure at a system level test is a defect in the testing process prior to that, so the first step is to create a test earlier in the pipeline (as early as possible) and then to review testing in that area to see of there are other potential holes in the current test plan.
-
As a general rule, ANY failure at a system level test is a defect in the testing process prior to that, so the first step is to create a test earlier in the pipeline (as early as possible) and then to review testing in that area to see of there are other potential holes in the current test plan.
Just to be sure that I am understanding what you are saying. You are stating that if a test at the highest level fails, you should go back and determine where in the pipeline you could place a test to catch this issue?
-
Just to be sure that I am understanding what you are saying. You are stating that if a test at the highest level fails, you should go back and determine where in the pipeline you could place a test to catch this issue?
If you have an established, healthy stack of tests that test your code on all levels (units, class integration, black-box service tests, database tests, front-end tests, etc), this is precisely what you should do, as it would surely help pinpoint the exact source of the bug and understand why it's happening. This also means your suite is lacking.However, it is possible that the bug will still be reproducible only on the system level, because of wacky timing/PRNG deficiencies on that particular platform/race conditions/confounding factors (which require defensive programming and not much else can be done).
If your stack is missing component integration tests, and filling in that gap requires implementing a whole new layer of testing that was never there in the first place, and you have budgets and deadlines to meet, that might be not a good decision, though.
-
While "functional verification" can be done by QA, "Classic" QA skill sets can not "assure quality".
How does this sentence relate to:
QA is good for verifying that what the developers think the code should do matches what the system is supposed to do.
? QA is absolutely good at validating that what was built matches a spec, given a good spec. It should be good at understanding what was desired at the beginning of a project and validating that, at the end of the project, what was built fills the need that was explained. Often, when working on a project, developers can get a bit myopic about what they're actually meant to be building.
-
Thanks to whoever changed the topic title for resolving my cognitive dissonance
-
I was going to make the same point, which is why I wanted to make sure I understood what he was saying.
-
INB4: A guy walks into a bar, orders a beer...
A QA guy walks into the bar before him...
-
QA is absolutely good at validating that what was built matches a spec, given a good spec
There is never a good spec......
Consider a project where the development team creates a solution for a large application. During load testing, it is determined that 15 servers (load balanced) will be needed to handle the anticipated (with growth) volume..... a pretty simple and common scenario.
Now, how is a "classic QA": person going to "assure the quality" to the degree that design or implementation differences could have allowed for a result where 10 servers (or 11,12,13,14) would have met the need???
The "functional part" is what maps to real world specifications, and that I conceded in my original post.
If you look back to hardware/engineering, there were historically two organizations QC [which ran tests, including, but not limited to functional tests] and QA which had a much broader spectrum. At many companies the people in QA were more knowledgeable across a wider base than the people in engineering.
-
I think you're vastly overloading the word "validate". Validation is what SQA departments are good at. Validation is not the same activity as creating the spec to validate against. If you say "The application must serve 5k users on 10 servers" and instead it falls over when the load test hits 4k users, it fails validation. What we're not meant to be doing is deciding that actually, 12 servers is fine, or better yet, 15.
I don't disagree with what you're saying, but I have no idea how you think that relates to the idea that QA can take what the system is supposed to do, from a business perspective, and verify that yes, the developers built that system. That's the easy part.
-
but I have no idea how you think that relates to the idea that QA can take what the system is supposed to do, from a business perspective, and verify that yes, the developers built that system. That's the easy part
It is very much a transformative process given the current state of most software development environments. But remember this was fairly normal in hardware engineering going back to the 1960's and 1970's.
"what the system is supposed to do, from a business perspective" is simple. It is to meet a set of needs in a manner that generates the maximum (weighted) return on investment. It is not about "the software does what a specification says".
-
There is never a
goodspec...... just a bunch of people telling you what they'd like the app to do today, which might contradict what they said yesterday, what the other people said today, or the laws of physics...sigh.
-
@TheCPUWizard said:
There is never a spec...... just a bunch of people telling you what they'd like the app to do today, which might contradict what they said yesterday, what the other people said today, or the laws of physics
...sigh.
bug 123: Widgets don't get frobnicated when baz is true! How could they not be getting frobnicated?! Customers are calling to complain!
bug 124: A Widget got frobnicated (I think baz was true?)! How could they be getting frobnicated?! Customers are calling to complain!double sigh...
Filed under: I wish the bug numbers being so close together wasn't the only exaggeration...
-
@Vault_Dweller said:
INB4: A guy walks into a bar, orders a beer...
A QA guy walks into the bar before him...
A Senior Application Developer walks into a bar....
[spoiler]OUCH![/spoiler]
-
@TheCPUWizard said:
There is never a
goodspec...... just a bunch of people telling you what they'd like the app to do today, which might contradict what they said yesterday, what the other people said today, or the laws of physics...sigh.
You forgot "What they'll claim to have said today, tomorrow."
-
@DCRoss said:
@Vault_Dweller said:
INB4: A guy walks into a bar, orders a beer...
A QA guy walks into the bar before him...
A Senior Application Developer walks into a bar....
[spoiler]OUCH![/spoiler]
...and forgets to drop a "how shitty my day is" macro?
-
@accalia said:
@DCRoss said:
@Vault_Dweller said:
INB4: A guy walks into a bar, orders a beer...
A QA guy walks into the bar before him...
A Senior Application Developer walks into a bar....
[spoiler]OUCH![/spoiler]
...and forgets to drop a "how shitty my day is" macro?
-
Let's see if you can figure out what I meant, Drax.
-
-
Yes, exactly.
Hey, everybody, I think I found @blakeyrat's alt!
-
Hey, everybody, I think I found @blakeyrat's alt!
-
@FrostCat said:
Hey, everybody, I think I found @blakeyrat's alt!
That in no way disproves what I said! In fact, it sounds like an image he might use, if it weren't so anime-y.
HTH, HAND, YMMV, etc.
-
bug 124: A Widget got frobnicated (I think baz was true?)! How could they be getting frobnicated?! Customers are calling to complain!
As if you'd ever get any useful technical detail in your bug reports... I think you mean:bug 124: Widgets are broken! Customers are calling to complain!
-
I once worked with a QA guy who would try things like entering "cat" for a time limit.
You mean actual QA work to make sure that bollox doesn't get into the logic layer and fuck the application over. God fucking forbid. The tester probably did that because it happened once and development can't be trusted to not fuck up validation of data.We really wanted to make the error message for that to be "You can't do that, Bob."
I actually included an exception once that with text "Tester name is about to delay another release". I felt like a prophet when it turned out to be true. Good QA is awesome.Hey, everybody, I think I found @blakeyrat's alt!
I'm still positive it's you.
-
We really wanted to make the error message for that to be "You can't do that, Bob."
I always wanted to have an error like the one in Hitchhiker's Guide To The Galaxy. When you push the button, a sign lights up saying "please don't push this button again".
The only problem was that I couldn't think of what it would do if you did push it again.
-
The only problem was that I couldn't think of what it would do if you did push it again.
one of three things:
- Repeat the request,
- issue a sharp electric shock to the button presser
- initiate self destruct
- launch all the missiles!
- counting is hard!
-
@David_C said:
The only problem was that I couldn't think of what it would do if you did push it again.
- initiate self destruct
Thank you for pressing the self destruct button. This ship will self destruct in three minutes.
-
ERR_NO_FOX_FOUND: User's claim to be @accalia rejected. Suspect account hacked or puppetmaster fracked-up.
:P
-
@accalia said:
ERR_NO_FOX_FOUND: User's claim to be @accalia rejected. Suspect account hacked or puppetmaster fracked-up.
:P
-
@abarker said:
@accalia said:
ERR_NO_FOX_FOUND: User's claim to be @accalia rejected. Suspect account hacked or puppetmaster fracked-up.
:P
See, that's what I was talking about!
-
@accalia said:
@abarker said:
@accalia said:
ERR_NO_FOX_FOUND: User's claim to be @accalia rejected. Suspect account hacked or puppetmaster fracked-up.
:P
See, that's what I was talking about!
-
-
-
-
-
Does someone need
-