Having problems with unit testing philosophy.
-
@PleegWat said in Having problems with unit testing philosophy.:
@Unperverted-Vixen said in Having problems with unit testing philosophy.:
An integration test might have caught it, but I’ve yet to see automated integration tests actually touching a database out in the wild.
Half a hand for me too - we have internal integration tests that are hooked up to an in memory database, but not a complete mock, so things like misconfigured Hibernate mappings or invalid/incompletely validated queries do get tested. It's really useful sometimes.
-
@Gąska said in Having problems with unit testing philosophy.:
@xaade said in Having problems with unit testing philosophy.:
@Gąska said in Having problems with unit testing philosophy.:
what do clear-box integration tests give you that black-box unit tests don't?
Easy.
WHY are you calling the dependencies in this manner. That's what's missed by unit tests.
Can't you infer that from the input provided in public interfaces? If the logging module is retrieving user data from server #3, it's because it was told earlier that data resides in server #3, and it was told that because the test says so? Can you give me an example scenario where that wouldn't be sufficient and clear-box really becomes necessary?
I'm quite disappointed this was left unanswered. There's clearly something I'm missing here, and I knowing that I miss something but not knowing what.
-
@Unperverted-Vixen said in Having problems with unit testing philosophy.:
An integration test might have caught it, but I’ve yet to see automated integration tests actually touching a database out in the wild.
Not quite DB, but our integration tests use a real (locally run) instance of web server for one external service.
-
@xaade said in Having problems with unit testing philosophy.:
// given a screw and bolt with matching threads, can they connect?Well, usually a screw might join up with another screw, and a bolt might have a nut on it, but I've never heard of putting a screw and a bolt together.
-
@djls45
The nut is the person writing the unit tests
-
@izzion said in Having problems with unit testing philosophy.:
@djls45
The nut is the person writing the unit testsIt gets better.
I found a datautility that calls out to a webapi. There are 6 tests for it, for each method available. Except we're not testing what it returns, because that doesn't make sense.
Instead, the datautility calls a wrapper that calls the actual method for each method, and that wrapper shows a notification if the call fails. And we're testing to see if the notification is displayed for each method.
Why? Because We hAVe to wRITe UniT tESTs for every public method of every interface.
Please, begging here, WTF do I do?
-
@djls45 said in Having problems with unit testing philosophy.:
@xaade said in Having problems with unit testing philosophy.:
// given a screw and bolt with matching threads, can they connect?Well, usually a screw might join up with another screw, and a bolt might have a nut on it, but I've never heard of putting a screw and a bolt together.
SHIT....
You get what I mean though, right?
-
If I have class A that Jump() calls class B Jump() that calls class C Jump().
Right now, we have unit test like this
Test_A_Jump_Calls_B_Jump() Mock B A.Jump() Assert(B.Received().Jump()) Test_B_Jump_Calls_C_Jump() Mock C B.Jump() Assert(C.Received().Jump())Is this right? What's wrong with it? Is it necessary?
-
@boomzilla @Mason-Wheeler @apapadimoulis (forgot to @)
-
@Unperverted-Vixen said in Having problems with unit testing philosophy.:
@error said in Having problems with unit testing philosophy.:
A unit test would have told me about that.
Would it? Normally I’d expect anything touching a database to be mocked for a unit test.
An integration test might have caught it, but I’ve yet to see automated integration tests actually touching a database out in the wild.
Ours do. We'd never get anything done if we had to mock everything. Plus, how do you test your queries if you're not hitting a database?
-
@boomzilla said in Having problems with unit testing philosophy.:
Plus, how do you test your queries if you're not hitting a database?
Isn't there a way to make a local db for the unit tests? I thought that was a thing.
-
@xaade said in Having problems with unit testing philosophy.:
If I have class A that Jump() calls class B Jump() that calls class C Jump().
Right now, we have unit test like this
Test_A_Jump_Calls_B_Jump() Mock B A.Jump() Assert(B.Received().Jump()) Test_B_Jump_Calls_C_Jump() Mock C B.Jump() Assert(C.Received().Jump())Is this right? What's wrong with it? Is it necessary?
If the side effect is that it calls some external thing, then it's OK.
-
@boomzilla said in Having problems with unit testing philosophy.:
If the side effect is that it calls some external thing, then it's OK
Do you mean it calls something outside the A->B->C heirarchy?
-
Also, if there's no other side effects, can you not mock b, mock c, and have one unit test that checks if A calls C?
-
@xaade said in Having problems with unit testing philosophy.:
@boomzilla said in Having problems with unit testing philosophy.:
Plus, how do you test your queries if you're not hitting a database?
Isn't there a way to make a local db for the unit tests? I thought that was a thing.
We actually have 3 separate PDBs for running automated tests. It's "local" in that it's in our test environment. We periodically put (scrubbed) production data on there. We do the same in development. Every developer has their own "private" DB.
-
@xaade said in Having problems with unit testing philosophy.:
@boomzilla said in Having problems with unit testing philosophy.:
If the side effect is that it calls some external thing, then it's OK
Do you mean it calls something outside the A->B->C heirarchy?
Uh...no? I meant, "The expected side effect of B is that it calls C."
@xaade said in Having problems with unit testing philosophy.:
Also, if there's no other side effects, can you not mock b, mock c, and have one unit test that checks if A calls C?
Personally I would fart in their general direction and call it a day. I'm not following you, but I suspect your best course of action at this point is to stop fighting stupid and go along to get along.
-
@xaade said in Having problems with unit testing philosophy.:
@boomzilla said in Having problems with unit testing philosophy.:
Plus, how do you test your queries if you're not hitting a database?
Isn't there a way to make a local db for the unit tests? I thought that was a thing.
With our dev/autotest setup code, deploying the product including a local oracle database takes about 20 minutes. Schema and contents (needed for each individual integration test) is another minute or so. Most of the tests simulate collection of data and then use normal processing; we generally don't use seed data.
When running specific tests in dev I can use my dev DB instance so I just need the schema.
-
@PleegWat said in Having problems with unit testing philosophy.:
@xaade said in Having problems with unit testing philosophy.:
@boomzilla said in Having problems with unit testing philosophy.:
Plus, how do you test your queries if you're not hitting a database?
Isn't there a way to make a local db for the unit tests? I thought that was a thing.
With our dev/autotest setup code, deploying the product including a local oracle database takes about 20 minutes. Schema and contents (needed for each individual integration test) is another minute or so. Most of the tests simulate collection of data and then use normal processing; we generally don't use seed data.
When running specific tests in dev I can use my dev DB instance so I just need the schema.
When I first started programming, it took hours to get through compile and testing of the first product I worked on. 30 mins isn't so bad.
When some of the people I worked with started, it took overnight to compile at all.
-
@xaade Compile's about 16 minutes, to build both dev and release binaries. But that's managed centrally - When I load latest changes I also get the corresponding binaries so I just need to compile my own changes. That 20 minutes setup time is mostly configuring the DB (I think - I'd have to look it up). I hardly ever need to redo that.
The whole system with central compilation was designed for much larger projects. I think I heard that one of them, probably 15 to 20 years ago, took 26 hours to build.
-
@xaade said in Having problems with unit testing philosophy.:
Is this right?
All I can say is that it's not wrong.
Is it necessary?
Depends. Are classes A, B and C closely related, so much that they can be treated like a single unit? Like, A is the only class thst uses B, and B is the only class that uses C? If so, then units tests should test them all at once, becsuse that's the unit. "A class is a unit" is a rough guideline, not an absolute rule.
As an example, one time I've made a state machine for a communication protocol, where every state was a separate class. When I wrote the unit tests, I wrote them for the entire state machine at once, not individual states. I didn't test if particular states move to other particular states - I just tested if sending a particular sequence of messages produces the right responses at each step. But the message-sending thingy was fully mocked.
-
@boomzilla said in Having problems with unit testing philosophy.:
Plus, how do you test your queries if you're not hitting a database?
Not in an automated fashion.
-
@xaade said in Having problems with unit testing philosophy.:
You get what I mean though, right?
Yeah, I understood, hence the
.It sounds like also fits quite well with the sort of thing you're describing, so it's fine that way, too. :)
-
@xaade said in Having problems with unit testing philosophy.:
Is this right? What's wrong with it? Is it necessary?
What's wrong with it, hmm let's see... you have classes named A, B, C. Methods named Jump and Received.
But you won't get any good/definitive advice on that, because there's not nearly enough context. What you shared was way too abstract, and if you provided the full context, the answer would most likely not be WTF!? It'd be "that's not ideal, and it could be improved as such". But this is true about nearly everything.
I did my best to give that context in the
TestElementInnerTextexample, but some of those other classes I wanted to show require a lot more explanation.@boomzilla said in Having problems with unit testing philosophy.:
I suspect your best course of action at this point is to stop fighting stupid and go along to get along.
Well the best course of action is to hire me as a consultant. My rates are surprisingly affordable.
-
@apapadimoulis said in Having problems with unit testing philosophy.:
But this is true about nearly everything.
And particularly code that apapadimoulis guy wrote two years ago?
-
@apapadimoulis said in Having problems with unit testing philosophy.:
Well the best course of action is to hire me as a consultant. My rates are surprisingly affordable.
Best for you, definitely.
-
@PleegWat obviously that was the old version, and I disavow everything The Alex of 2018 said. v2020 is the latest and greatest.
@dkf are you implying that "best" and "best for Alex" are different concepts?
-
@Gąska said in Having problems with unit testing philosophy.:
"A class is a unit" is a rough guideline, not an absolute rule.
I don't think you should follow any guideline other than "Is this behavior part of a tangible requirement?" Classes and methods are mostly meaningless implementation details.
-
@xaade said in Having problems with unit testing philosophy.:
When I first started programming, it took hours to get through compile and testing of the first product I worked on. 30 mins isn't so bad.
I worked at one place that a typical full build took 6 hours. To completely build all dependencies was around 12 hours. On a beefy build server.
Thankfully, I worked on a smaller subsystem so I could just use the prebuilt stuff, so my builds were much more manageable.
-
@PleegWat said in Having problems with unit testing philosophy.:
Last week, I had a test (or actually most of them) identify a broken source control system.
Was it Perforce? I love how it has a field named “filetype” with the possible values including “text”, “unicode” and “utf8” and it controls whether the file is auto-prepended with a BOM during certain operations. When someone checked in a binary resource file, and Perforce wrongly guessed that it’s a text file, it resulted in a mysterious production bug costing months to track down.
-
@marczellm It's an in-house system, which has never been released externally. The problem had to do with adding a directory which had been deleted a few months prior, and caused old files to re-appear during nightly processing.
-
@xaade For an orchestrating unit like a top-level business service it can make sense to verify that it makes some specific calls. For a confined-scope unit like a controller it can make sense to verify even that it does not make other than certain calls. For other code behavioral testing gets useless sooner. In-out testing is the default for pure functions, not all units are pure functions.
Your manager does not understand this better than you, unfortunately.
