Unit testing - where do I start?


  • kills Dumbledore

    I'm trying to be a good boy and unit test all the things. Unfortunately, I didn't have time to add tests while writing the project, it had to be out of the door ASAP, with nice-to-haves after delivery. So what I have is a C# project which is mostly one big class, and an empty Tests project.

    I find myself staring at the list of methods and wondering whereTF I'm meant to start. Do I go from the constructor down the list of methods, start with the most complicated and brittle logic that's more likely to break, or some other strategy?

    My other main concern is that a lot of the logic relies on external dependencies. The software is an add-on for a third party application, and works with orders that have been entered through the main application. Some of the tests will necessarily involve completing the orders and checking things like item quantities went through OK. Should I have a dedicated database set up for the tests, and restored each time I run them? Is this the sort of situation where mocking frameworks make sense, so I can fake out things like the committing method and just check what's going on within my logic?

    As might be obvious, I'm a bit clueless. I've done some very basic unit testing in the past, and played with tools like FakeItEasy for mocking, but this will be the first time I've tried it on anything more complicated


  • FoxDev

    I'd say start with the simple unit tests first; you may be able to build the more complex tests out of the simple tests


  • FoxDev

    hmm... here's how i would approach that.

    1. Test standalone functionality, refactor any logic that doesn't have external dependencies into separate methods/classes and test them
    2. Mock external dependencies, test logic that depends on external dependecies using mock objects
    3. Test more complex portions of the code, these would be more integration tests than unit tests and should probably be in their separate project
    4. Test not only code coverage but path coverage. try to test as many possible paths through your code as you can. static analysis tools will help you find those. Resharper has a great test runner that includes this functionality if you don't have access to VS Ultimate (i prefer the VS tools, but you need ultimate to get them and that's EXPENSIVE!)
    5. Ask someone who is a QA engineer ot a Software Developer In Test. (someone like @Yamikuronue)

  • I survived the hour long Uno hand

    @Jaloopa said:

    Do I go from the constructor down the list of methods, start with the most complicated and brittle logic that's more likely to break, or some other strategy?

    I would say, since you're always likely to run out of time, take a risk-based approach: what would do the most damage if it was wrong? Probably your business logic functions more than the constructors and so on.

    @Jaloopa said:

    Is this the sort of situation where mocking frameworks make sense, so I can fake out things like the committing method and just check what's going on within my logic?

    Yes, absolutely. The basic idea is that anywhere your class calls out to another class, there's a "seam" that you need to cut for unit testing. Ideally, you're injecting dependencies somehow (not talking frameworks here, often it's just in the constructor) so you can pass in a fake version and not hit the database at all. In actuality, you probably didn't do that very well, which you'll discover as you try to cut the seams and realize you can't. As a rule of thumb, wherever you see the word "new", that's a code smell for testability.

    Unit tests should never execute actual production code outside of the one unit you're testing. Typically a unit is a class, though a unit being a single method is also a common definition to use. It depends; I like to ignore private methods when I consider my units, as it's often just about readability and reusability rather than separate concerns.

    @Jaloopa said:

    Should I have a dedicated database set up for the tests, and restored each time I run them?

    That would be an integration test, making sure that your database and your code play nice together. Also a good thing to have, and usually written with unit testing tools, but part of a different suite.

    Hope that helps!


  • ♿ (Parody)

    @Yamikuronue said:

    That would be an integration test, making sure that your database and your code play nice together. Also a good thing to have, and usually written with unit testing tools, but part of a different suite.

    These are also easier to get inspired to write. I like to write one whenever I get a bug report from a user. And honestly, they're a lot more useful to me than unit tests, where you can get lost in the details of something and lose site of what you're actually trying to accomplish.


  • BINNED

    I'm just here to say that I'm envious of you bastards here, since I have no damned idea how I should even begin to write tests for some parts of code any farther than "does it segfault if I send it a NULL?"

    Stupid poorly documented APIs that spew inconsistent streams of realtime data dependent on the phase of the moon...


  • kills Dumbledore

    @Yamikuronue said:

    That would be an integration test, making sure that your database and your code play nice together

    OK, so at the moment there's a lot of linq to SQL within the code, meaning pretty much every method has at least one call to the database. Should each of these be refactored into a private method so I can mock it out?

    There are also external dependencies riddled throughout. As an example, my constructor logs into the third party software as an administrator account, initialises some stuff and logs out again. This is required because if you try to log in as a limited user (it uses AD, so logins are via impersonation), it tries to initialise the objects, which seems to require some HKLM registry access and errors out with an access denied error. So before the object is fully created, I've already called out to dependencies. This can't really be injected since the particular version of the third party software should be an implementation detail and completely opaque to the caller of my class

    It's starting to look like this is going to be a lot of work to make it properly testable...


  • I survived the hour long Uno hand

    @Jaloopa said:

    This can't really be injected since the particular version of the third party software should be an implementation detail and completely opaque to the caller of my class

    What? That's exactly why it should be injected, preferably by interface rather than implementation. I mean, you have a contract of some kind (interface, abstract parent, et cetera) that you're calling against, right? So you make a mock version of the object that implements the contract but doesn't do anything real, and hand it that instead. Later, if you change what the real implementation does or how or what version, you know it won't be a problem for your code.

    @Jaloopa said:

    there's a lot of linq to SQL within the code

    I'm not in .net land, so I'm not as familiar with the challenges of mocking out linq to SQL. Someone who is wrote this: https://cuttingedge.it/blogs/steven/pivot/entry.php?id=84

    @Jaloopa said:

    It's starting to look like this is going to be a lot of work to make it properly testable...

    That's always the case when you don't do TDD and you don't code with testing in mind.

    Some more reading:



  • @Jaloopa said:

    So what I have is a C# project which is mostly one big class

    Uh.

    You have bigger problems than missing unit tests. Refactor that shit, yo! Unit tests won't do you any good until you have units to test in any case.



  • @Jaloopa said:

    database

    Our company uses a SQL Server database file, checked-in with the test schema.



  • I would say your project sounds like the kind where unit tests aren't actually that useful. That shouldn't stop you from refactoring your code, or adding tests where they would give most benefit. But...

    Here's the thing Alex wrote an article a few months ago which I can't look up because I'm on mobile. His position is that programmers like unit tests because to them writing code is "fun", while going down a spreadsheet of manual test items is not fun. That means programmers vastly overestimate the value of unit testing. (I can guarantee my co-workers do. So far I've caught one bug using a unit test, and our QA people have caught like 30 without unit tests.)

    So anyway, just be rational when determining how to spend your time.



  • @Jaloopa said:

    OK, so at the moment there's a lot of linq to SQL within the code, meaning pretty much every method has at least one call to the database. Should each of these be refactored into a private method so I can mock it out?

    There are also external dependencies riddled throughout. As an example, my constructor logs into the third party software as an administrator account, initialises some stuff and logs out again. This is required because if you try to log in as a limited user (it uses AD, so logins are via impersonation), it tries to initialise the objects, which seems to require some HKLM registry access and errors out with an access denied error. So before the object is fully created, I've already called out to dependencies. This can't really be injected since the particular version of the third party software should be an implementation detail and completely opaque to the caller of my class

    It's starting to look like this is going to be a lot of work to make it properly testable...

    Edit: Yes it sounds like it's going to be hell.

    Depends on how you created the database.

    We usually go Entity First now at my job. Essentially, every class constructor by default initializes the production web.config DB connection but has an optional constructor that accepts your test database.

    In unit testing, you build a custom DbContext for your EF implementation that has a "standard" data set and contains a method to indicate if the loaded DB is dirty. If the currently loaded test DB is dirty (i.e. some form of data modification was tested in the previous iteration), you nuke the DB and start over.



  • @Jaloopa said:

    nice-to-haves after delivery

    priorities!
    @Jaloopa said:

    it had to be out of the door ASAP

    Because delivery date is more important than stability.

    You know what. We'll never overcome this problem.


  • ♿ (Parody)

    @blakeyrat said:

    Here's the thing Alex wrote an article a few months ago which I can't look up because I'm on mobile. His position is that programmers like unit tests because to them writing code is "fun", while going down a spreadsheet of manual test items is not fun. That means programmers vastly overestimate the value of unit testing. (I can guarantee my co-workers do. So far I've caught one bug using a unit test, and our QA people have caught like 30 without unit tests.)

    And ideally, you figure out a way to turn that manual stuff into something automated, at which point they're probably integration tests.



  • @Onyx said:

    I'm just here to say that I'm envious of you bastards here, since I have no damned idea how I should even begin to write tests for some parts of code any farther than "does it segfault if I send it a NULL?"

    Stupid poorly documented APIs that spew inconsistent streams of realtime data dependent on the phase of the moon...

    Is that your code or a third party library you're talking about?

    You should only unit test your code, not 3rd party code. Part of those tests should verify your code behaves correctly when it's fed inconsistent data, so you can be confident it's not going to crap out when the 3rd party code is Discoursistent.

    I'm not trying to be condescending or anything with this remark. It's just that I've stumped my toe on that particular table too many times: start writing unit tests, realize an hour or so later that the focus of the code getting tested drifted from my code to how a particular 3rd party dependency behaves and that a sizeable portion of my tests actually don't test my own code at all.


  • I survived the hour long Uno hand

    @blakeyrat said:

    I've caught one bug using a unit test

    That's not the purpose of unit tests. The idea is to make it safe to do large-scale refactors or even smaller bugfixes while being absolutely sure that the new code is functionally equivalent. It will never find spec problems or developer misunderstandings or mismatches between components.



  • Well fine, but it consumes a HUGE amount of time for that meager benefit. Which is my point.


  • BINNED

    @OffByOne said:

    Is that your code or a third party library you're talking about?

    It's a service I'm getting realtime data from. To be specific, Asterisk PBX.

    Inconsistent data is a pretty tricky term here. Can I test getting garbage from it? Yes. Can I test something not being set when it should have to check my application doesn't die and / or hoses the DB? Yes. Is it actually wrong data? Maybe.

    Now, to be fair to it, I never saw it not send events it should have, so far. Knowing what events it should have sent is the problem. It's not documented, anywhere. Oh, I have a list of them, but which ones do I get in a certain situation and in what order? Nope. I pretty much had to reverse-engineer the thing to get to the point where I'm at right now. So, when I miss a piece of data, was it because my application failed to parse something or because that never actually happens in that specific case? No way to find out than to log everything and go through it all by hand.

    So yes, testing if my part of it is stable and won't crash when handed invalid objects? Doable. Knowing if that's a path I forgot to cover or something else went wrong? Back to logging...



  • @Onyx said:

    It's a service I'm getting realtime data from. To be specific, Asterisk PBX.

    That's 3rd party software then.

    @Onyx said:

    testing if my part of it is stable and won't crash when handed invalid objects? Doable. Knowing if that's a path I forgot to cover or something else went wrong? Back to logging...

    Are we still talking about unit level tests or about integration tests?

    Looks like you can use the contents of your log files to create a test that simulates a specific scenario where things go wrong.
    With that test in place and your code passing it, you can be confident that that particular scenario won't bite you again in the future.

    I have been known to reference a particular test in the comments in my code preceding a specific workaround for a single (or single group of) workaround of buggy 3rd party dependencies.
    In the comment block at the top of the referenced test, I document what the test exactly tests and when that particular scenario can happen. That's also the place I'd put the relevant log entries, to document what that scenario looks like if it happens.

    I use my tests as extra technical, low-level documentation. I don't know whether that's doing it wrong or a smart idea.
    It did help answering the question "Now why do I check for this (on first glance overly specific) set of conditions and handle them separately from the general case?" quickly after not touching my code for a few months, so at least I got some benefit out of doing it that way.



  • @blakeyrat said:

    Uh.

    You have bigger problems than missing unit tests. Refactor that shit, yo! Unit tests won't do you any good until you have units to test in any case.

    QFT. What the hell.

    Sounds like the Java batch app on our servers I got to peek at the other day and it seemed written by someone who thought he was programming in BASIC or something. 100K LOC monstrosity absolutely impossible to keep track of in a single file, "run.java". No classes, nothing, just a main method and a billion different private methods. Like fuck off dude, I wish I could fucking kill the guy who got put in charge of that.


  • Java Dev

    @Jaloopa said:

    it had to be out of the door ASAP, with nice-to-haveshalf the essentials after delivery

    FTFT



  • @Yamikuronue said:

    so you can pass in a fake version and not hit the database at all.

    Is there a reason you can't have a local test DB as part of your test suite? Throwing mocks around is fine and dandy for some things, but I wouldn't trust 'em over having my test suite direct the code-under-test to say, a SQLite DB...at least for the tests that should be dealing with those types of objects to begin with ;)

    Agreed otherwise, though -- focus on pulling the logic of your application away from the plumbing, as the two really should be tested separately and with separate test strategies. Ideally, the logic lives so that it's called from the plumbing, in a functional style, so you can test it easily just by passing it parameters and seeing what it returns.

    @Jaloopa said:

    OK, so at the moment there's a lot of linq to SQL within the code, meaning pretty much every method has at least one call to the database. Should each of these be refactored into a private method so I can mock it out?

    It's better to refactor it so that you have two layers -- a plumbing layer with the Linq-to-SQL and calls into the logic layer, and a logic layer that does nothing but crunch bits passed to it as parameters and return more bits back. That way, everything's cleanly testable without mocks!

    @Jaloopa said:

    It's starting to look like this is going to be a lot of work to make it properly testable...

    Unfortunately, it looks like there's some nasty stuff in your external dependencies, yes...which is all the more reason to factor your logic out religiously.

    @dstopia said:

    No classes, nothing, just a main method and a billion different private methods. Like fuck off dude, I wish I could fucking kill the guy who got put in charge of that.

    I'd actually be much more OK with that than with a 100k LOC main()...at least you have bite-sized chunks to work with!


  • I survived the hour long Uno hand

    @tarunik said:

    Is there a reason you can't have a local test DB as part of your test suite?

    No, it's just not a unit test anymore once you do.

    @tarunik said:

    Throwing mocks around is fine and dandy for some things, but I wouldn't trust 'em

    That's why you do integration testing. Once you're sure each piece works, you probably want to test that they work together with a real DB



  • Read onward -- I already mentioned how to do it in a way where you don't have to deal with this whole "mock the database" faff to begin with ;)


  • ♿ (Parody)

    @tarunik said:

    "mock the database" faff

    Do people really do that? It seems like a lot of work for very little benefit. Possibly negative.


  • I survived the hour long Uno hand

    @boomzilla said:

    It seems like a lot of work

    Eh, it's not really. Node:


  • ♿ (Parody)

    @Yamikuronue said:

    Eh, it's not really.

    I'm not entirely sure what I'm looking at, but there's not much in the way of data coming out of that.


  • FoxDev

    IIRC, part of that unit test is populating the mock DB with mock tables and mock data


  • I survived the hour long Uno hand

    No. What you're looking at is me using sinon to mock out the database driver :D


  • ♿ (Parody)

    @RaceProUK said:

    IIRC, part of that unit test is populating the mock DB with mock tables and mock data

    Right, that's what I don't want to do. PITA.



  • @boomzilla said:

    Right, that's what I don't want to do. PITA.

    You shouldn't do that anyway (for unit tests).

    @Yamikuronue said:
    @Yamikuronue said:

    you can pass in a fake version and not hit the database at all

    not "hit a database populated with fake data".

    There are several techniques of telling the database driver mock what to return, before you call the code to be unit tested (which in its turn calls your database driver mock, gets back the results you specified and your unit test then verifies correct behavior of your code for the data/error you told the DB driver mock to return).

    As @Yamikuronue said: populating a (real) database with fake (known at test time) data and testing against that is not a unit test anymore, but an integration test.

    Thought experiment: replace "database" in the previous posts with "clock". Assume the behavior of your application is time sensitive (e.g. it uses time-outs, it needs to perform a cleanup when data is older than X seconds, an operation should finish within Y seconds, certain operations should only be allowed during business hours or when the 2nd of the month is a Tuesday, ...).
    If you have a mockable wrapper around the RTC, testing all code paths becomes almost trivial, your unit tests never even have to look at the real RTC.
    If you would have to write unit tests and jeff around with the real system clock instead, things would get... interesting.



  • You don't need data (you insert it as-needed for tests), you only need schema. Since the DB is only schema, it's like 200k. Since it's like 200k, you can just add it to your project as a SQL Database File and shove the connection string into your .testsettings file. Since you've followed my advice, you can pay me $500.



  • @blakeyrat said:

    That means programmers vastly overestimate the value of unit testing. (I can guarantee my co-workers do. So far I've caught one bug using a unit test, and our QA people have caught like 30 without unit tests.)

    Let's say you and QA each spent 8 hours on this update. You wrote a bunch of unit tests and caught nothing. QA people clicked all over the app and found a bunch of bugs. Wow, QA is so much more efficient than unit tests!

    But next week, the client changes their mind and wants the feature to be different. You make the changes and the code needs to be retested. QA spends another 8 hours doing the testing. But you already have your unit tests from the last time. You can just change a few methods and re-run the tests. Done in half an hour. Next month, boss wants a new widget, and it's back to the same thing. 8 hours for QA, a few minutes to update the tests.

    So every time there's a new change, your initial time investment is paying dividends, while QA is a constant drain on resources.


    That said, RE: the original topic.

    It's too late. The time investment now isn't worth the effort.

    There's a reason people practice TDD. It's because it forces them to write the code that CAN be unit-tested. Yours apparently can't.

    That said, what you CAN do is integration testing. That is, instead of checking if this or that function will work in isolation, just set up a simulation of an entire processes in your app.

    So, for example, attach a throwaway database on one end and simulate a POST request on the other. Then check if that caused a proper record to be added to the DB. Or whatever your app is doing.

    That won't catch all the bugs, but will give you a basic sanity check that your app is generally doing what it's supposed to be doing.

    Also, don't listen to me. Listen to @Yamikuronue and other people who actually know what they are talking about :)



  • @cartman82 said:

    Let's say you and QA each spent 8 hours on this update. You wrote a bunch of unit tests and caught nothing. QA people clicked all over the app and found a bunch of bugs. Wow, QA is so much more efficient than unit tests!

    But next week, the client changes their mind and wants the feature to be different. You make the changes and the code needs to be retested. QA spends another 8 hours doing the testing. But you already have your unit tests from the last time. You can just change a few methods and re-run the tests. Done in half an hour. Next month, boss wants a new widget, and it's back to the same thing. 8 hours for QA, a few minutes to update the tests.

    So every time there's a new change, your initial time investment is paying dividends, while QA is a constant drain on resources.

    QA is for exploratory and acceptance testing -- making sure user-visible functions work to spec and/or as the users expect, and finding bugs. Automated unit and integration testing is for regression and core function testing -- making sure that individual algorithms are a semblance-of-correct/sane, and making sure that bugs don't come back to haunt.

    @cartman82 said:

    That said, what you CAN do is integration testing. That is, instead of checking if this or that function will work in isolation, just set up a simulation of an entire processes in your app.

    So, for example, attach a throwaway database on one end and simulate a POST request on the other. Then check if that caused a proper record to be added to the DB. Or whatever your app is doing.

    That won't catch all the bugs, but will give you a basic sanity check that your app is generally doing what it's supposed to be doing.


    Agreed in his case, as far as his code will let him -- thankfully, most of the tools you use for unit testing can be used for integration testing as well.



  • @boomzilla said:

    Do people really do that? It seems like a lot of work for very little benefit. Possibly negative.

    It's not as gigantic as it sounds, but it's still completely avoidable in your business logic/algorithm testing. Why people insist on binding business logic directly to the database, UI, external API, etal continues to escape me...the only reason I do it in the code that I'm working on these days is to match the existing code style.


  • I survived the hour long Uno hand

    @tarunik said:

    QA is for exploratory and acceptance testing

    QA is for anything and everything related to quality. A specific QA team might specialize in exploratory or acceptance or functional or regression testing, or they might play those roles in a given organization, but my expertise is a lot broader than you'd think. It even extends to improving the quality of static documents produced by BAs, such as requirements documents.



  • @Yamikuronue said:

    QA is for anything and everything related to quality. A specific QA team might specialize in exploratory or acceptance or functional or regression testing, or they might play those roles in a given organization, but my expertise is a lot broader than you'd think. It even extends to improving the quality of static documents produced by BAs, such as requirements documents.

    I was thinking "QA testing" when I said "QA", based on what my typical view of QA vs QC is from the meatspace world...sorry :) and yes -- requirements defects are important!



  • @Jaloopa said:

    Unfortunately, I didn't have time to add tests while writing the project, it had to be out of the door ASAP

    Sure you did, you simply made a choice not to.

    , with nice-to-haves after delivery
    Most people would think that having a solid working maintainable product is actually what was paid for, and not something that was merely "nice to have".

  • ♿ (Parody)

    @OffByOne said:

    You shouldn't do that anyway (for unit tests).

    But then...if I don't make up any data, I have no data to use for my tests. Perhaps I wasn't clear...

    I DON'T WANT TO HAVE TO FABRICATE DATA.


  • ♿ (Parody)

    @blakeyrat said:

    You don't need data

    Sure I do.

    @blakeyrat said:

    (you insert it as-needed for tests)

    This is the PITA I want to avoid.

    Also, I don't want to have to test my test data.



  • @boomzilla said:

    But then...if I don't make up any data, I have no data to use for my tests. Perhaps I wasn't clear...

    I DON'T WANT TO HAVE TO FABRICATE DATA.

    I got that. I even told you
    @OffByOne said:

    You shouldn't do that

    What I meant was YOU SHOULD NOT FABRICATE DATA.

    I think we're violently agreeing.

    Maybe I wasn't clear. It took me some time to grasp the concept of mocking a database driver instead of using a real database with mocked up (fake) data.

    Your unit tests shouldn't touch a database at all. Heck, if there is a database connection string somewhere in your test code, you are most likely doing it wrong!

    I wanted to write some pseudocode, but I'll link to https://metacpan.org/pod/DBD::Mock instead. It's Perl, but it should give you a reasonable idea of what I mean.

    A unit test conceptually looks like:

    1. tell mocked database driver how to behave next time it's called
      a. allow/refuse connection
      b. what data to return if (for example) a SELECT query is executed
      c. ...
    2. have unit test call the code to be tested (which has been set up in the unit test initialization to use the mocked database driver instead of the real/default one)
    3. assert your code behaved correctly under the circumstances you set up in the first step

    Don't think step 1b. means you have to make up any data. Well, you do have to make up (some) data to see if the function under test behaves correctly when the real database would have returned that data during normal program execution.
    It's conceptually exactly the same as choosing which function parameters to call a function with in a unit test in order to test the behavior of the function. If you need to test behavior of function int factorial(int n) and you decide to test it for values 0, 1, 10 and -1, would you call choosing those values "making up data"?


    Testing your code with a real database with faked/made up values in it, is no longer unit testing. That's integration testing. They test different things and hence require a different test setup.


  • ♿ (Parody)

    @OffByOne said:

    A unit test conceptually looks like:

    Wait for it...

    @OffByOne said:

    tell mocked database driver how to behave next time it's called

    ...wait for it...

    @OffByOne said:

    allow/refuse connection

    ...almost there...

    @OffByOne said:

    what data to return if (for example) a SELECT query is executed

    BAM! Fabricated data.

    @OffByOne said:

    Don't think step 1b. means you have to make up any data.

    But where did it come from?

    @OffByOne said:

    Well, you do have to make up (some) data to see if the function under test behaves correctly when the real database would have returned that data during normal program execution.

    Ah, right, like I said.

    @OffByOne said:

    It's conceptually exactly the same as...

    In the same way that 10 hours is conceptually the same as 10 minutes. Both are an interval of time.

    @OffByOne said:

    If you need to test behavior of function int factorial(int n) and you decide to test it for values 0, 1, 10 and -1, would you call choosing those values "making up data"?

    That's a mind numbingly trivial thing. That wouldn't bother me at all. But I'm talking about passing around an object that has a shit ton of data behind it. I don't want to make the data up. Now, imagine that to create the equivalent of 0, 1, 10 and -1 you had to specify 30 additional data points for each one. That's a WAG, but also definitely a gross underestimate of the sort of stuff I'd have to come up with for some places in my code.

    @OffByOne said:

    Testing your code with a real database with faked/made up values in it, is no longer unit testing. That's integration testing.

    Yes, ain't nobody around here got time for that sort of unit testing. At least a little bit in part because it's a crap ton of data we'd have to fabricate.



  • @boomzilla said:

    At least a little bit in part because it's a crap ton of data we'd have to fabricate.

    Can you use examples curated from the production dataset instead, or would the amount of sanitization needed to avoid leaking data be excessive?


  • ♿ (Parody)

    @tarunik said:

    Can you use examples curated from the production dataset instead, or would the amount of sanitization needed to avoid leaking data be excessive?

    That's what we do for our automated tests. I have a sanitized version of production data and just pick stuff from there.



  • @boomzilla said:

    But where did it come from?

    The data for a unit test comes from one place: careful analysis of the minimal set of data points that will validate all of the permutative data paths (minimum value being the cyclometric complexity). Any attempt to use "live data" as your source is likely to not actually test the code....

    double foo(double a, double b)
    {
    if (a1.2344254 == b) return 0;
    return a
    b;
    }

    You could run this for quintillions of rows from a database of "real data" without ever hitting the condition where 0 is incorrectly returned. Full unit testing can be accomplished by a minimum of 3 value pairs, though I would probably use 5.


  • ♿ (Parody)

    Yay, another trivial function that takes trivial data.



  • @boomzilla said:

    Yay, another trivial function that takes trivial data.

    Don't focus on the code (which was deliberately trivial, posting a few thousand lines would not really be appropriate or effective. Instead go back and re-read the first paragraph of the post.

    From a unit testing perspective, if you run two (or more) points that exercise identical functionality then you are "Doing It Wrong", same as if you miss a set. Getting this minimal, yet complete, collection of data is not deterministically practical when selecting from "real live data".


  • ♿ (Parody)

    @TheCPUWizard said:

    Instead go back and re-read the first paragraph of the post.

    No, I read it correctly the first time. Go back and read my posts, because you didn't get them.

    @TheCPUWizard said:

    Don't focus on the code (which was deliberately trivial

    I haven't been focusing on the code at all. I'm talking about the data that's required to test something. When your function just takes something trivial like an int, it's no big deal to make up some test cases to explore the space. That's not the space I'm talking about.

    @TheCPUWizard said:

    Getting this minimal, yet complete, collection of data is not deterministically practical when selecting from "real live data".

    Real live data has the benefit of being something that's already there that I don't have to go through and create, which is a major PITA and now I'm running the risk of constructing it poorly and now I have another thing to worry about.

    And when I need something specific, I can pick it out from my existing data. Or get something close and change it a little bit for the test.

    But again, integration tests are much more valuable to me than honest to god unit tests (for this application, anyways).



  • @boomzilla said:

    But again, integration tests are much more valuable to me than honest to god unit tests (for this application, anyways).

    Please do not confuse the matter. The two are completely different and this topic is specifically about UNIT tests <grin>

    Seriously, I do not see where "complex data" is needed for unit tests, provided SOLID and DRY principles are followed. Consider:

    [code]
    ComplexC Foo(ComplexA a, Complex B, int c)
    {
    int z = Bar(a,b);
    int w = z * c + a.Value;
    return new ComplexC(w);
    }
    [/code]

    While this may appear to need lots of data to create the instances of ComplexA and ComplexB, and knowledge about ComplexC, this is not true from a unit test perspective. All that is needed:

    1. "a" and "b" are properly passed to Bar [whatever that is] which will return an integer
    2. a have a Value property of type integer.
    3. w is properly calculated and passed to the constructor of ClassC
    4. the instance that is created is returned.

    (provided I did not miss something), that is the scope of a unit test. The functioning of all of the items that are passed to/from or used is completely out of scope.

    Under those constraints, the minimum set of values that must be passed to fully validate the tests is small.


  • ♿ (Parody)

    @TheCPUWizard said:

    Seriously, I do not see where "complex data" is needed for unit tests, provided SOLID and DRY principles are followed.

    It depends on your application, don't it?

    @TheCPUWizard said:

    Please do not confuse the matter. The two are completely different and this topic is specifically about UNIT tests

    I'm not confusing them, and while that was what the title refers to, we've been talking about unit vs integration testing, too.


Log in to reply