A tale of testing



  • Between the barbecues, failed rescue attempts, and summoning in necromancy rituals, sometimes I write a little Java code.

    Before diving in the WTF material, and no, it is not Java, some context is needed.

    The platform I develop for is called by management/system architects embedded. Okay, granted it runs on a PowerPC SoC, but apart from that, is as powerful as my 2012 laptop I have at home, and for external access it runs a Sort of Network Management Protocol.

    The way it was first programmed, unit tests are almost impossible to write, since almost every class uses one of the "core" classes which in turn expect dozens of other classes to be instantiated. (Some mocking progress is being done on this area, pun intended 😁)

    So we resort to have JUnit tests that use a SNMP library to connect to the software, do some sets and gets, and check the responses.


    ####The Bug Report
    Hey Cheryl
    This changed between version 4.5.6.90 and version 4.5.6.90.10. I can't set this bitmask to all ones on this object x.y.z.0.0 Can you check please what's happening?
    Oh, and be a dear! File the bug report for me and send me the Ticket #.

    KTNXBYE
    P.S.: Can you give me @the_dragon's phone number. He's sooooo HOT.

    Ok, I'm filing the bug report, creating a work order and assign it all to myself. At least I'm not QA also.

    Here the issue is simple, and in the first 2 minutes I can reproduce the problem, and I know right away where I messed up between versions.
    There is this little feature, that allows a bitmask set-request to have extra bits not applicable to the object. As long as one of them is applicable, there is no error (this is by design, and might have some logical explanation. I don't care).

    I open the test for this, and I see: damn'it. I have tests for extra bits, but none for ALL bits.
    New test step:

        response=comm.set("x.y.z.0.0", 0xffffffff);
        assertEquals(0, response.errorCode);
    

    Et voilà, test reproduces the bug. Fix it. Tests all green again.


    ####The Code Review
    Everything we do, needs to be code reviewed. A good thing, I guess. Team leader, comes by and looks at the test.

    - Don't use ticket# as test case description.

    • Hey boss, after the ticket# there is a line saying "a bitmask of all 1s is rejected by system"
    • Okay okay, I didn't see it. Look, why aren't you checking the return value of the response and comparing it to what is expected?
    • Boss, return values are already checked on other test cases.
    • I know, but you should check the return value of this set!
      (Deep breath, take a second)
    • Boss, this is a test step only. It recreates a bug scenario, and it checks if the set was successful.

    And repeat the last two dialogue lines during 10 minutes. On the first iteration, I thought my boss meant "now you should get the object value, and check it also". The object value would be only the bits that were 'accepted'. But then I understood he meant the set response. SNMP protocol says the response is the same as the request, minus the errorCode and errorIndex fields.

    Another day at work1

    [1] I would say office, but most of the time I work from home.



  • @presidentsdaughter said:

    - Boss, return values are already checked on other test cases.

    • I know, but you should check the return value of this set!

    Solution: invoke the other test methods from this one.



  • @presidentsdaughter said:

    P.S.: Can you give me @the_dragon's phone number. He's sooooo HOT.

    Your co-worker is welcome to join you in the dungeon.



  • I thought it was a cave. Is this some kind of furnished cave dungeon?



  • @ben_lubar said:

    I thought it was a cave. Is this some kind of furnished cave dungeon?
    You're not paying attention. @the_dragon has a cave and a dungeon.


Log in to reply