We don't need to test; we have configurations!



  • As part of an ongoing effort to improve performance, I'm making major structural changes to our application (threading, calling a single wrapper stored proc vs calling the db for every single row update, etc). For major changes like these, I usually put a flag in a config table in the db so that once deployed, if something bad is discovered, we can just flip the flag and revert to the old code while we're figuring out what went wrong - at least for a month or so (it's a cheap form of insurance). Once I get things debugged, I usually try to scale up to a production scale test.

    However, around here, we have no environment that is even remotely close to production level capability. As such, the level of testing I can do is severely limited. I tell my boss that if we deploy this stuff without a full scale test, we'll be risking all sorts of potential problems from taxed cpu's to memory limitations to I/O bandwidth, etc.

    No problem, we don't need to test it; if something goes wrong we can just revert using the db flag.

    !!!?!

     



  • @snoofle said:

    As part of an ongoing effort to improve performance, I'm making major structural changes to our application (threading, calling a single wrapper stored proc vs calling the db for every single row update, etc). For major changes like these, I usually put a flag in a config table in the db so that once deployed, if something bad is discovered, we can just flip the flag and revert to the old code while we're figuring out what went wrong - at least for a month or so (it's a cheap form of insurance). Once I get things debugged, I usually try to scale up to a production scale test.

    However, around here, we have no environment that is even remotely close to production level capability. As such, the level of testing I can do is severely limited. I tell my boss that if we deploy this stuff without a full scale test, we'll be risking all sorts of potential problems from taxed cpu's to memory limitations to I/O bandwidth, etc.

    No problem, we don't need to test it; if something goes wrong we can just revert using the db flag.

    !!!?!

     

     

     

    Aha, the "make it use less resources" flag. I tend to have that next to the big shiny red "fix the problem" button.

     



  • @snoofle said:

    I tell my boss that if we deploy this stuff without a full scale test, we'll be risking all sorts of potential problems from taxed cpu's to memory limitations to I/O bandwidth, etc.
    Do you have a copy of this conversation?
    I have had the experience of getting a totally different response when I forced the conversation to email that could be audited.



  • @snoofle said:

    calling a single wrapper stored proc vs calling the db for every single row update

    Wait, you are ADDING an uber-do-everything stored procedure? That sounds like TRWTF to me...



  • @ekolis said:

    @snoofle said:
    calling a single wrapper stored proc vs calling the db for every single row update
    Wait, you are ADDING an uber-do-everything stored procedure? That sounds like TRWTF to me...
    He has multiple rows he needs to update.  Instead of calling once per row, he's calling once with all the updates that need to be processed.  You're of the opinion that you need as many database round-trips as possible?



  • @ekolis said:

    @snoofle said:
    calling a single wrapper stored proc vs calling the db for every single row update
    Wait, you are ADDING an uber-do-everything stored procedure? That sounds like TRWTF to me...
    Yeah, you're right.  Making individual calls to the database for each record is much better, AND more flexible.  You only need to create a database connection, populate the parameters, call the database, process the response and close the database connection 500k times, all without a transaction.  Those pesky transactions always get in the way anyway...


  • Trolleybus Mechanic

    @ekolis said:

    @snoofle said:
    calling a single wrapper stored proc vs calling the db for every single row update

    Wait, you are ADDING an uber-do-everything stored procedure? That sounds like TRWTF to me...

     

    I read that as instead of:


    For Each objectRowToSave

        Call("proc_save_one_row(row vals....)")

    End

    Being upgraded to either one of:

    @ThisOne said:

    For Each objectRowToSave

        Sql += "proc_save_one_row(row vals...);"

    End

        Call(sql)

    OR

    @ThatOne said:

    Call("proc_save_collection(" + objectRowsToSave.toXML() + ")");




  • @Lorne Kates said:

    I read that as instead of:

    I'm assuming it's more along the lines of:

    @dumb code said:

    Call("UpdateUserTableAddPostCount", 1)

    Call("UpdatePostTableAddPost", post_contents)

    Call("UpdateRecentPostsTableAddPost", post_id)

    Call("UpdateImageTableAddPostImage", post_id, image_url)

    @smarter code said:

    Call("InsertNewPost", all_the_params_you_need_all_at_once)

    I know this is shocking to the posters here, but sometimes relational databases actually have relations in them, instead of tables just being used as a giant spreadsheet page.


  • Trolleybus Mechanic

    @blakeyrat said:

    I'm assuming it's more along the lines of:

    @dumb code said:

    Call("UpdateUserTableAddPostCount", 1)

    Call("UpdatePostTableAddPost", post_contents)

    Call("UpdateRecentPostsTableAddPost", post_id)

    Call("UpdateImageTableAddPostImage", post_id, image_url)
     

    Since it's snoofle's company's code-base, I'm actually assuming it's much worse than we're all assuming.



  • @Lorne Kates said:

    @ekolis said:

    @snoofle said:
    calling a single wrapper stored proc vs calling the db for every single row update

    Wait, you are ADDING an uber-do-everything stored procedure? That sounds like TRWTF to me...

     

    I read that as instead of:


    For Each objectRowToSave

        Call("proc_save_one_row(row vals....)")

    End

    Being upgraded to either one of:

    @ThisOne said:

    For Each objectRowToSave

        Sql += "proc_save_one_row(row vals...);"

    End

        Call(sql)

    OR

    @ThatOne said:

    Call("proc_save_collection(" + objectRowsToSave.toXML() + ")");


    It's more along the lines of "that one", but using lists of Oracle TYPEs instead of xml.

    Essentially each TYPE is a struct that contains each arg to the relevent existing stored proc. Then it's lists of [Type], then a master arg containing all the lists, and the big stored proc iterates over all the lists, calling the relevant stored proc for each row. The first pass just had the big stored proc call all the other stored procs in turn, but that resulted in one context switch for each row to be inserted. Then I optimized that to use FORALL/EXECUTE IMMEDIATE and bulk binding to reduce the numberof context switches between <whatever it is that processes PL/SQL> and the underlying SQL engine. I reduced ~5500 db calls inserting one row each to one db call, with 31 context switches, and cut the execution time from 8+ minutes to about 4 seconds.

    I am really not a db person, but I do have a habit of reading manuals - (sometimes) it's amazing what's in there!

     

     



  • @snoofle said:

    No problem, we don't need to test it; if something goes wrong we can just revert using the db flag.
    Your third mistake was telling your boss about the flag.  Your second was keeping your boss exposed to the technology to prevent the inevitability of his mind's enfeeblement and atrophy, which would ordinarily raise the necessity of your aide and cause him to respect and fear you over time, raising your bargaining position.  Your first mistake was not learning how to manipulate your employer into becoming little more than a hollowed out flesh puppet in your hand.



  • @hoodaticus said:

    Your first mistake was not learning how to manipulate your employer into becoming little more than a hollowed out flesh puppet in your hand.
     

    I know, right?



  • @hoodaticus said:

    Your fist mistake was not learning how to manipulate your employer into becoming little more than a hollowed out flesh puppet in your hand.

    Fisted that for ya.



  • It takes a great deal of time and concerted effort, but the end results are well worth it, I tell you.

    (especially if you have a PHB - the extra padding keeps the wrist warm in these winter months).



  • @hoodaticus said:

    @snoofle said:

    No problem, we don't need to test it; if something goes wrong we can just revert using the db flag.
    Your third mistake was telling your boss about the flag.  Your second was keeping your boss exposed to the technology to prevent the inevitability of his mind's enfeeblement and atrophy, which would ordinarily raise the necessity of your aide and cause him to respect and fear you over time, raising your bargaining position.  Your first mistake was not learning how to manipulate your employer into becoming little more than a hollowed out flesh puppet in your hand.

    Actually, he requested the flag, so not telling him about it wasn't an option. As for manipulating him, he lets me get away with murder around here, so I don't push him; others yes, but not him.


  • @snoofle said:

    he lets me get away with murder around here
    Would your boss also be willing to help you get rid of, um, evidence?  If so, I think I may have found a solution to your "problem" developers...



  • @snoofle said:

    As for manipulating him, he lets me get away with murder around here
     

    Man, I wish my boss would have done that.

    Everybody's using smartphones now. Wasn't like that when I went in.



  • @Sutherlands said:

    He has multiple rows he needs to update.  Instead of calling once per row, he's calling once with all the updates that need to be processed.  You're of the opinion that you need as many database round-trips as possible?

    Oh, I'm sorry, I read the OP as replacing this:

    exec CreateFoo 'foo #1', 'this is the first foo'

    exec CreateFoo 'foo #2', 'this is the second foo'

    exec DeleteBar 3

    with this:

    exec UberMeister 'foo', 'create', 'foo #1', 'this is the first foo'

    exec UberMeister 'foo', 'create', 'foo #2', 'this is the second foo'

    exec UberMeister 'bar', 'delete', 3



  • @snoofle said:

    @hoodaticus said:

    @snoofle said:

    No problem, we don't need to test it; if something goes wrong we can just revert using the db flag.
    Your third mistake was telling your boss about the flag.  Your second was keeping your boss exposed to the technology to prevent the inevitability of his mind's enfeeblement and atrophy, which would ordinarily raise the necessity of your aide and cause him to respect and fear you over time, raising your bargaining position.  Your first mistake was not learning how to manipulate your employer into becoming little more than a hollowed out flesh puppet in your hand.

    Actually, he requested the flag, so not telling him about it wasn't an option. As for manipulating him, he lets me get away with murder around here, so I don't push him; others yes, but not him.
    Ah, so you made no mistakes, and it sounds like you already hollowed out your flesh puppet.  Good job!

     


Log in to reply