The simplest solution to a problem (that shouldn't exist)



  • We have an ASP.NET MVC website built on top of a 15-year-old MSSQL database. Because of the way the business processes on the database are setup, there's a once-a-month job that has to run; during this time, nothing is allowed to touch the database. Since the website is supposed to be up 24/7/365, this is a problem. The "solution" arrived at by my esteemed colleagues (one of whom is our company's technical director) is: fix the DB job to not be retarded.

    ... HA! Just kidding, actually the solution is "take the parts of the website that touch the DB down, for however long it takes the DB job to run, then bring them back online". This is WTF #1.

    So the 24/7/365 website becomes "24/7/365 except for that one bit of time every month" and I have to implement the code to take it offline and bring it back online. My solution: add a rewrite rule to the web.config that sends all DB-touching page traffic to a "sorry we're closed" page; and disable said rule. When our Windows service (that currently exists, and runs on the same box) starts the DB job, it modifies the web.config to enable the rewrite rule. When the job finishes, the service re-disables the rewrite rule, and all is peachy. (This web.config modifying is all done programmatically via the Microsoft.Web.Administration APIs - no fudging around with XML parsing.)

    The technical director wants something on every affected controller on the website that checks "is the DB job running, if so, serve up the "sorry we're closed" page, else serve up the ordinary page". Not only does this require the website code to change (which requires a redeploy and hence downtime), it also ensures that the database gets hit on EVERY PAGE LOAD, regardless of whether the job is running or not. His argument is that my approach will hose any currently active sessions in progress, but when I reminded him that that's KIND OF THE POINT SO THAT THE DB DOESN'T GET TOUCHED and HEY YOUR SOLUTION WILL HAVE THE EXACT SAME EFFECT, he went quiet. Nonetheless, he wants his solution implemented, which - in my opinion - is WTF #2.

    Am I correct in categorizing these decisions as WTFs, or am I the WTF here?



  • Your solution seems like less of a WTF than his. And that's only a WTF assuming that there is a way to make the job unretarded. Which also assumes that the job and everything it does is really necessary.



  • How big is the database? Too big to make a sneaky copy before the job runs and temporarily switch web traffic to that?



  • The job might be writing, too.


  • Discourse touched me in a no-no place

    @The_Assimilator said:

    The "solution" arrived at by my esteemed colleagues (one of whom is our company's technical director) is: fix the DB job to not be retarded.

    It's a shame there's no way to do some sort of snapshot of the database, some sort of near-instantaneous way of getting a consistent state that you can then read away at until you are done with it. I bet nobody's ever thought of doing that before…



  • It's a bit WTFey but relatively easy (and inexpensive) to implement by putting an Attribute at the head of the controller.
    The job can then either set a column on a table (if the db is still accessible) or do something a wee bit 1990s and set some kind of (say) file mutex to say the maintenance job is in progress.

    So just a;

       class MycController : Controller
       {
          ..
       }
    

    kinda construct on the impacted controllers and a pretty inexpensive check within the attribute.

    Not perfect scenario, no, but not a difficult fix.



  • @boomzilla said:

    The job might be writing, too.

    Indeed it is, apologies for not making that clear in the original post.

    @skotl said:

    It's a bit WTFey but relatively easy (and inexpensive) to implement by putting an Attribute at the head of the controller.

    That's exactly the solution proposed by the technical director. While it's not terribad, I honestly don't see how it can be considered better than the one I came up with.


  • Discourse touched me in a no-no place

    @The_Assimilator said:

    While it's not terribad, I honestly don't see how it can be considered better than the one I came up with.

    Make sure you track the performance of this. If it causes a problem, you've got a good case to improve.

    I recommend calling it something other than CheckDatabase; that's how it does it's thing and not what it's purpose is. Perhaps CheckForClosed would be better; that it does it by checking the database is merely part of the implementation. With declarative programming, you have really got to avoid putting the how in the name of things to keep stuff comprehensible when you may be updating implementations later (e.g., to improve performance…)



  • @dkf said:

    I recommend calling it something other than CheckDatabase; that's how it does it's thing and not what it's purpose is. Perhaps CheckForClosed would be better;

    Absolutely correct. A victim of me over-pseudoing my pseudocode.

    And, of course, [CheckDatabase] is right up there with ProcessChanges() and WriteSomeStuff()



  • It could be worse, he might have asked you to render out the entire website to static HTML into a separate directory for each user, mock the user login and auth with fancy rewrite rules, modify the web server config on-the-fly to point the primary domain to the static sites and serve those pages up while the task runs and then reverting the web server config to point back to the dynamic site after the maintenance is done.


    Filed under: OMGWTF #3


  • Winner of the 2016 Presidential Election

    @dfcowell said:

    It could be worse, he might have asked you to render out the entire website to static HTML into a separate directory for each user, mock the user login and auth with fancy rewrite rules, modify the web server config on-the-fly to point the primary domain to the static sites and serve those pages up while the task runs and then reverting the web server config to point back to the dynamic site after the maintenance is done.

    I see you've played knifey-spoony before.


  • Discourse touched me in a no-no place

    @error said:

    knifey-spoonie

    Really⸮ o_O



  • If you use Oracle, this problem will go away. Remember writers do not block readers!

    After reading entire post, I think changing configuration file is also a minor WTF. I have noted that IIS does not in a few cases pick up the changed config file. This will cause much grief for all parties involved. In that case, some one on vacation in disney land will get a call to logon and bring it up manually. I am describe worst-case-scenario, but in our line of work, that has happened.

    Your director solution on first read sound like retarded solution, but on closer inspection, it sound like even more WTF to your SIMPLE solution.

    So how would Nagesh do this? First what is job doing? Is it batch processing something like creation of invoices or calculating late fees or sending out notification? I want to know that. Sometime it is possible to separate out the database so that the batch process run on another area of the database where it will not impact the main website. Many company is doing this. Second option is to study lock escalation procedure in database and write code accordingly. So that this locking and blocking is not happening in retarded fashion.


  • Winner of the 2016 Presidential Election

    @PJH said:

    Really⸮ o_O

    I played it competitively for years; made it to the nationals. Could have been the champion, but my opponent produced this:



  • @error said:

    I played it competitively for years; made it to the nationals. Could have been the champion, but my opponent produced this:

    Impressive, but you'd need two of them to make good use of the knife part, so it kind of defeats the purpose.



  • @error said:

    … this:



  • Isnt there a scheduled time (like every 14th.) for the job? So you can check only on those days?
    You could make a global provider for the Database state, and let it check like once every minute (if there is a request). I think it wont hurt ur database very much. Still a good WTF....



  • I crossed a fork with a spoon, but I ended up with something that had two handles and no business end.

    I call it a "foon".


Log in to reply
 

Looks like your connection to What the Daily WTF? was lost, please wait while we try to reconnect.