Did You Really Say 1.129 billion?



  • So 6 years ago I worked for a different company. At
    that company I worked on a project for a non-profit to handle all of the
    aspects of the company in ASP.NET 2.0 (which was pretty much brand-new
    at the time) and I wasn't an amazing developer then.

    I'm working
    on that project again at my current company as we took over the account.
    Between when I left 6 years ago and now, some 18 developers have
    paraded through the code. So far I have yet to see a single thing that
    has been an enhancement from my original design but this particular one
    takes the cake.

    I came across a part of code that, without any
    data ran very fast but with data, took hours. Clearly something was
    wrong. So I started an investigation one evening, and this is what I
    discovered:

    1. Executes method Controller()
    2. Controller() calls DisplayVoucherSub1()
    3. DisplayVoucherSub1() calls VoucherPreliminary()
    4. VoucherPreliminary() calls CreateFringeTransactions()
    5. CreateFringeTransactions
      returns 726 rows, for each of which a query is executed. If the type
      returned is T, the record is deleted (they're all T due to the query),
      running up to two queries per row for 1452 total queries.
    6. FillForPBByContractAndDate
      dataset is called and returns 9,821 rows. For each row a query is
      executed to add the fringe transaction if the amount is more than 0.
    7. AddFringeTransaction()
      executes a single query to start that returns a whopping 115,018
      records every time (for no reason other than to get the schema so new
      rows can be added), then executes another query, returns 4 records, and
      loops through each of those records to perform a new query to insert the
      transactions (x 4)
    8. For each of the 9,821 records a concurrent query is executed to get the CC type to determine how the amounts are adjusted
    9. At
      the end of this giant loop the AddFringeTransactions() method is called
      a second time for each row, repeating step 7, essentially executing 10
      transactions 9,821 times

    The total number of queries
    performed against the database is slightly more than 109 thousand,
    transferring just over 1.129 billion records for no reason. A worse flaw
    is that as more data is added to the tables the loops will execute that
    many more queries and transfer that much more data.

    I converted
    it to a stored procedure that executes in about 2 seconds with about 4
    total queries (mostly insert set statements) but it's something I will
    never forget.

    								    </div>
    								    
    								    </div>


  • @Darsithis said:

    I came across a part of code that, without any
    data ran very fast but with data, took hours. Clearly something was
    wrong. So I started an investigation one evening, and this is what I
    discovered:

    A data-processing application taking longer to work with data than without data is pretty much what I would expect...



  • @dtech said:

    @Darsithis said:
    I came across a part of code that, without any
    data ran very fast but with data, took hours. Clearly something was
    wrong. So I started an investigation one evening, and this is what I
    discovered:

    A data-processing application taking longer to work with data than without data is pretty much what I would expect...

    Maybe it's O(1)?



  • @dtech said:

    A data-processing application taking longer to work with data than without data is pretty much what I would expect...
     

     Did you really read it? They're performing over 100,000 queries and transferring 1.129 billion records with no need to do so. The point is that the operation took so incredibly long that clearly something was wrong. I guess the humor of it is lost on you.

     



  • @Darsithis said:

    @dtech said:

    A data-processing application taking longer to work with data than without data is pretty much what I would expect...
     

     Did you really read it? They're performing over 100,000 queries and transferring 1.129 billion records with no need to do so. The point is that the operation took so incredibly long that clearly something was wrong. I guess the humor of it is lost on you.

     

    You neither included information about the complexity of the calculations nor the size of the data first - that part you delivered afterwards. How are we to reproduce your line of thinking if you omit important details like that?

    For instance, if your software has a billion of entries and/or is doing some highly complex calculations with said data, then it would be expected for a single run to take hours. And with an empty set of data, there's nothing to be calculated which makes the program run fast. That's a given.



  • @Darsithis said:

    @dtech said:

    A data-processing application taking longer to work with data than without data is pretty much what I would expect...
     

     Did you really read it? They're performing over 100,000 queries and transferring 1.129 billion records with no need to do so. The point is that the operation took so incredibly long that clearly something was wrong. I guess the humor of it is lost on you.

     

    Without looking at the situation from a system perspective it's hard to say if changing that was actually an improvement. As an example, queries that return too much data can sometimes feed a cache under the hood, allowing other parts of the system to get their data quite faster in later operations.

    Lean principle #7: optimize the whole.



  • @Ronald said:

    Without looking at the situation from a system perspective it's hard to say if changing that was actually an improvement. As an example, queries that return too much data can sometimes feed a cache under the hood, allowing other parts of the system to get their data quite faster in later operations.

    Lean principle #7: optimize the whole.

    Umm...@Darsithis said:
    I came across a part of code that ... took hours.

    I converted it to a stored procedure that executes in about 2 seconds with about 4 total queries

    I'd call that an actual improvement from just about any perspective I can think of.

     


  • Winner of the 2016 Presidential Election

    Why is it that everyone tries to come up with reasons why it might not be a WTF? The OP says he optimized it from 109,000+ queries to 4. You're all just giving this guy a hard time.



  • @joe.edwards said:

    Why is it that everyone tries to come up with reasons why it might not be a WTF? The OP says he optimized it from 109,000+ queries to 4. You're all just giving this guy a hard time.

    Because this is TheDailyWtf. A solution to a problem may be provided to give context, but when it's more self-congratulatory it becomes irritating. As the Roman tesserarius used to say: "observa et nunciabo".



  • @Ronald said:

    @joe.edwards said:
    Why is it that everyone tries to come up with reasons why it might not be a WTF? The OP says he optimized it from 109,000+ queries to 4. You're all just giving this guy a hard time.

    Because this is TheDailyWtf. A solution to a problem may be provided to give context, but when it's more self-congratulatory it becomes irritating. As the Roman tesserarius used to say: "observa et nunciabo".

     

    As the American Ill Stew said: "When on an English forum, speaka da English." :)

     



  • @Ronald said:

    As the Roman tesserarius used to say: "observa et nunciabo".
     

    Do you think Romans used Lorem Ipsum when designing their scrolls?



  • @HardwareGeek said:

    @Ronald said:

    Without looking at the situation from a system perspective it's hard to say if changing that was actually an improvement. As an example, queries that return too much data can sometimes feed a cache under the hood, allowing other parts of the system to get their data quite faster in later operations.

    Lean principle #7: optimize the whole.

    Umm...@Darsithis said:
    I came across a part of code that ... took hours.

    I converted it to a stored procedure that executes in about 2 seconds with about 4 total queries

    I'd call that an actual improvement from just about any perspective I can think of.

    Ditto, with a special exception if you work for snoofle's contracting company.

     



  • @Zecc said:

    Ditto, with a special exception if you work for snoofle's contracting company.
    Oy. I'm looking forward to the day Snoofie can name-and-shame WTF, Inc., so that I can be sure to stay far, far away from them (not that I'm likely to have any say in the matter).

     



  • @dhromed said:

    @Ronald said:

    As the Roman tesserarius used to say: "observa et nunciabo".
     

    Do you think Romans used Lorem Ipsum when designing their scrolls?

    Romans did not scroll, they had 80x25 text terminals and it was enough for them to manage a huge empire, they did not need to "upgrade" to a fancy web interface or code a responsive UIs that jumps awkwardly when the screen is resized.



  • @Ronald said:

    Because this is TheDailyWtf. A solution to a problem may be provided to give context, but when it's more self-congratulatory it becomes irritating. As the Roman tesserarius used to say: "observa et nunciabo".
     

    I didn't post for congrats; I get that from my project manager already. I only posted the part about the new proc taking 2 seconds to give some sense of scope and show how astounding it is that the previous developers didn't even test the ridiculously-designed looping code they wrote under production conditions that would immediately show them there was a problem. 1.129 billion records? That's amazing considering there are only 115k in the table (but returned nearly 10,000 times) that they're accessing only to get the schema as it is.

    But to elaborate, the command is solely to display a voucher report to print. It's intended to only take seconds as they have to do dozens a day.

    I was told to post it here by another developer who wondered why I wasn't posting most of the incredibly stupid stuff I've found in this project but honestly given the reaction from it I don't think I'll be posting anything again. Really unfortunate because I've always liked the funny dev stuff my coworkers have linked from here.

     



  • @Zecc said:

    Ditto, with a special exception if you work for snoofle's contracting company.
     

    We do have a project here that feels JUST like that...

     



  • @Ronald said:

    As the Roman tesserarius used to say: "observa et nunciabo".
     

    [url=https://www.google.ca/search?q=%22observa+et+nunciabo%22]According to Google[/url], someone named Ronald said that on TDWTF forums.

    Either that or Seth Rogen just took a Latin class.

     



  • @Darsithis said:

    @dtech said:
    A data-processing application taking longer to work with data than without data is pretty much what I would expect...
     

     Did you really read it? They're performing over 100,000 queries and transferring 1.129 billion records with no need to do so. The point is that the operation took so incredibly long that clearly something was wrong. I guess the humor of it is lost on you.

    Apparently you forgot what you wrote: @Darsithis said:
    I came across a part of code that, without any
    data ran very fast.
    Which is pretty much what you would expect from any application.

     



  • @Darsithis said:

    I was told to post it here by another developer who wondered why I wasn't posting most of the incredibly stupid stuff I've found in this project but honestly given the reaction from it I don't think I'll be posting anything again.

    At least 90% of the users here are either trolls or aspie pedantic dickweeds. Keep posting WTFs and ignore the negative responses. I enjoyed the WTF.



  • @DCRoss said:

    @Ronald said:

    As the Roman tesserarius used to say: "observa et nunciabo".
     

    According to Google, someone named Ronald said that on TDWTF forums.

    This is history in the making and you have a front row seat!



  • @mott555 said:

    @Darsithis said:
    I was told to post it here by another developer who wondered why I wasn't posting most of the incredibly stupid stuff I've found in this project but honestly given the reaction from it I don't think I'll be posting anything again.
    At least 90% of the users here are either trolls or aspie pedantic dickweeds. Keep posting WTFs and ignore the negative responses. I enjoyed the WTF.

    Seconded.



  • @mott555 said:

    At least 90% of the users here are either trolls or aspie pedantic dickweeds.
    While technically correct, this is misleading, as it gives the impression that non-troll, non-dickweed users might comprise close to 10% of the users here, rather than the 0.01% that is more accurate.Note, too, that the "or" is not an exclusive or; many users are both.

    @mott555 said:

    Keep posting WTFs and ignore the negative responses. I enjoyed the WTF.
    Absolutely!

     



  • @HardwareGeek said:

    @mott555 said:
    At least 90% of the users here are either trolls or aspie pedantic dickweeds.
    While technically correct, this is misleading, as it gives the impression that non-troll, non-dickweed users might comprise close to 10% of the users here, rather than the 0.01% that is more accurate.Note, too, that the "or" is not an exclusive or; many users are both.
     

    I DISPUTE YOUR MATH FOR STUPID REASONS!

    @HardwareGeek said:

    @mott555 said:
    Keep posting WTFs and ignore the negative responses. I enjoyed the WTF.
    Absolutely!

    Yes, keep posting. If we don't have inane details to argue about, we might accidentally revert to civil conversation. If that happens, the Great Underbelly of CS will have no more angry souls to consume. It will stretch its tentacles out and through the Internet, causing much cat-stabbings and porn-erasings. THINK OF THE KITTY PORN!


  • Winner of the 2016 Presidential Election

    @Lorne Kates said:

    THINK OF THE KITTY PORN!

    Ben L. always does.



  • @Darsithis said:

    I was told to post it here by another developer who wondered why I wasn't posting most of the incredibly stupid stuff I've found in this project but honestly given the reaction from it I don't think I'll be posting anything again.

    Really unfortunate because I've always liked the funny dev stuff my coworkers have linked from here.

    Dear Diary,
    
    Today on TheDailyWtf forum I posted a terrible piece of shit I stumbled upon at work. I was expecting people to
    applause and send me Facebook Friend requests but for some reasons the thread went to shit and some people were
    really mean to me.
    
    Because I am so special it should not happen that people disagree with me or make nasty comments. I deserve 
    rainbows and unicorns. So before leaving this place forever and ever and ever I will write down those words who
    will keep me company as I cry myself to sleep.
    
    There's a hero
    If you look inside your heart
    You don't have to be afraid
    Of what you are
    There's an answer
    If you reach into your soul
    And the sorrow that you know
    Will melt away
    
    
    -Darsithis X0X
    
    

    DTFY.



  • @Ronald said:

    @Darsithis said:

    I was told to post it here by another developer who wondered why I wasn't posting most of the incredibly stupid stuff I've found in this project but honestly given the reaction from it I don't think I'll be posting anything again.

    Really unfortunate because I've always liked the funny dev stuff my coworkers have linked from here.

    Dear Diary,
    

    Today on TheDailyWtf forum I posted a terrible piece of shit I stumbled upon at work. I was expecting people to
    applause and send me Facebook Friend requests but for some reasons the thread went to shit and some people were
    really mean to me.

    Because I am so special it should not happen that people disagree with me or make nasty comments. I deserve
    rainbows and unicorns. So before leaving this place forever and ever and ever I will write down those words who
    will keep me company as I cry myself to sleep.

    There's a hero
    If you look inside your heart
    You don't have to be afraid
    Of what you are
    There's an answer
    If you reach into your soul
    And the sorrow that you know
    Will melt away

    -Darsithis X0X

    DTFY.

     

    Yes, that was exactly my entry today!

     In the meantime maybe I should apply to moderate here as well...

     



  • @Darsithis said:

    In the meantime maybe I should apply to moderate here as well...

    You are "moderately" funny so you would be perfect. But first you need an avatar so people can visually identify your posts and skip over them as they page-down their way in the thread. Why not use a picture of yourself? As an example here is a picture of Ben L. as he is installing a new Bittorrent server so his mother can download entire seasons of Santa Barbara without blocking the family iPad.





  • @Ronald said:

    @Darsithis said:

    In the meantime maybe I should apply to moderate here as well...

    You are "moderately" funny so you would be perfect. But first you need an avatar so people can visually identify your posts and skip over them as they page-down their way in the thread. Why not use a picture of yourself? As an example here is a picture of Ben L. as he is installing a new Bittorrent server so his mother can download entire seasons of Santa Barbara without blocking the family iPad.



    Santa Barbara? Really? You couldn't come up with a show even slightly more recent? Fuck, I'm pretty sure I'm the only person here other than you who actually remembers that show. And that's just because a) I grew up in a place that got shows a decade after they air in the US and b) I have a weird memory for trivia.

     



  • @Snooder said:

    @Ronald said:

    @Darsithis said:

    In the meantime maybe I should apply to moderate here as well...

    You are "moderately" funny so you would be perfect. But first you need an avatar so people can visually identify your posts and skip over them as they page-down their way in the thread. Why not use a picture of yourself? As an example here is a picture of Ben L. as he is installing a new Bittorrent server so his mother can download entire seasons of Santa Barbara without blocking the family iPad.



    Santa Barbara? Really? You couldn't come up with a show even slightly more recent? Fuck, I'm pretty sure I'm the only person here other than you who actually remembers that show. And that's just because a) I grew up in a place that got shows a decade after they air in the US and b) I have a weird memory for trivia.

     

    The plot from Santa Barbara is very straightforward, that's why it was such a huge success.

    @IMDB said:


    Santa Barbara revolves around the shock waves that ensue from Sophia Capwell's child by Lionel Lockridge, Channing, Jr. passed off as the second son of her husband, C.C. Capwell. Channing impregnates the maid's daughter, Santana, and as Sophia argues with her son about it, she accidentally shoots him to death. C.C. is furious at the loss of his son. Sophia flees the country, eventually returning disguised as a man, with the aid of daughter Kelly. In the meantime, C.C. wants his new grandson, Channing's son by Santana, and to get custody of him, Santana is institutionalized. C.C. remarries that vixen too endearing to be a villain, Robin Mattson's Gina, and they adopt Brandon, Santana's baby by the late Channing. Santana tries to reclaim her son, but winds up with visitation rights. Brandon is happy with Gina as his mother, with his father being the man who was his grandmother's husband, but biologically not related to him at all. Gina and C.C. wind up divorcing, Gina remarries Mason, C.C.'s son by first wife Pamela. That marriage does not last, either. Sophia and C.C. reunite, but their remarriage does not last one year before they separate again. The enormity of Sophia's infidelity and the true paternity of Channing serve as the pretext for the disappearance of her most famous offspring, daughter Eden. While the extraordinarily popular actress who portrayed Eden tried other roles in other productions, her absence was explained by her insanity at not being able to cope with the discovery of Channing's true parentage.

    Unfortunately this plot summary does not put the emphasis on the real hero of the series: Cruz Castillo (Eden's soulmate). Cruz was a troubled, profoundly damaged man and the depth of his pathos has put Eden through a huge emotional roller coaster. Also the actor who played Cruz Castillo has the coolest stage name ever.



  • @Ronald said:

    Anyone else notice the cat?



  • @bgodot said:

    Anyone else notice the cat?
    No. I was trying to avoid looking at the picture.



  • @Ronald said:

    @Snooder said:
    @Ronald said:
    @Darsithis said:
    In the meantime maybe I should apply to moderate here as well...

    You are "moderately" funny so you would be perfect. But first you need an avatar so people can visually identify your posts and skip over them as they page-down their way in the thread. Why not use a picture of yourself? As an example here is a picture of Ben L. as he is installing a new Bittorrent server so his mother can download entire seasons of Santa Barbara without blocking the family iPad.


    Santa Barbara? Really? You couldn't come up with a show even slightly more recent? Fuck, I'm pretty sure I'm the only person here other than you who actually remembers that show. And that's just because a) I grew up in a place that got shows a decade after they air in the US and b) I have a weird memory for trivia.
    General Hospital is much better.  They have RELISH!

     



  • @HardwareGeek said:

    @bgodot said:

    Anyone else notice the cat?
    No. I was trying to avoid looking at the picture.

    Why don't you avoid looking at this one instead?







    This could be someone we all know and despise in 25 years if he does not give up Cheetos and Go programming.



  • @El_Heffe said:

    General Hospital is much better.  They have RELISH!

     

    Every single actor in these scenes deserves an Oscar for putting up with the awful script.



  • @Ronald said:

    @HardwareGeek said:

    @bgodot said:

    Anyone else notice the cat?
    No. I was trying to avoid looking at the picture.

    Why don't you avoid looking at this one instead?



    <img redacted>



    This could be someone we all know and despise in 25 years if he does not give up Cheetos and Go programming.

    Oh, my eyes! It burns!



  • @HardwareGeek said:

    @Ronald said:
    @HardwareGeek said:

    @bgodot said:

    Anyone else notice the cat?
    No. I was trying to avoid looking at the picture.

    Why don't you avoid looking at this one instead?



    <img redacted>



    This could be someone we all know and despise in 25 years if he does not give up Cheetos and Go programming.

    Oh, my eyes! It burns!

    I wonder how exactly he would proceed if he wanted to rub one out. Maybe he could set his keyboard sideways to jack up his gut; that way he could reach his dick while having one hand available to control the mouse, and he could also type blindly on the keyboard if needed.



  • @Ronald said:

     

    OH GOD MEMORIES

     

    ... BAD, AWFUL MEMORIES

    It's one of those realizations that even as a small child you suddenly understand that the world is full of utter shit and low quality productions.

     

    I loved Amazing Discoveries though. At that age I honestly believed it was a fun comedy show.



  • @HardwareGeek said:

    I need brain bleach
     

    Tip: brain bleach is actually regular bleach. Works wonders. Huff that shit. Huff it.



  • @Ronald said:

    Also the actor who played Cruz Castillo has the coolest stage name ever.
     

    No, [url="http://www.imdb.com/name/nm1564024/?ref_=fn_al_nm_1"]he doesn't.[/url]



  • @Lorne Kates said:

    @Ronald said:
    Also the actor who played Cruz Castillo has the coolest stage name ever.
     

    No, he doesn't.

    Even better if it's a real name, not a stage name

     



  • @El_Heffe said:

    @Lorne Kates said:

    @Ronald said:
    Also the actor who played Cruz Castillo has the coolest stage name ever.
     

    No, he doesn't.

    Even better if it's a real name, not a stage name

     

    Even better, not a real cartoon character name:





  • @Ronald said:

    @El_Heffe said:

    @Lorne Kates said:

    @Ronald said:
    Also the actor who played Cruz Castillo has the coolest stage name ever.
     

    No, he doesn't.

    Even better if it's a real name, not a stage name

     

    Even better, not a real cartoon character name:



    Max Power

     Also appearing in the same episode of Little House on the Prarie with Max Power was Rick Hurst.  According to his IMDB bio, he graduated from Tulane University with a Bachelor's Degree in
    Psychology and Theatre and graduated from Temple University with a
    Master of Fine Arts Degree in Acting.  His only major acting credit seems to be appearing in 47 episodes of The Dukes of Hazzard from 1979-1982 as "Cletus".  Glad to see he put that Masters Degree to good use.



  • @El_Heffe said:

    Also appearing in the same episode of Little House on the Prarie with Max Power was Rick Hurst.  According to his IMDB bio, he graduated from Tulane University with a Bachelor's Degree in
    Psychology and Theatre and graduated from Temple University with a
    Master of Fine Arts Degree in Acting.  His only major acting credit seems to be appearing in 47 episodes of The Dukes of Hazzard from 1979-1982 as "Cletus".  Glad to see he put that Masters Degree to good use.

    The guy played in Steel Vaginas Magnolias. Show some respect.


Log in to reply
 

Looks like your connection to What the Daily WTF? was lost, please wait while we try to reconnect.