The Pipe



  • We have 64 heavy duty production servers, each with 16-3.2GHz cores, each of which is running a thread of our application to crunch data. Some threads read, some crunch data and some write results. In production, for each core there is an I/O "pipe" through which I/O to the network (and by extension the databases on the SANs) is performed.

    We also have 12 light duty demonstration environment servers, each with 16-2.1GHz cores, where our application also runs. When it's being used by one user to demonstrate some screen, it runs smoothly. However, the sales drone tried to show how well our software could perform and tried to do a load test in that environment.

    He ran the scripts to launch 1024 instances of our application on these 12 servers. They groaned, but eventualy got everything running. Then he tried to impose a load by kicking off a large job which would spawn off load that would grind each of the application instances. The CPU load dropped to near zero across the entire environment. The network load in no way increased. The SANs were very lightly loaded.

    The customer raised an eyebrow. The drone called us. We called the SA's who ran their monitors. It turns out there is only one I/O "pipe" for this entire environment (co$t); it was like 1024 lanes of traffic merging into a single lane tunnel; everything was blocked waiting on I/O.

    The drone was told you can't do that in this environment. "But we can do it in production!" Yes, but you have a lot more hardware there. "So why don't we have the same hardware here?" Sure, no problem, you get the purchase order approved and we'll buy it and set it up. "So, how long do you think it will take this operation to complete?"

    Apparently the customer had more of a head for I/O than the drone and bit his tongue to keep from getting hysterical.
     



  • Sounds like the salesman is operating on a single lane, gravel road as well.



  • Wait a minut. You called SA, and they correctly identified the problem the first time you talked to them? When did you change job???



  • What is this IO pipe you speak of?


  • Considered Harmful

    So they connected Lake Mead to the Hoover Dam via a drinking straw.



  • @russ0519 said:

    What is this IO pipe you speak of?

     I'm not sure, but I'll they smoke some mighty good dope in it.



  •   "So, how long do you think it will take this operation to complete?"

    I think, that right answer goes along this line: "We can set-up it in 5 days, our suplier can deliver such amounth of servers in 2 months, so HOW LONG IT WILL TAKE YOU TO  GET PURCHASE ORDER APPROVED to finish this operation?



  • @joe.edwards said:

    So they connected Lake Mead to the Hoover Dam via a drinking straw.

    More like the Atlantic to Pacific with a stir stick.



  • @Master Chief said:

    @joe.edwards said:
    So they connected Lake Mead to the Hoover Dam via a drinking straw.

    More like the Atlantic to Pacific with a stir stick.

    More like a few teaspoons a few cm with dhromed's urethra.



  • The sales drone tried to demonstrate the principles behing the Panama Canal with two swimming pools and a drinking straw, and then phoned sysadmins to ask why he couldn't sail his yacht down it.


  • Garbage Person

    @snoofle said:

    It turns out there is only one I/O "pipe" for this entire environment (co$t); it was like 1024 lanes of traffic merging into a single lane tunnel; everything was blocked waiting on I/O.
    So... Why have a demo environment that outstrips its I/O?

     

    In awful automotive metaphors, it's like trying to swap a big block into a Geo Metro without changing the transmission. If you really want it hard enough, you can make it physically fit - and it'll start up and look really cool... And then blow up as soon as you shift into gear.



  • @Weng said:

    In awful automotive metaphors, it's like trying to swap a big block into a Geo Metro without changing the transmission. If you really want it hard enough, you can make it physically fit - and it'll start up and look really cool... And then blow up as soon as you shift into gear.
    1. Automobile, not automotive.
    2. Ambiguity: hyphen needed in 'awful automotive metaphors'.
    3. A Geo Metro is transverse engined. A big block won't fit transversely, whether you change the trans or not.
    4. The transmission in a Metro is tougher than you think, anyway.
    @Weng said:
    So... Why have a demo environment that outstrips its I/O?
    Because a billionaire is trolling Snoofle in real life?



  • @TDWTF123 said:

    The transmission in a Metro is tougher than you think, anyway.

    [citation needed] Please provide evidence that it won't get destroyed by a big block.



  • "heavy duty production servers, each with 16-3.2GHz cores"

    Um, no. With 16 cores, they are no heavy duty. I worked for a while at ((large financial information provider)), and we had production servers with 192 cores.


  • Garbage Person

    @TDWTF123 said:

    A Geo Metro is transverse engined. A big block won't fit transversely, whether you change the trans or not.
    Son, in my crowd of deranged car enthusiasts, that ain't no problem. It'd require entirely too much attention with the sawzall and probably a variety of strange, exotic modified parts, but I've seen a cohort put a world war 2 radial engine in an MR2... By way of parts from a Mitsubishi 3000GT, Honda Goldwing, some Subaru or another, and diesel truck parts. The driveline had to turn 180 degrees and reverse direction. In the case of a big block, we really only need to turn 90 degrees at most. No big deal.


  • Discourse touched me in a no-no place

    @Steve The Cynic said:

    With 16 cores, they are no heavy duty. I worked for a while at ((large financial information provider)), and we had production servers with 192 cores.
    True; 16 cores is small potatoes. I've seen systems get up to 1024 cores with a shared memory backplane and hardly anything is built like that now. I suspect that such backplanes are still eye-wateringly expensive, and I've got a vague feeling that the cost varies with the square of the number of nodes (if not a higher power). They certainly don't build supercomputers like that any more; it's just too expensive, and usually too hard to program for efficiently.



  • @gilhad said:

    "We can set-up it in 5 days, our suplier can deliver such amounth of servers in 2 months, so HOW LONG IT WILL TAKE YOU TO  GET PURCHASE ORDER APPROVED to finish this operation?"
     

    +1. Trying to make people understand it's dependent upon actions they themselves have responsibility for is frustrating and futile (when reading the rest of CFH).

    @Steve The Cynic said:

    "heavy duty production servers, each with 16-3.2GHz cores"

    Um, no. With 16 cores, they are no heavy duty. I worked for a while at
    ((large financial information provider)), and we had production servers
    with 192 cores.


    I take it that meant 16 x 3.2GHz ...? I originally read it as 1.6-3.2GHz.

    Heavy-duty is all subjective. Snoofle's production environment is just super-heavy, yours is ultra-heavy.



  • @Steve The Cynic said:

    "heavy duty production servers, each with 16-3.2GHz cores"

    Um, no. With 16 cores, they are no heavy duty. I worked for a while at ((large financial information provider)), and we had production servers with 192 cores.

    Me too, but compared to the other junk around here, they are (by comparison) heavy duty.



  • @morbiuswilters said:

    @TDWTF123 said:
    The transmission in a Metro is tougher than you think, anyway.

    [citation needed] Please provide evidence that it won't get destroyed by a big block.

    It will eventually, but if it's the one I think it is it's a Suzuki box that was used in lots of other applications with way more power. Am I completely confused? Geo Metro is your name for the Suzuki Swift, isn't it?
    @Weng said:
    Son, in my crowd of deranged car enthusiasts, that ain't no problem.
    Fair point, but the fact that there isn't a big-block swapped Metro to be found on the internet suggests that even amongst loonies that's considered a bit problematic...



  • @TDWTF123 said:

    1. Ambiguity: hyphen needed in 'awful automotive metaphors'.

    A hyphen would be incorrect, but a comma would do nicely.



  • Sounds like someone has just issued a challenge.



  • @Sarcarsm said:

    Sounds like someone has just issued a challenge.

    HAVE AT YOU!
    slaps your face with leather glove
    You, sir, are a bounder, a cad, and a filthy blackguard, and I demand my satisfaction.



  • @Steve The Cynic said:

    "heavy duty production servers, each with 16-3.2GHz cores"

    Um, no. With 16 cores, they are no heavy duty. I worked for a while at ((large financial information provider)), and we had production servers with 192 cores.

    Well, if it was a single 16-core machine, I'd probably agree with you, but he's running 64 of them, so that's actually a pretty decent amount of power. Regardless, nobody is impressed with your proxy e-penis.



  • @Weng said:

    ..but I've seen a cohort put a world war 2 radial engine in an MR2..

    Oh shit, I just spontaneously ejaculated into my shorts.



  • @eViLegion said:

    @TDWTF123 said:
    1. Ambiguity: hyphen needed in 'awful automotive metaphors'.

    A hyphen would be incorrect, but a comma would do nicely.

    A hyphen would be correct: a hyphen is used to link two (or more) words to make it clear which is being modified. Two commas would also do it. One would be OK as a breath-marker, but meaningless grammatically.



  • @TDWTF123 said:

    @eViLegion said:
    @TDWTF123 said:
    1. Ambiguity: hyphen needed in 'awful automotive metaphors'.

    A hyphen would be incorrect, but a comma would do nicely.

    A hyphen would be correct: a hyphen is used to link two (or more) words to make it clear which is being modified. Two commas would also do it. One would be OK as a breath-marker, but meaningless grammatically.

    The way it is written it needs neither - there are two adjectives and one noun so it is obvious which one is being modified (the noun). If, as someone else suggested, it was rewritten as "awful automobile metaphors" (two nouns and one adjective) then it would need a hyphen so the reader could work out whether it was the automobile or the metaphors which were awful.



  • @snoofle said:

    "So, how long do you think it will take this operation to complete?"

    The right answer would be: "until someone decides to cancel it".



  • @LurkNoMore said:

    there are two adjectives and one noun so it is obvious which one is being modified (the noun)
    Round objects. It is perfectly possible for one adjective to modify another. @LurkNoMore said:
    If, as someone else suggested, it was rewritten as "awful automobile metaphors"
    That was me as well. And it's required, unless the intention is to imply that the metaphor was self-propelled.



  • @TDWTF123 said:

    @LurkNoMore said:
    there are two adjectives and one noun so it is obvious which one is being modified (the noun)
    Round objects. It is perfectly possible for one adjective to modify another. @LurkNoMore said:
    If, as someone else suggested, it was rewritten as "awful automobile metaphors"
    That was me as well. And it's required, unless the intention is to imply that the metaphor was self-propelled.

    "I'm going to show everyone how clever I am by writing several posts pretending I can't understand a simple phrase."



  • @morbiuswilters said:

    @TDWTF123 said:
    @LurkNoMore said:
    there are two adjectives and one noun so it is obvious which one is being modified (the noun)
    Round objects. It is perfectly possible for one adjective to modify another. @LurkNoMore said:
    If, as someone else suggested, it was rewritten as "awful automobile metaphors"
    That was me as well. And it's required, unless the intention is to imply that the metaphor was self-propelled.

    "I'm going to show everyone how clever I am by writing several posts pretending I can't understand a simple phrase."

    "I'm going to show everyone how stereotypically American I am by making loads of posts about how guns are great and then another in which I prove myself totally oblivious to irony."



  • @TDWTF123 said:

    @morbiuswilters said:
    @TDWTF123 said:
    @LurkNoMore said:
    there are two adjectives and one noun so it is obvious which one is being modified (the noun)
    Round objects. It is perfectly possible for one adjective to modify another. @LurkNoMore said:
    If, as someone else suggested, it was rewritten as "awful automobile metaphors"
    That was me as well. And it's required, unless the intention is to imply that the metaphor was self-propelled.

    "I'm going to show everyone how clever I am by writing several posts pretending I can't understand a simple phrase."

    "I'm going to show everyone how stereotypically American I am by making loads of posts about how guns are great and then another in which I prove myself totally oblivious to irony."

    Once I saw a Belgian movie while I was high on a mix of coke and sawdust, and the scenario made more sense than the discussion in this thread.



  • @TDWTF123 said:

    "I'm going to show everyone how stereotypically American I am by making loads of posts about how guns are great and then another in which I prove myself totally oblivious to irony."

    So you're claiming that you repeatedly harping on someone's grammar is ironic? Please explain.



  • @morbiuswilters said:

    So you're claiming that you repeatedly harping on someone's grammar is ironic? Please explain.
    If you thought I was being serious about it at any point, that would explain your confusion. I wasn't actually grammar naziing, I was ragging on the Metro by suggesting that 'awful' might refer to the car instead of the metaphor. Someone then turned up and actually started grammar-pedanting, so I was poking him with a stick. I understood that to be the culturally correct thing to do around here under those circumstances.
    @morbiuswilters said:
    You have til the count of three. Then I'm shooting you in the face.
    Stereotypical American, remember? You'll shoot someone somewhere, but the chances are it'll be friendly fire or yourself in the foot. Wait, can you even see over that huge gut to hit your own foot?



  • @TDWTF123 said:

    Someone then turned up and actually started grammar-pedanting, so I was poking him with a stick. I understood that to be the culturally correct thing to do around here under those circumstances.

    Hmm.. I see.

    @TDWTF123 said:

    You'll shoot someone somewhere, but the chances are it'll be friendly fire or yourself in the foot.

    Hey, that's not fair, we've produced lots of accurate marksmen. Oswald comes to mind..

    @TDWTF123 said:

    Wait, can you even see over that huge gut to hit your own foot?

    ...yes, Mr. Smarty-Pants. What I can't see over are, uh, my massively-engorged testes. Yeah, that's it.





  • @dkf said:

    @Steve The Cynic said:
    With 16 cores, they are no heavy duty. I worked for a while at ((large financial information provider)), and we had production servers with 192 cores.
    True; 16 cores is small potatoes. I've seen systems get up to 1024 cores with a shared memory backplane and hardly anything is built like that now. I suspect that such backplanes are still eye-wateringly expensive, and I've got a vague feeling that the cost varies with the square of the number of nodes (if not a higher power). They certainly don't build supercomputers like that any more; it's just too expensive, and usually too hard to program for efficiently.

    Don't feel bad if you think supercomputers are a thing of the past. See http://en.wikipedia.org/wiki/TOP500 and http://en.wikipedia.org/wiki/Blue_Gene for inspiration. The Titan listed in the TOP500 has a peak of 27.1 PETA flops per second.


  • Considered Harmful

    Isn't FLOPS per second a rate of acceleration?



  • @joe.edwards said:

    Isn't FLOPS per second a rate of acceleration?

    FLoating point OPerationS per second?


  • Considered Harmful

    @Ben L. said:

    @joe.edwards said:
    Isn't FLOPS per second a rate of acceleration?

    FLoating point OPerationS per second?

    Nice try. FLoating point Operations Per Second.

    I guess FPOPS doesn't roll off the tongue as easily.



  • @joe.edwards said:

    @Ben L. said:
    @joe.edwards said:
    Isn't FLOPS per second a rate of acceleration?

    FLoating point OPerationS per second?

    Nice try. FLoating point Operations Per Second.

    I guess FPOPS doesn't roll off the tongue as easily.

    Why not call them FOPs? We already have NOPs...


  • @joe.edwards said:

    I guess FPOPS doesn't roll off the tongue as easily.

    I don't know about fpops, but I know you had some fops rolling around on your tongue last night.



  • @Ben L. said:

    Why not call them FOPs?

    Damn you, why did you steal my joke?



  • @morbiuswilters said:

    @joe.edwards said:
    I guess FPOPS doesn't roll off the tongue as easily.

    I don't know about fpops, but I know you had some fops rolling around on your tongue last night.

    Where are these sex parties and why am I not invited?



  • @Ben L. said:

    Filed under: Right?, fops means vaginas

    fop noun A dandy! A poof! A bugger! An Englishman!



  • @morbiuswilters said:

    fop noun A dandy! A poof! A bugger! An Englishman!

    Why the fop is your dictionary so excited?!



  • @DrPepper said:

    27.1 PETA flops per second

    yielding 27.1 ROLFs per second squared.



  • @Weng said:

    @TDWTF123 said:

    A Geo Metro is transverse engined. A big block won't fit transversely, whether you change the trans or not.
    Son, in my crowd of deranged car enthusiasts, that ain't no problem. It'd require entirely too much attention with the sawzall and probably a variety of strange, exotic modified parts, but I've seen a cohort put a world war 2 radial engine in an MR2... By way of parts from a Mitsubishi 3000GT, Honda Goldwing, some Subaru or another, and diesel truck parts. The driveline had to turn 180 degrees and reverse direction. In the case of a big block, we really only need to turn 90 degrees at most. No big deal.





    You mean this car? That thing is awesome... the old style aviator helmet makes it work.



  • @Arnavion said:

    @morbiuswilters said:
    fop noun A dandy! A poof! A bugger! An Englishman!
    Why the fop is your dictionary so excited?!
     

    Perhaps!  He!  Looked!  It!  Up!  On!  Yahoo!

     



  • @TDWTF123 said:

    @LurkNoMore said:
    there are two adjectives and one noun so it is obvious which one is being modified (the noun)
    Round objects. It is perfectly possible for one adjective to modify another. @LurkNoMore said:
    If, as someone else suggested, it was rewritten as "awful automobile metaphors"
    That was me as well. And it's required, unless the intention is to imply that the metaphor was self-propelled.

    Automotive (adj.)

    1. relating to motor vehicles
    2. self-propelling

    The poster was using definition #1; you were using definition #2 and pretending it was the only definition. "Automotive" is perfectly acceptable here.

    And no, a comma is not required, because the word "awful" is modifying the noun phrase "automotive metaphors." Hyphenating any two of those three words would be completely wrong in this instance.


Log in to reply