Ye' Cannae Break Th' Laws o' Physics!!



  • I'm subcontracting doing some design work for a client, working on part of an overall system that basically images data and shoves it in a database. A company man is doing the actual imaging part, sending the data over Ethernet to an FTP server and dropping it onto a hard drive in a known location. My end of the work involves grabbing the data off the hard drive and shoving it into the DB. Nothing too hard about this work and I have done similar projects with them in the past.

    Except today I realized that in this particular case they want to push the data down the Ethernet at a rate of over 15MegaBytes per Second and the Ethernet is no better than 100Base-T which has a practical data rate of about 12.3MegaBytes per Second. Ooops



  •  So just use twice the Ethernets.  Duh.



  • @bstorer said:

     So just use twice the Ethernets.  Duh.

    I'd recommend 3 at the very least. But I think in their quote to company that will actually be buying this particular piece of work, they only budgeted for one Ethernets worth of data transfer and the imaging bit on the front end doesn't come at a cheap price. The other D'Oh part is that they have been working on their design doc for over a month - yet after I see it for 1 day I'm spotting issues in it.



  • Eh. Gigabit's cheap enough of an upgrade these days.



  • @OzPeter said:

    The other D'Oh part is that they have been working on their design doc for over a month - yet after I see it for 1 day I'm spotting issues in it.
    To be fair, most of that month has been used to form the committee to recommend a font for the design document.  Once the subcomittee on serif vs. sans serif has their initial findings back, things should move forward pretty smoothly.



  • Is that the throughput rate for the uncompressed data? If so, all you have to do is put it in an inefficient format and specify Gzip in the HTTP headers.



  • @fennec said:

    Eh. Gigabit's cheap enough of an upgrade these days.
    Sure we can put in gigabit networks, but if the key device on the network only does 100Base-T then Gigabit-E physical layer doesn't do squat for us.

    The problem is that we have a single hardware device that requires a data bandwidth budget that exceeds the network interface built into that device. A smart design would have seen this on day one and designed the system around having 2 or 3 of these devices as the problem is actually suitable for a parallel data transfer solution in the physical world. Of course that would have required some intelligence.



  • @bstorer said:

    @OzPeter said:

    The other D'Oh part is that they have been working on their design doc for over a month - yet after I see it for 1 day I'm spotting issues in it.
    To be fair, most of that month has been used to form the committee to recommend a font for the design document.  Once the subcomittee on serif vs. sans serif has their initial findings back, things should move forward pretty smoothly.

    First of all this is a great time to mention the documentary I saw last week on Helvetica Yes, a documentary about a font! But I found it surprisingly interesting as it was also a good documentary on differing schools of design and how the desire for Helvetica arose

    But actually you are wrong about the time frame. You see the basic job was proposed by a salesman who thinks he understands technology, but doesn't even understand the products he tries to sell. The (lack of) engineering design was performed by him and another technical sales person who also doesn't understand basic networking but is at least humble enough to accept his own screw ups. Between the two of them it was a case of the blind leading the blind. Also I am a contractor and they pay me by the hour, and they were also too cheap to bring me in earlier to spec out my part of the job. Basically they got the crap design that they deserved. I can't wait until Monday to see how this plays out. The basic problem is solvable by throwing more hardware at it. But that means a lot more money to spend, and I am guessing that their overall budget has already been set by the quote that they gave to the final customer - and they tend to cheap out on quotes for some reason; desperate to get work perhaps?

    Another reason I am waiting for Monday is that I have an interview with a medium sized engineering firm that pays market rates for their staff, pays full benefits, pays a yearly bonus and pays overtime! The down side is that I may have to work with the tobacco industry - and cigarette smoke makes me feel sick



  • @Seahen said:

    Is that the throughput rate for the uncompressed data? If so, all you have to do is put it in an inefficient format and specify Gzip in the HTTP headers.
    Well its not a web device. It is actually a smart camera squirting out BMP images. It does have the ability to convert images from BMP to JPG in situ, but I am not sure if the time taken to do the conversion would actually give enough time left over to do the data transmission within the remaining time budget. I haven't seen a sample image so I don't know how well it will compress, but I do know that it is imaging printed text on a complex background - so it may not compress very easily. In addition there is network latency and contention to deal with. Basically they have less than 20mS to establish an FTP session to a local computer and squirt the image out, followed by another FTP session to send a small text file. My feeling is that even if they get the physical data to send below the 12MB/S limit of 100Base-T, the overall latencies of the system will keep practical data rates down below 6MB/S - and there is no way they can do that with a single device and still keep the image throughput above the minimum requirements.



  • @OzPeter said:

    @Seahen said:
    Is that the throughput rate for the uncompressed data? If so, all you have to do is put it in an inefficient format and specify Gzip in the HTTP headers.
    Well its not a web device. It is actually a smart camera squirting out BMP images. It does have the ability to convert images from BMP to JPG in situ, but I am not sure if the time taken to do the conversion would actually give enough time left over to do the data transmission within the remaining time budget. I haven't seen a sample image so I don't know how well it will compress, but I do know that it is imaging printed text on a complex background - so it may not compress very easily. In addition there is network latency and contention to deal with. Basically they have less than 20mS to establish an FTP session to a local computer and squirt the image out, followed by another FTP session to send a small text file. My feeling is that even if they get the physical data to send below the 12MB/S limit of 100Base-T, the overall latencies of the system will keep practical data rates down below 6MB/S - and there is no way they can do that with a single device and still keep the image throughput above the minimum requirements.
     

     

    hey i once met a client who also had not done the basic maths to do a similar job.  It was to send image captures of all people on a roller coaster ready to view and print at a kiosk by the end of the ride.



  • @Helix said:

    hey i once met a client who also had not done the basic maths to do a similar job.  It was to send image captures of all people on a roller coaster ready to view and print at a kiosk by the end of the ride.
    Really?  We're talking, what, groups of 12 images at 2 minute intervals (at least)?  Were they transmitted the images smoke signals to be processed by a chimp with an abacus?  What's the bottleneck I'm missing here?



  • @bstorer said:

    What's the bottleneck I'm missing here?
     

    The only thing I can think of is you'll need a proper printer at the other side that can print a picture in less than 10 second. That appears to be pretty fast, but it can be done.



  • @bstorer said:

    To be fair, most of that month has been used to form the committee to recommend a font for the design document.  Once the subcomittee on serif vs. sans serif has their initial findings back, things should move forward pretty smoothly.

    I co-chaired the committee that reviewed the recommendation to revise the color of the book that regulation's in.

    We kept it grey.



  • @b_redeker said:

    @bstorer said:

    What's the bottleneck I'm missing here?
     

    The only thing I can think of is you'll need a proper printer at the other side that can print a picture in less than 10 second. That appears to be pretty fast, but it can be done.

    But you don't have to print all the pictures, just the ones people buy.  Typically they just roll through them on a couple monitors at then end of the ride.


  • @bstorer said:

    Typically they just roll through them on a couple monitors at then end of the ride.
     

    Er, yes, indeed. So maybe they have really slow monitors?



  • @b_redeker said:

    @bstorer said:

    Typically they just roll through them on a couple monitors at then end of the ride.
     

    Er, yes, indeed. So maybe they have really slow monitors?

    Perhaps they've decided monitors are too expensive so they're just using the caricature artists' free time?


  • bstorer, why does your signature contain Nazi propaganda? Is it funny? Irony? Sarcasm? Ignorance? Admiration of the graphical design? Take it out, please.

     

    mod: toned down hysterical font size. -dh



  • @bstorer said:

    Perhaps they've decided monitors are too expensive so they're just using the caricature artists' free time?
     

    Most likely the same artists provided the data transfer using the internal Sneaker Network.



  • @TGV said:

    bstorer, why does your signature contain Nazi propaganda? Is it funny? Irony? Sarcasm? Ignorance? Admiration of the graphical design? Take it out, please.

    TGV, why does your post use <font>? It's deprecated and horrible, in fact there's a place in hell reserved for people who still use it, rumor has it that Nazis actually get off lighter. As for why he has it, he disagrees with the new administration, AmmoQ was a good an decent admin, but now dhromed is one too? We can't just stand idly by while fascists take over this forum! I'm ashamed that you've committed so many hate crimes in one post, not only do you blatantly support the Reds, you mark yourself as a sinner of the worst kind. Your ignorance and vileness sickens me to the core.
    @TGV said:
    mod: toned down hysterical font size. -dh

    Note: The original post that I was quoting had <font size="+3"> or some such deplorable use of HTML. Just a friendly reminder kids, using <font> is quite a bit worse than many other things, I'd mention them but I'd rather not have people go batshit over a semi-sarcastic comment -- Ling



    PS: <font> is still evil though. Avoid it with the same frequency that you'd avoid a skinhead or a neo-nazi. Thanks.



  • @Lingerance said:

    As for why he has it, he disagrees with the new administration, AmmoQ was a good an decent admin, but now dhromed is one too? We can't just stand idly by while fascists take over this forum!
     

    oh i am SO having you edited out of the group photos we took on that picknick.

    Photoshop CS5 has a feature for that, actually.



  • @dhromed said:

    oh i am SO having you edited out of the group photos we took on that picknick.

    Photoshop CS5 has a feature for that, actually.

    Which the Gimp people will tell you that they already had that as a feature well before CS5 was published. But I'm still considering to upgrade to CS5 rather than install the Gimp.



  • @TGV said:

    bstorer, why does your signature contain Nazi propaganda?
    It's a joking protest against dhromed's recent promotion to demigod.  At first I accused him of being socialist, but he disputed that claim, obviously making him a Nazi.@TGV said:
    Is it funny? Irony? Sarcasm?
    Yes, pretty much.  Plus, the addition of the "We Can Do It" using morb's avatar makes it the logical conclusion of all of his anti-Europe rhetorical.@TGV said:
    Ignorance?
    Oh, yeah, totally.  I was under the impression that it's a poster promoting hugs.  Was I wrong?@TGV said:
    Admiration of the graphical design?
    Actually, yes.  Look at the original; message aside, it's fucking beautiful. @TGV said:
    Take it out, please.
    No.  I don't think you get any say in the matter.  I might take it out if dhromed asked me to, since it's his avatar on display.

    Other than accusing dhromed of Nazism, I really don't believe this particular poster is excessively offensive.  In my opinion, the fact that it has Nazi origins really isn't enough.   It doesn't denegrate Jews or any other racial or ethnic group.  Yeah, it does oppose the Bolsheviks, but I find it really hard to take a side in a war of extremists.



  • @OzPeter said:

    @dhromed said:
    oh i am SO having you edited out of the group photos we took on that picknick.

    Photoshop CS5 has a feature for that, actually.

    Which the Gimp people will tell you that they already had that as a feature well before CS5 was published. But I'm still considering to upgrade to CS5 rather than install the Gimp.
    I use Gimp, and it's fine for what I occassionally need, which can be defined as "more than MS Paint offers" (or "putting dhromed's avatar into politcal posters").  It really is Photoshop's ugly, retarded little brother, and I wouldn't encourage any real graphic designer to use it, but it's plenty effective for the masses.


  • @bstorer said:

    I use Gimp, and it's fine for what I occassionally need, which can be defined as "more than MS Paint offers" (or "putting dhromed's avatar into politcal posters").  It really is Photoshop's ugly, retarded little brother, and I wouldn't encourage any real graphic designer to use it, but it's plenty effective for the masses.

    Paint.NET can do pretty much every non-pro thing GIMP can do, with the benefit of a name and UI that don't suck.



  • @blakeyrat said:

    Paint.NET can do pretty much every non-pro thing GIMP can do, with the benefit of a name and UI that don't suck.
     

    You beat me to it. Especially on the UI part. It resembles Paint Shop Pro, which is my roots, and also has a proper Windows UI.



  • @dhromed said:

    It resembles Paint Shop Pro
    I just got a little moist hard.  I never really paid Paint.NET any attention, but now I guess I should.



  • @bstorer said:

    I just got a little moist hard.  I never really paid Paint.NET any attention, but now I guess I should.
     

    PSP killed itself with that 9 or 10 upgrade, when it tried to be More Advanced and instead became More Suck. Its fucked type features were mostly what drove me to Photoshop, which seems exceptional in its completely solid type editing methods. I still believe that PSP's vector stuff and specifically all the node handling features were vastly superior to the Adobe method.

    It resembles PSP in that it has a floating palette here and there and also a common toolbar, and doesn't reinvent the UI.

    @bstorer said:

    I just got a little moist hard.

    FTFY, you bitch.



  • @OzPeter said:

    The down side is that I may have to work with the tobacco industry - and cigarette smoke makes me feel sick
    You obviously missed South Park's documentary on the tobacco industry.  It was quite eye opening.  The tobacco workers and execs are bright, cheery, lovable people and the anti-smoking activists are conniving bastards.



  • Title: Reference to Star Trek or the song Star Trekkin'?  I know the line from the song but I didn't watch a whole lot of original Star Trek.



  • When a problem comes along, you must ZIP it!

    Zip it good!



  • @huai said:

    you must ZIP it!
     

    ZIP! JAM! BIFF! POW!



  • @belgariontheking said:

    Title: Reference to Star Trek or the song Star Trekkin'? 
     

    This reminds me; I have to go see that (remix) clip again where captain Kirk is climbing the mountain.



  • Update: Ye' Cannae Break Th' Laws o' Physics!!

    Exchanged some emails this morning with the guy specifying the cameras that generate the data. They only had enough in the budget for one of the cameras, so they can't parallelize the problem due to finances. But there still seems to be pesky customer some requirements that require the sending out and responding to the data really fast. Converting BMPs to JPGs is their only option, but I have no idea what will be the I/O response time of the cheap piece of Dell trash they will end up specifying to collect the data on the other end. The Dell will end up controlling how fast the FTP session is created and hence how fast the data can be produced/consumed.



  • @bstorer said:

    Yeah, it does oppose the Bolsheviks, but I find it really hard to take a side in a war of extremists.

    So because you cannot take sides in a war between Nazis and Communists, it's ok to use their propaganda? Here's a reminder of what happened as a consequence.





  • TGV, you might want to take it here: [url]http://forums.thedailywtf.com/forums/t/18021.aspx[/url]



  • @huai said:

    When a problem comes along, you must ZIP it!

    Zip it good!

    Actually, the saying in these parts is JAM IT!

Log in to reply